在华为昇腾服务器Ascend 300I Pro 310P芯片( 310P3)安装QWQ32B大模型以及deepseek蒸馏版!
前提条件:服务器已安装docker
1.下载镜像: 1.0.0-300I-Duo-py311-openeuler24.03-lts
备注:官网镜像下载,需要申请,审批还得1,2天,这时你肯定想骂HW!没事,我已为您准备好了:请发私信!
申请地址: https://www.hiascend.com/developer/ascendhub/detail/af85b724a7e5469ebd7ea13c3439d48f

2.下载模型:魔乐社区(https://modelers.cn/models/Models_Ecosystem/QwQ-32B)
服务器上安装社区下载的比较快:
pip install modelscope
modelscope download "Qwen/QwQ-32B" --local_dir "/home/models/qwq"
注意事项:模型上传到服务器需要给于模型下config.json权限
chmod 750 config.json
3.docker 启动
注意映射的模型文件到服务器中:
docker run -it -d --net=host --shm-size=50g --privileged --name qwq-i --device=/dev/davinci_manager --device=/dev/hisi_hdc --device=/dev/devmm_svm -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro -v /usr/local/sbin:/usr/local/sbin:ro -v /home/models/qwq:/home/models/qwq:rw swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:1.0.0-300I-Duo-py311-openeuler24.03-lts
4.进入docker容器中(以下的操作全部是在docker容器中)
编辑配置文件:
注意点:
ipAddress: 本地服务器IP
httpsEnabled : false, 关闭https
modelName:模型名称
modelWeightPath:模型路径(容器内的)
npuDeviceIds:显卡ID (根据自己情况,npu-smi info 查看)
vim /usr/local/Ascend/mindie/latest/mindie-service/conf/config.json
{
"Version" : "1.1.0",
"LogConfig" :
{
"logLevel" : "Info",
"logFileSize" : 20,
"logFileNum" : 20,
"logPath" : "logs/mindservice.log"
},
"ServerConfig" :
{
"ipAddress" : "192.168.0.203",
"managementIpAddress" : "127.0.0.2",
"port" : 1025,






