大模型续写评测

news2024/12/26 21:24:56

环境准备

通过remote-ssh连接云AI服务器，配置文件如下，供参考

# Read more about SSH config files: https://linux.die.net/man/5/ssh_config
Host deepseekv1
    HostName root@connect.bjc1.seetacloud.com
    User root
    Port 52984
    Password xxx # 密码从申请网址处获取

下载ollama

# 需proxy
curl -fsSL https://ollama.com/install.sh | sh
# 更改ollama模型下载位置，默认root目录下空间不足
export OLLAMA_MODELS=//root/autodl-tmp/suxikang/model/ollama/
# 启动ollama 服务默认监听端口 127.0.0.1:11434 
ollama serve
# 命令行执行模型,ollama 模型库支持deepseek-coder/deepseek-coder-v2/codestral
ollama run deepseek-coder:6.7b-base
ollama run codestral

参考:https://ollama.com/download/linux
ollama支持的模型:https://ollama.com/library

vscode安装continue插件，支持配置文件更换大模型，配置文件参考如下

{
  "models": [
    {
      "title": "deepseek-coder-v2:latest",
      "provider": "ollama",
      "model": "deepseek-coder-v2:latest"
    }
  ],
  "tabAutocompleteModel": {
    "title": "deepseek-coder-v2:latest",
    "provider": "ollama",
    "model": "deepseek-coder-v2:latest"
  }
}

参考：https://ollama.com/blog/continue-code-assistant
参考：https://youtu.be/Th2DMNgu_RQ?si=plhsmMaZSY5EoIv1

当前环境支持模型如下

(base) root@autodl-container-bf1a4f8116-ecc7efd9://root/autodl-tmp/suxikang/model/ollama/blobs# ollama list
NAME                            ID              SIZE    MODIFIED          
codestral:latest                fcc0019dcee9    12 GB   50 minutes ago   
deepseek-coder:6.7b-base        585a5cb3b219    3.8 GB  51 minutes ago   
deepseek-coder:6.7b             ce298d984115    3.8 GB  About an hour ago
deepseek-coder-v2:latest        8577f96d693e    8.9 GB  2 hours ago