1. 确认 FactChat 支持的 Model

访问 model_support.md，确认 codellama/CodeLlama-7b-Instruct-hf 在支持列表中，

2. 升级依赖

pip install -e ".[model_worker,webui]"
pip install git+https://github.com/huggingface/transformers.git@main accelerate

3. 启动 controller

python -m fastchat.serve.controller

4. 启动 CodeLlama

python -m fastchat.serve.model_worker --model-names "codellama-34b-instruct,gpt-3.5-turbo,gpt-3.5-turbo-16k,gpt-4,gpt-4-32k,text-davinci-003" --model-path codellama/CodeLlama-7b-Instruct-hf

5. 启动 api server

python -m fastchat.serve.openai_api_server --host 0.0.0.0 --port 8000

6. VSCode 中使用 CodeLlama

参考连接：

• https://continue.dev/docs/walkthroughs/codellama
• https://continue.dev/docs/customization#local-models-with-openai-compatible-server

配置 Continue 插件的 config 如下，

from continuedev.src.continuedev.libs.llm.openai import OpenAI
...
config = ContinueConfig(
    ...
    models=Models(default=OpenAI(
        api_key="EMPTY",
        model="CodeLlama-7b-Instruct-hf",
        api_base="http://localhost:8000/v1")
    ),

完结！