基础

https://huggingface.co/OpenGVLab/InternVL2-8B#%E7%AE%80%E4%BB%8B

InternVL2-26B应该更好，但显存没那么大，只能跑InternVL2-8B了。

下载：

cd /ssd/xiedong/InternVL2-26B
git clone https://huggingface.co/OpenGVLab/InternVL2-8B

运行docker：

docker run -it -v /ssd/xiedong/InternVL2-26B:/ssd/xiedong/InternVL2-26B --gpus device=3 -p 7895:7860 kevinchina/deeplearning:pytorch2.3.0-cuda12.1-cudnn8-devel-InternVL2 bash

进去路径:

cd /ssd/xiedong/InternVL2-26B

执行此python代码：

from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig
from lmdeploy.vl import load_image

model = '/ssd/xiedong/InternVL2-26B/InternVL2-8B'
system_prompt = 'Describe this image in English with no more than 50 words.'
image = load_image('https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg')
chat_template_config = ChatTemplateConfig('internvl-internlm2')
chat_template_config.meta_instruction = system_prompt
pipe = pipeline(model, chat_template_config=chat_template_config,
                backend_config=TurbomindEngineConfig(session_len=8192))
response = pipe(('Describe this image in English with no more than 50 words.', image))
print(response.text)

或者传一张图片后执行这种代码:

from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig
from lmdeploy.vl import load_image

model = '/ssd/xiedong/InternVL2-26B/InternVL2-8B'
system_prompt = 'Describe this image in English with no more than 50 words.'
image = load_image('/ssd/xiedong/InternVL2-26B/000030982.jpg')
chat_template_config = ChatTemplateConfig('internvl-internlm2')
chat_template_config.meta_instruction = system_prompt
pipe = pipeline(model, chat_template_config=chat_template_config,
                backend_config=TurbomindEngineConfig(session_len=8192))
response = pipe(('Describe this image in English with no more than 50 words.', image))
print(response.text)

fastapi服务

执行这个代码可以开启一个fastapi接口，7860端口会被docker映射到7895.

from fastapi import FastAPI, File, UploadFile
from fastapi.responses import JSONResponse
from lmdeploy import pipeline, TurbomindEngineConfig, ChatTemplateConfig
from lmdeploy.vl import load_image
from PIL import Image
import io

app = FastAPI()

model = '/ssd/xiedong/InternVL2-26B/InternVL2-8B'
system_prompt = 'Describe this image in English with no more than 50 words.'
chat_template_config = ChatTemplateConfig('internvl-internlm2')
chat_template_config.meta_instruction = system_prompt
pipe = pipeline(model, chat_template_config=chat_template_config,
                backend_config=TurbomindEngineConfig(session_len=8192))

@app.post("/describe-image")
def describe_image(file: UploadFile = File(...)):
    try:
        # 将上传的文件转为Pillow图像对象
        image = Image.open(io.BytesIO(file.file.read()))

        # 使用load_image方法加载图像
        loaded_image = load_image(image)

        # 调用模型处理图片
        response = pipe(('Describe this image in English with no more than 50 words, just need to output a captioning of the image.', loaded_image))

        # 返回描述结果
        return JSONResponse(content={"description": response.text})

    except Exception as e:
        return JSONResponse(content={"error": str(e)}, status_code=500)
    finally:
        file.file.close()

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=7860)

请求fastapi接口

import requests

url = "http://10.136.19.26:7895/describe-image"

# 要上传的图片文件路径
file_path = "output_image.png"

# 打开文件并发送POST请求
with open(file_path, "rb") as file:
    files = {"file": file}
    response = requests.post(url, files=files)

# 检查响应并打印结果
if response.status_code == 200:
    print("Description:", response.json().get("description"))
else:
    print("Error:", response.json().get("error"))