最近在做大模型对话相关功能,需要将对话内容流式返回给前端页面(类似GPT的效果)。下面直接说下如何实现:
1.首先导入fastapi和sse流式返回所需要的包
from fastapi import APIRouter, Response, status
from sse_starlette.sse import EventSourceResponse
2.用EventSourceResponse来调用生成方法,在对话方法用yield推送对话内容
@api_router.post("/stream-chat")
def stream_chat(ask_form: ChatParam, response: Response):
# 设置响应头部信息
response.headers["Content-Type"] = "text/event-stream"
response.headers["Cache-Control"] = "no-cache"
return EventSourceResponse(stream_generate_text(ask_form, stream=True))
def stream_generate_text(ask_form: ChatParam, stream: bool = False):
for res in model.chat(query=ask_form.prompt, stream=stream, historys=history, temperature=temperature):
yield json.dumps(
{
"answer": res
},
ensure_ascii=False,
)