题意：使用本地部署的LLM与Langchain的OpenAI LLM封装器

问题背景：

I have deployed llm model locally which follows openai api schema. As it's endpoint follows openai schema, I don't want to write separate inference client.

我已经本地部署了一个遵循OpenAI API架构的LLM模型。由于它的端点遵循OpenAI架构，我不想编写单独的推理客户端

Is there any way we can utilize existing openai wrapper by langchain to do inference for my localhost model.

有没有办法利用Langchain中现有的OpenAI封装器来对我本地部署的模型进行推理？

I checked there is a openai adapter by langchain, but it seems like it require provider, which again I have to write separate client.

我查了一下，发现Langchain有一个OpenAI适配器，但它似乎需要一个提供者，这样我还是得编写一个单独的客户端。

Overall goal it to not write any redundant code as it's already been maintained by langchain and may change with time. We can modify our api wrt openai and it works out of the box.

总体目标是不写任何冗余代码，因为这些代码已经由Langchain维护，并且可能会随时间变化。我们可以根据OpenAI的标准修改我们的API，这样它就可以直接使用。

Your suggestion is appreciated.        感谢您的建议

问题解决：

It turns out you can utilize existing ChatOpenAI wrapper from langchain and update openai_api_base with the url where your llm is running which follows openai schema, add any dummy value to openai_api_key can be any random string but is necessary as they have validation for this and finally set model_name to whatever model you've deployed.

事实证明，你可以利用Langchain中的现有ChatOpenAI封装器，并将openai_api_base更新为你的LLM运行所在的URL，该URL遵循OpenAI的架构。openai_api_key可以设置为任意随机字符串，尽管它只是一个占位符，但由于需要通过验证，必须提供。最后，将model_name设置为你部署的模型名称即可。

Rest other params are same as what you'll set in openai, if you want you can set it.

其他参数与在OpenAI中设置的相同，如果需要，你可以自行设置。

from langchain_community.chat_models import ChatOpenAI

llm = ChatOpenAI(
        openai_api_base="http://<host-ip>:<port>/<v1>",
        openai_api_key="dummy_value",
        model_name="model_deployed")