How to tune agent _executor for better understanding of the database

news2024/9/20 16:39:28

题意：如何调整agent _executor以更好地理解数据库

问题背景：

I have a database in which I have connected an agent too. However, I have noticed that it sometimes gets confused between whether or not it should return a column ID or persons first name when asked "which person sold the most....?" Is there a way to tune/adjust the create_sql_agent from langchain.agents at which I can tell the agent to not return column ID but return first and last name based on questions structured like that?

我有一个数据库，并且我也已经连接了一个智能代理。然而，我注意到当被问到“哪个人卖得最多....？”时，它有时会混淆是否应该返回列ID还是人的名字。有没有办法调整或配置langchain.agents中的create_sql_agent，以便我告诉代理不要返回列ID，而是基于这样的问题结构返回名和姓？

I think the question may be related to this post but I am unsure how to include that/and structure that properly: Training SQL Agent · langchain-ai/langchain · Discussion #9591 · GitHub

我认为这个问题可能与这篇帖子有关，但我不确定如何正确地将其包括在内/以及如何组织它：

System Info langchain-openai==0.1.3 Python 3.11.7 Windows 11

Basic Model

from langchain_openai import ChatOpenAI
from langchain.agents.agent_toolkits import SQLDatabaseToolkit
from langchain.agents import create_sql_agent
from langchain.sql_database import SQLDatabase


llm = ChatOpenAI(model_name="gpt-3.5-turbo-1106", temperature=0, openai_api_key=os.environ.get('OPENAI_API_KEY'))

toolkit = SQLDatabaseToolkit(db=db, llm=llm)

agent_executor = create_sql_agent(
    llm=llm,
    toolkit=toolkit,
    verbose=False,
    agent_type="openai-tools")


print(agent_executor.invoke("What is my data about"))

Nothing, unsure how to progress as I can not find examples.

没什么进展，因为找不到例子，所以不知道如何继续。

问题解决：

You can customize the prompts given to the LLM by passing a prompt argument to create_sql_agent function.

您可以通过向create_sql_agent函数传递一个提示参数来定制给大语言模型（LLM）的提示。

Looking at create_sql_agent source code at langchain/libs/community/langchain_community/agent_toolkits/sql/base.py at master · langchain-ai/langchain · GitHub

在查看create_sql_agent的源代码时，

you can see if prompt is None it will create a prompt using:

你可以看到，如果prompt是None，它将使用以下方式创建一个提示：

messages = [
                SystemMessage(content=cast(str, prefix)),
                HumanMessagePromptTemplate.from_template("{input}"),
                AIMessage(content=suffix or SQL_FUNCTIONS_SUFFIX),
                MessagesPlaceholder(variable_name="agent_scratchpad"),
            ]
prompt = ChatPromptTemplate.from_messages(messages)

using prefix and suffix from there: 从那里使用前缀和后缀：

from langchain_community.agent_toolkits.sql.prompt import (
    SQL_FUNCTIONS_SUFFIX,
    SQL_PREFIX,
)

So you can do the same thing to create a prompt with more customized instructions.

所以你可以做同样的事情来创建一个带有更多自定义指令的提示。

And then pass it to the agent. 然后将其传递给代理。

Here is an example with the prefix and suffix can explicitly be customized, to add a line about your issue.

这里有一个例子，可以明确自定义前缀和后缀，以添加一行关于您的问题的描述。

from langchain_core.prompts.chat import (
    ChatPromptTemplate,
    HumanMessagePromptTemplate,
    SystemMessagePromptTemplate,
    AIMessagePromptTemplate,
    MessagesPlaceholder,
)

prefix = """
You are an agent designed to interact with a SQL database.
Given an input question, create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer.
Unless the user specifies a specific number of examples they wish to obtain, always limit your query to at most {top_k} results.
You can order the results by a relevant column to return the most interesting examples in the database.
Never query for all the columns from a specific table, only ask for the relevant columns given the question.
You have access to tools for interacting with the database.
Only use the below tools. Only use the information returned by the below tools to construct your final answer.
You MUST double check your query before executing it. If you get an error while executing a query, rewrite the query and try again.

DO NOT make any DML statements (INSERT, UPDATE, DELETE, DROP etc.) to the database.

If the question does not seem related to the database, just return "I don't know" as the answer.
"""

suffix = """I should look at the tables in the database to see what I can query.  Then I should query the schema of the most relevant tables.
"""

messages = [
                SystemMessagePromptTemplate.from_template(prefix),
                HumanMessagePromptTemplate.from_template("{input}"),
                AIMessagePromptTemplate.from_template(suffix),
                MessagesPlaceholder(variable_name="agent_scratchpad"),
            ]
prompt = ChatPromptTemplate.from_messages(messages)

agent_executor = create_sql_agent(llm,
                                   toolkit=toolkit,
                                   agent_type="openai-tools",
                                   prompt = prompt,
                                   verbose=False)