AutoGen - Build Powerful AI Agents with ChatGPT/GPT-4

原文:AutoGen - Build Powerful AI Agents with ChatGPT/GPT-4 | MLExpert - Crush Your Machine Learning interview

In this tutorial, we'll explore AutoGen1, a Microsoft library that lets you create LLM applications with agents. These agents can communicate and help you solve complex tasks.

Join the AI BootCamp!

Ready to dive deep into the world of AI and Machine Learning? Join our BootCamp to transform your career with the latest skills and real-world project experience. LLMs, ML best practices, and more!

JOIN NOW

We'll begin with an introduction to AutoGen and its benefits. Then, we'll kick off with a basic example of building a single agent for analyzing stock price trends. Afterward, we'll delve into a more advanced demonstration, using four agents to construct a cryptocurrency indicator, drawing insights from historical prices and news.

In this part, we will be using Jupyter Notebook to run the code. If you prefer to follow along, you can find the notebook on GitHub: GitHub Repository(opens in a new tab)

What Makes AutoGen Cool?

Flexible Chats: AutoGen can make agents talk to each other to solve problems. This is much more powerful than just one LLM doing everything.
Customization: You can create agents to do specific tasks. You can choose which LLM to use, how people can join the conversation, and even bring in tools to help.
Human Input: AutoGen lets humans join the conversation, too. They can give their ideas and feedback to help the agents.

AutoGen is like having a bunch of smart friends who work together to get things done, and it's made with help from top-notch researchers.

Setup

You can install AutoGen with pip:

pip install -Uqqq pip --progress-bar offpip install -qqq pyautogen==0.1.10 --progress-bar off

Let's add the required libraries:

import jsonfrom getpass import getpass import autogenimport pandas as pdimport requestsfrom autogen import AssistantAgent, UserProxyAgentfrom IPython.display import Image

Next, you need to enter your API key for OpenAI (get yours from https://platform.openai.com/account/api-keys(opens in a new tab)):

OPENAI_API_KEY = getpass()

We'll get news and historical prices for Bitcoin and Ethereum from Alpha Vantage2. You can get your API key from https://www.alphavantage.co/support/#api-key(opens in a new tab). Let's enter it:

ALPHA_VANTAGE_API_KEY = getpass()

You can also load the data for this tutorial (if you want to reproduce the results) from my Google Drive:

gdown 1YIw3kRmPmWPeVu-wJlG3uNtJ4YjBVDEqgdown 1zBz5qC1TmweDOKsmFo2AXSjk8uI6jT7dgdown 1BpCOBIz-3OVgW2s337n9lgxgL5z8G4mG

Simple Agent

Let's start with a simple example. We'll create an agent that can fetch historical prices for Bitcoin, Ethereum, Tesla stock and plot them.

First, we need to create a configuration for our agent:

gpt_config_list = [    {        "model": "gpt-4", # or use "gpt-3.5-turbo"        "api_key": OPENAI_API_KEY,    }] llm_config = {"config_list": gpt_config_list, "use_cache": False, "temperature": 0}

The important part is the selection of model and the API key. We'll use GPT-4 for this example. You can also use GPT-3.5 Turbo (ChatGPT). Let's create the agent:

assistant = AssistantAgent(    name="assistant",    llm_config=llm_config,)

To start the conversation, we need another agent that will represent the user (you). We'll use the UserProxyAgent for this:

def is_termination_msg(data):    has_content = "content" in data and data["content"] is not None    return has_content and "TERMINATE" in data["content"] user_proxy = UserProxyAgent(    name="user_proxy",    human_input_mode="NEVER",    max_consecutive_auto_reply=10,    is_termination_msg=is_termination_msg,    code_execution_config={"work_dir": "coding"},)

We'll use the is_termination_msg function to stop the conversation when the agent replies with TERMINATE anywhere in the text. The code_execution_config will tell the agent to save any assets (code, images etc) in the coding directory. Let's start the conversation:

user_proxy.initiate_chat(    assistant,    message="What is the current year? Compare the year-to-date gain for BTC, ETH and TESLA.",)

The assistant goes through a couple of rounds of fixing code and replying with the results (check the Colab notebook for the complete output). Here's the final report:

assistant

The code has successfully fetched the year-to-date (YTD) gain for BTC (Bitcoin),ETH (Ethereum), and TSLA (Tesla). Here are the results: - The YTD gain for BTC (Bitcoin) is approximately 61.25%- The YTD gain for ETH (Ethereum) is approximately 29.91%- The YTD gain for TSLA (Tesla) is not available (NaN) The NaN result for Tesla could be due to various reasons such as lack of datafor the specified period or issues with the data source. It's recommended tocheck the data source or try a different one if necessary. TERMINATE

Isn't that cool? For some reason, the Tesla stock price is unavailable. We can even ask the agent to plot the data for us:

user_proxy.send(    recipient=assistant,    message="Plot a chart of their price change YTD and save to price_ytd.png.",)

Here's the final message from the assistant:

assistant

Great! The code has successfully fetched the data and created the plot. The plothas been saved to a file named 'price_ytd.png'. You can find this file in thecurrent directory. Please open the 'price_ytd.png' file to view the plot. It shows the price changeyear-to-date (YTD) for BTC (Bitcoin), ETH (Ethereum), and TSLA (Tesla). TERMINATE

Let's look at the plot:

Starting plot from the assistant

The plot is not very useful because the price of Bitcoin is much higher than the price of Ethereum and Tesla stock. We can ask the assistant to scale the prices for us:

user_proxy.send(    recipient=assistant,    message="I can't see the ETH stock price on the chart",)

assistant

Great! The code has successfully fetched the data and created the plot. The plothas been saved to a file named 'price_ytd.png'. You can find this file in thecurrent directory. Please open the 'price_ytd.png' file to view the plot. It shows the cumulativepercentage change in price year-to-date (YTD) for BTC (Bitcoin), ETH (Ethereum),and TSLA (Tesla). This should make it easier to compare the performance of theseassets over the same period. TERMINATE

Scaled prices

Much better! We now even have the Tesla prices showing. Though, the dates are not properly showing. We can ask the assistant to fix that:

user_proxy.send(    recipient=assistant,    message="The dates overlap, increase the plot size",)

assistant

Great! The code has successfully fetched the data and created the plot with alarger size and rotated date labels. The plot has been saved to a file named'price_ytd.png'. You can find this file in the current directory. Please open the 'price_ytd.png' file to view the plot. It shows the cumulativepercentage change in price year-to-date (YTD) for BTC (Bitcoin), ETH (Ethereum),and TSLA (Tesla). The dates on the x-axis should now be more readable. TERMINATE

Final plot from the assistant

I am impressed. After a couple of rounds of fixing code, the assistant was able to produce a nice plot for us and save it to a file.

Build AI Agent Workforce for Cryptocurrency Analysis

Your first agent was very useful, but the AutoGen library shines when you have multiple agents working together. Let's build a cryptocurrency indicator that can analyze historical prices and news to give us recommended trading actions for Bitcoin and Ethereum.

Let's start with getting the latest news and prices for cryptocurrencies. We'll use the Alpha Vantage API to get the data:

def fetch_prices_for_symbol(symbol: str, days: int, use_cache: bool = False) -> str:    if use_cache:        if symbol == "BTC":            price_df = pd.read_csv("btc_price.csv")        else:            price_df = pd.read_csv("eth_price.csv")        return price_df[["date", "close", "volume"]].head(days)     url = f"https://www.alphavantage.co/query?function=DIGITAL_CURRENCY_DAILY&symbol={symbol}&market=USD&apikey={ALPHA_VANTAGE_API_KEY}"    r = requests.get(url)    price_data = r.json()     rows = []    for key, day_data in price_data["Time Series (Digital Currency Daily)"].items():        rows.append(            {                "date": key,                "high": day_data["2a. high (USD)"],                "low": day_data["3b. low (USD)"],                "close": day_data["4a. close (USD)"],                "volume": day_data["5. volume"],            }        )    df = pd.DataFrame(rows)    return df[["date", "close", "volume"]].head(days)

The fetch_prices_for_symbol function will fetch the latest prices for a given symbol (BTC or ETH). We can also use the cache to reproduce the results from this tutorial. Let's create a helper function that will fetch the prices for both cryptocurrencies and convert it to a string:

def fetch_prices(days: int) -> str:    btc_df = fetch_prices_for_symbol("BTC", days, use_cache=True)    eth_df = fetch_prices_for_symbol("ETH", days, use_cache=True)     btc_txt = btc_df.to_string(index=None)    eth_txt = eth_df.to_string(index=None)     return f"""$BTC prices (last {days} days){btc_txt} $ETH prices (last {days} days){eth_txt}    """.strip()

Next, we'll create a function that will fetch the latest news for the blockchain topic:

def fetch_news(latest_n: int) -> str:    news_df = pd.read_csv("news.csv", sep=";")    return news_df[["summary", "sentiment_score"]].head(latest_n).to_string(index=None)    url = f"https://www.alphavantage.co/query?function=NEWS_SENTIMENT&sort=LATEST&topics=blockchain&apikey={ALPHA_VANTAGE_API_KEY}"    r = requests.get(url)    data = r.json()     rows = []    for item in data["feed"]:        rows.append(            {                "title": item["title"],                "summary": item["summary"],                "time_published": item["time_published"],                "sentiment_score": item["overall_sentiment_score"],            }        )    df = pd.DataFrame(rows)    return df[["summary", "sentiment_score"]].head(latest_n).to_string(index=None)

Again, this can use the cache to reproduce the results from this tutorial. Let's try out the fetch_prices function to fetch the 14 latest prices for Bitcoin and Ethereum:

prices_txt = fetch_prices(days=14)print(prices_txt)

$BTC prices (last 14 days) date close volume 2023-10-11 27420.00 422.773042023-10-10 27390.12 22776.84383 2023-10-09 27590.12 31534.74639 2023-10-0827917.05 19693.56921 2023-10-07 27956.67 13348.83825 2023-10-06 27931.0937983.24277 2023-10-05 27410.39 30681.49619 2023-10-04 27778.57 29816.142002023-10-03 27426.46 28928.96555 2023-10-02 27494.51 57071.14241 2023-10-0127992.57 24602.81468 2023-09-30 26962.56 12804.62307 2023-09-29 26906.9628478.76219 2023-09-28 27021.39 44517.83491 $ETH prices (last 14 days) date close volume 2023-10-11 1569.76 3369.10262023-10-10 1567.63 240659.3287 2023-10-09 1580.13 340141.0988 2023-10-08 1632.84149424.6635 2023-10-07 1633.57 101241.5732 2023-10-06 1645.03 220507.92862023-10-05 1611.79 237378.7449 2023-10-04 1646.58 222861.7167 2023-10-03 1656.88198287.1164 2023-10-02 1662.40 378575.6898 2023-10-01 1733.79 225611.58682023-09-30 1670.89 136924.9141 2023-09-29 1667.45 251975.6728 2023-09-28 1652.99331409.7015

We have our functions ready, but who and when will call them? Fortunatelly, we can tell the AutoGen agents to use them. Let's create a configuration:

llm_config = {    "config_list": gpt_config_list,    "use_cache": False,    "temperature": 0,    "functions": [        {            "name": "fetch_prices",            "description": "Fetch daily prices for $BTC and $ETH",            "parameters": {                "type": "object",                "properties": {                    "days": {                        "type": "integer",                        "description": "number of historical days",                    }                },                "required": ["days"],            },        },        {            "name": "fetch_news",            "description": "Fetch the latest news about cryptocurrencies",            "parameters": {                "type": "object",                "properties": {                    "latest_n": {                        "type": "integer",                        "description": "number of latest news articles to get",                    }                },                "required": ["latest_n"],            },        },    ],}

The configuration tells the agents that they can use the fetch_prices and fetch_news functions. Let's start with the first agent, the analyst:

analyst_system_message = """Analyst. You are a senior financial analyst for a cryptocurrency indicator. Follow the plan: - Get news and prices from the engineer - Give the news and prices (in compact format) to the bull and get their opinion - Give the news and prices (in compact format) to the bear and get their opinion - Write a report with BUY, SELL or HODL along with a number between 0 (extremely bearish)and 100 (extremely bullish) based on the discussion with the bear,the bull, and your own opinion for $ETH and $BTC. Add TERMINATE to the end of the message.""" analyst = AssistantAgent(    name="analyst",    system_message=analyst_system_message,    llm_config=llm_config,    is_termination_msg=is_termination_msg,    code_execution_config=False,)

The analyst will make the final decision about the cryptocurrency indicator. They will get the news and prices from the engineer, then ask the bull and the bear for their opinion. Finally, they will write a report with a recommendation (BUY, SELL or HODL) and a score between 0 and 100. Let's create the engineer:

engineer_system_message = """Engineer. You are a senior software engineer that executes the fetch_prices andfetch_news functions as requested by the analyst.""" engineer = AssistantAgent(    name="engineer",    system_message=engineer_system_message,    llm_config=llm_config,    function_map={"fetch_prices": fetch_prices, "fetch_news": fetch_news},    code_execution_config=False,)

The engineer will be the only one to execute the fetch_prices and fetch_news functions. It knows how to call them based on the llm_config and the function_map. Note that the code_execution_config=False disallows calling other functions. Let's create the bull and the bear:

bull_system_message = """Bull. You receive news and prices for $ETH and $BTC from the analyst.You're always on the bullish side for cryptocurrencies and try to find sentiment thatsupports explosive price increases in the short and long term.You reply to the senior analyst with 1-2 sentences why the data suggests bullish times ahead.""" bull = AssistantAgent(    name="bull",    system_message=bull_system_message,    code_execution_config=False,    llm_config=llm_config,)

The bull creates a short message that explains why the data suggests bullish times ahead and passes it back to the analyst. The bear searches for proof of bearish times ahead:

bear_system_message = """Bear. You receive news and prices for $ETH and $BTC from the analyst.You're always on the bearish side for cryptocurrencies and try to find sentiment thatsupports explosive price drops in the short and long term.You reply to the senior analyst with 1-2 sentences why the data suggests bearish times ahead.""" bear = AssistantAgent(    name="bear",    system_message=bear_system_message,    code_execution_config=False,    llm_config=llm_config,)

The final agent is the user_proxy that will represent you. It will initiate the chat with the analyst and send the message that will start the conversation:

admin_system_message = """A human admin. Interact with the analyst to discuss the cryptocurrency report.The final report needs to be approved by this admin.""" user_proxy = UserProxyAgent(    name="admin",    system_message=admin_system_message,    code_execution_config=False,    human_input_mode="NEVER",    is_termination_msg=is_termination_msg,)

Note that of all agents, only the engineer can execute code. The rest of the agents can only send messages to each other. Let's create the group chat and the manager:

groupchat = autogen.GroupChat(    agents=[user_proxy, analyst, engineer, bear, bull], messages=[], max_round=20)manager = autogen.GroupChatManager(groupchat=groupchat, llm_config=llm_config)

The max_round parameter tells the manager to stop the conversation after 20 rounds. We're ready to start the conversation. Let's pass our message to the manager:

message = """Analyze $ETH and $BTC based on the latest 20 news articles and prices for thelast 14 days."""user_proxy.initiate_chat(manager, clear_history=True, message=message)

The Analyst tells the Engineer to fetch the historical prices for the last 14 days. Here's the discussion after that:

Bear

The recent price data for both $BTC and $ETH shows a downward trend, indicatingbearish sentiment. The consistent decrease in prices over the past 14 dayssuggests that investors are selling off their holdings, which could lead tofurther price drops.

Bull

While it's true that there's been a slight decrease in prices over the past 14days, this could be seen as a healthy correction in a broader bullish trend. Thehigh trading volumes for both $BTC and $ETH suggest strong market interest,which could fuel a price surge in the near future.

The Analyst tells the Engineer to fetch the latest 20 news articles. Here's the discussion after that:

Bull

The latest news articles show a positive sentiment overall, with developmentslike Circle's initiative to make USDC accessible on Polygon PoS, Immutable'scollaboration with Amazon Web Services, and Binance's commitment to invest inpromising crypto startups. These advancements suggest a bullish future forcryptocurrencies as they gain more mainstream acceptance and infrastructuresupport.

Bear

The latest news articles show a mix of sentiment, but there are several bearishindicators. For instance, Bitfinex's parent company is considering a $150million share repurchase amidst rising regulatory attention, suggestingpotential instability. Additionally, the collapse of crypto exchange FTX and thehalted cryptocurrency accounts associated with Hamas indicate potential risksand volatility in the market. These factors could contribute to a bearishoutlook for cryptocurrencies.

Analyst

Based on the discussion with the bull and the bear, as well as my own analysisof the price data and news sentiment, here is my report: $BTC: The recent price trend is slightly bearish, but the overall news sentimentis positive. The high trading volumes suggest strong market interest. However,potential regulatory issues and market volatility could pose risks. Myrecommendation is HODL with a score of 50, indicating a neutral outlook. $ETH: Similar to $BTC, the price trend for $ETH is slightly bearish, but thenews sentiment is generally positive. The high trading volumes and developmentslike the collaboration between Immutable and Amazon Web Services suggest abullish future. My recommendation is HODL with a score of 60, indicating aslightly bullish outlook. TERMINATE.

The conversation went well. The analyst was able to get the latest news and prices from the engineer, then ask the bull and the bear for their opinion. Finally, you got a report with a recommendation (HODL) for both cryptocurrencies and a related score. The analyst also gave a short explanation for their decision.

Conclusion

Honestly, I am impressed with the AutoGen library. It's a great way to build AI agents and solve tasks that a single prompt can't. I hope you enjoyed this tutorial and learned something new. If you want to learn more about AutoGen, check out the official documentation(opens in a new tab).