{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "HJsHqaTksDHt" }, "source": [ "# Tutorial: Building a Chat Agent with Function Calling\n", "\n", "- **Level**: Advanced\n", "- **Time to complete**: 20 minutes\n", "- **Components Used**: [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore), [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder), [SentenceTransformersTextEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerstextembedder), [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), [ChatPromptBuilder](https://docs.haystack.deepset.ai/docs/chatpromptbuilder), [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator), [ToolInvoker](https://docs.haystack.deepset.ai/docs/toolinvoker)\n", "- **Prerequisites**: You must have an [OpenAI API Key](https://platform.openai.com/api-keys) and be familiar with [creating pipelines](https://docs.haystack.deepset.ai/docs/creating-pipelines)\n", "- **Goal**: After completing this tutorial, you will have learned how to build chat applications that demonstrate agent-like behavior using OpenAI's function calling feature.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "PWXXqq_MPn7y" }, "source": [ "## Overview\n", "\n", "\n", "\n", "📚 Useful Sources:\n", "* [OpenAIChatGenerator Docs](https://docs.haystack.deepset.ai/docs/openaichatgenerator)\n", "* [OpenAIChatGenerator API Reference](https://docs.haystack.deepset.ai/reference/generators-api#openaichatgenerator)\n", "* [🧑‍🍳 Cookbook: Function Calling with OpenAIChatGenerator](https://github.com/deepset-ai/haystack-cookbook/blob/main/notebooks/function_calling_with_OpenAIChatGenerator.ipynb)\n", "\n", "[OpenAI's function calling](https://platform.openai.com/docs/guides/function-calling) connects large language models to external tools. By providing a `tools` list with functions and their specifications to the OpenAI API calls, you can easily build chat assistants that can answer questions by calling external APIs or extract structured information from text.\n", "\n", "In this tutorial, you'll learn how to convert your Haystack pipeline into a function-calling tool and how to implement applications using OpenAI's Chat Completion API through `OpenAIChatGenerator` for agent-like behavior." ] }, { "cell_type": "markdown", "metadata": { "id": "K04cnh_IleMV" }, "source": [ "## Setting up the Development Environment\n", "\n", "Install Haystack and [sentence-transformers](https://pypi.org/project/sentence-transformers/) using pip:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "id": "zNyqNVFaPN1A" }, "outputs": [], "source": [ "%%bash\n", "\n", "pip install haystack-ai\n", "pip install \"sentence-transformers>=4.1.0\"" ] }, { "cell_type": "markdown", "metadata": { "id": "2owelT4NpXtw" }, "source": [ "Save your OpenAI API key as an environment variable:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "WM-sVkYonutA", "outputId": "7895e2b4-97a3-4cfe-e2cc-80bceac53b3e" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Enter OpenAI API key:··········\n" ] } ], "source": [ "import os\n", "from getpass import getpass\n", "\n", "if \"OPENAI_API_KEY\" not in os.environ:\n", " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter OpenAI API key:\")" ] }, { "cell_type": "markdown", "metadata": { "id": "0_mGdadLrcNr" }, "source": [ "## Learning about the OpenAIChatGenerator\n", "\n", "[OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator) is a component that supports the function calling feature of OpenAI through Chat Completion API. In contrary to `OpenAIGenerator`, the way to communicate with `OpenAIChatGenerator` is through [`ChatMessage`](https://docs.haystack.deepset.ai/docs/data-classes#chatmessage) list. Read more about the difference between them in [Generators vs Chat Generators](https://docs.haystack.deepset.ai/docs/generators-vs-chat-generators).\n", "\n", "\n", "To start working with the `OpenAIChatGenerator`, create a `ChatMessage` object with \"SYSTEM\" role using `ChatMessage.from_system()` and another `ChatMessage` with \"USER\" role using `ChatMessage.from_user()`. Then, pass this messages list to `OpenAIChatGenerator` and run:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "yzuTzWpJFVbH", "outputId": "9ea4c85b-b533-4bc1-dff3-304d30a24021" }, "outputs": [ { "data": { "text/plain": [ "{'replies': [ChatMessage(_role=, _content=[TextContent(text='Natural Language Processing (NLP) ist ein Teilbereich der Künstlichen Intelligenz, der sich mit der Interaktion zwischen Computern und menschlicher Sprache beschäftigt. Ziel ist es, natürliche Sprache zu verstehen, zu interpretieren und zu erzeugen, um Kommunikation zwischen Mensch und Maschine zu erleichtern.')], _name=None, _meta={'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 62, 'prompt_tokens': 33, 'total_tokens': 95, 'completion_tokens_details': CompletionTokensDetails(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0), 'prompt_tokens_details': PromptTokensDetails(audio_tokens=0, cached_tokens=0)}})]}" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from haystack.dataclasses import ChatMessage\n", "from haystack.components.generators.chat import OpenAIChatGenerator\n", "\n", "messages = [\n", " ChatMessage.from_system(\"Always respond in German even if some input data is in other languages.\"),\n", " ChatMessage.from_user(\"What's Natural Language Processing? Be brief.\"),\n", "]\n", "\n", "chat_generator = OpenAIChatGenerator(model=\"gpt-4o-mini\")\n", "chat_generator.run(messages=messages)" ] }, { "cell_type": "markdown", "metadata": { "id": "cbl0xs0MJ76Z" }, "source": [ "### Basic Streaming\n", "\n", "`OpenAIChatGenerator` supports streaming, provide a `streaming_callback` function and run the `chat_generator` again to see the difference." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "D0nEEd5PJ1X2", "outputId": "74613af0-282b-497a-c76a-4bee69b41d97" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Natural Language Processing (NLP) ist ein Teilbereich der Informatik und Linguistik, der sich mit der Interaktion zwischen Computern und Menschen in natürlicher Sprache befasst. Ziel ist es, Computer zu befähigen, menschliche Sprache zu verstehen, zu interpretieren und zu erzeugen. Anwendungen von NLP umfassen Spracherkennung, maschinelles Übersetzen, Sentiment-Analyse und Chatbots." ] } ], "source": [ "from haystack.components.generators.chat import OpenAIChatGenerator\n", "from haystack.components.generators.utils import print_streaming_chunk\n", "\n", "chat_generator = OpenAIChatGenerator(model=\"gpt-4o-mini\", streaming_callback=print_streaming_chunk)\n", "response = chat_generator.run(messages=messages)" ] }, { "cell_type": "markdown", "metadata": { "id": "FW91cHchCw_W" }, "source": [ "## Creating a Function Calling Tool from a Haystack Pipeline\n", "\n", "To use the function calling of OpenAI, you need to introduce `tools` to your `OpenAIChatGenerator`.\n", "\n", "For this example, you'll use a Haystack RAG pipeline as one of your tools. Therefore, you need to index documents to a document store and then build a RAG pipeline on top of it." ] }, { "cell_type": "markdown", "metadata": { "id": "FWkDXKbeoNqZ" }, "source": [ "### Index Documents with a Pipeline\n", "\n", "Create a pipeline to store the small example dataset in the [InMemoryDocumentStore](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore) with their embeddings. You will use [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder) to generate embeddings for your Documents and write them to the document store with the [DocumentWriter](https://docs.haystack.deepset.ai/docs/documentwriter).\n", "\n", "After adding these components to your pipeline, connect them and run the pipeline.\n", "\n", "> If you'd like to learn about preprocessing files before you index them to your document store, follow the [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline) tutorial." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 171, "referenced_widgets": [ "81e44362f359418f96a7fb064124e04c", "b96969f943d640a699feaa3a4ab28dad", "cabee03b724449678cffb53c9e46f630", "f59509f2462a4ba387a5c2ff6a6f911a", "2ec4c8a5f88e485abd8ce946239b59c5", "c495e6700dc3452fa76538ad1d9c1ed8", "ec769506baec4b5ca99c05d049940a6d", "a40c2224a30a43718f86a08e16c59d7e", "3f29fa86eea2420587cf3039908cc2e6", "47e025333d33466f802122517f3a584d", "7c23fd14ca8d48c2b63e13c5b599472e" ] }, "id": "ZE0SEGY92GHJ", "outputId": "5d7d3003-971b-453f-fc47-de612993d1aa" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/usr/local/lib/python3.11/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: \n", "The secret `HF_TOKEN` does not exist in your Colab secrets.\n", "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n", "You will be able to reuse this secret in all of your notebooks.\n", "Please note that authentication is recommended but still optional to access public models or datasets.\n", " warnings.warn(\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "81e44362f359418f96a7fb064124e04c", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Batches: 0%| | 0/1 [00:00 For a step-by-step guide to create a RAG pipeline with Haystack, follow the [Creating Your First QA Pipeline with Retrieval-Augmentation](https://haystack.deepset.ai/tutorials/27_first_rag_pipeline) tutorial." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "23JuUuql7PZ8", "outputId": "1ab36b2a-5f95-4580-ab3b-e60c8807795a" }, "outputs": [ { "data": { "text/plain": [ "\n", "🚅 Components\n", " - embedder: SentenceTransformersTextEmbedder\n", " - retriever: InMemoryEmbeddingRetriever\n", " - prompt_builder: ChatPromptBuilder\n", " - llm: OpenAIChatGenerator\n", "🛤️ Connections\n", " - embedder.embedding -> retriever.query_embedding (List[float])\n", " - retriever.documents -> prompt_builder.documents (List[Document])\n", " - prompt_builder.prompt -> llm.messages (List[ChatMessage])" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from haystack.components.embedders import SentenceTransformersTextEmbedder\n", "from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever\n", "from haystack.components.builders import ChatPromptBuilder\n", "from haystack.dataclasses import ChatMessage\n", "from haystack.components.generators.chat import OpenAIChatGenerator\n", "\n", "template = [\n", " ChatMessage.from_system(\n", " \"\"\"\n", "Answer the questions based on the given context.\n", "\n", "Context:\n", "{% for document in documents %}\n", " {{ document.content }}\n", "{% endfor %}\n", "Question: {{ question }}\n", "Answer:\n", "\"\"\"\n", " )\n", "]\n", "rag_pipe = Pipeline()\n", "rag_pipe.add_component(\"embedder\", SentenceTransformersTextEmbedder(model=\"sentence-transformers/all-MiniLM-L6-v2\"))\n", "rag_pipe.add_component(\"retriever\", InMemoryEmbeddingRetriever(document_store=document_store))\n", "rag_pipe.add_component(\"prompt_builder\", ChatPromptBuilder(template=template))\n", "rag_pipe.add_component(\"llm\", OpenAIChatGenerator(model=\"gpt-4o-mini\"))\n", "\n", "rag_pipe.connect(\"embedder.embedding\", \"retriever.query_embedding\")\n", "rag_pipe.connect(\"retriever\", \"prompt_builder.documents\")\n", "rag_pipe.connect(\"prompt_builder.prompt\", \"llm.messages\")" ] }, { "cell_type": "markdown", "metadata": { "id": "3vjtLi5A-XAi" }, "source": [ "### Run the Pipeline\n", "Test this pipeline with a query and see if it works as expected before you start using it as a function calling tool." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 101, "referenced_widgets": [ "d2e3e46dec734d848382ccde2f753a2c", "31585686320941e79444270287fa3bbf", "5944ad9184ab49aa999b62d152f81fc0", "3c191fd5ff34400fa703b24fc09d57e3", "480a7a302d1e45bcb2d54573f3568ad6", "1bdb07730cb146b79d0c0bfd1c9284dd", "3be0dedaea4d4a86adf089699d0814a8", "bd6e62494b8745399905a713169f5957", "1f450b7f47124f8196b3b7714ec77c2a", "825074d7daf54c8f92a0c4637246f904", "f20aca19ac55499089519e86c8c63d49" ] }, "id": "h4bXXC2Z7eaj", "outputId": "805cd637-e7f6-4099-baa0-93257b7e8aa0" }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "d2e3e46dec734d848382ccde2f753a2c", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Batches: 0%| | 0/1 [00:00, _content=[TextContent(text='Mark lives in Berlin.')], _name=None, _meta={'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 6, 'prompt_tokens': 83, 'total_tokens': 89, 'completion_tokens_details': CompletionTokensDetails(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0), 'prompt_tokens_details': PromptTokensDetails(audio_tokens=0, cached_tokens=0)}})]}}" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "query = \"Where does Mark live?\"\n", "rag_pipe.run({\"embedder\": {\"text\": query}, \"prompt_builder\": {\"question\": query}})" ] }, { "cell_type": "markdown", "metadata": { "id": "xeQf1QT0_OA0" }, "source": [ "### Convert the Haystack Pipeline into a Tool\n", "\n", "Wrap the `rag_pipe.run` call inside a function called `rag_pipeline_func`. This function should take a `query` as input and return the response generated by the LLM in the RAG pipeline you previously built. Next, initialize a `Tool` by following the steps in [Tool Initialization](https://docs.haystack.deepset.ai/docs/tool#tool-initialization); define `parameters`, set a `name` and `description` for the tool. \n", "\n", "The `Tool` abstraction in Haystack enables seamless function calling/tool calling with multiple ChatGenerators. As of Haystack 2.9, this includes support for [AnthropicChatGenerator](https://docs.haystack.deepset.ai/docs/anthropicchatgenerator), [OllamaChatGenerator](https://docs.haystack.deepset.ai/docs/ollamachatgenerator), and [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator), with more integrations expected in the future." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "id": "hCUFdKV5_Z7k" }, "outputs": [], "source": [ "from haystack.tools import Tool\n", "\n", "\n", "def rag_pipeline_func(query: str):\n", " result = rag_pipe.run({\"embedder\": {\"text\": query}, \"prompt_builder\": {\"question\": query}})\n", " return {\"reply\": result[\"llm\"][\"replies\"][0].text}\n", "\n", "\n", "parameters = {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"query\": {\n", " \"type\": \"string\",\n", " \"description\": \"The query to use in the search. Infer this from the user's message. It should be a question or a statement\",\n", " }\n", " },\n", " \"required\": [\"query\"],\n", "}\n", "\n", "rag_pipeline_tool = Tool(\n", " name=\"rag_pipeline_tool\",\n", " description=\"Get information about where people live\",\n", " parameters=parameters,\n", " function=rag_pipeline_func,\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "zJt-mzb4oHxj" }, "source": [ "## Creating a Tool from Function\n", "\n", "In addition to the `rag_pipeline_tool`, create a new tool called `get_weather_tool` to be used to get weather information of cities.\n", "\n", "First, create a function that simulates an API call to an external weather service. Instead of passing parameters as JSON (like in the previous tool), use [`create_tool_from_function`](https://docs.haystack.deepset.ai/docs/tool#create_tool_from_function). This function requires additional details using the `Annotated` type to describe tool parameters. However, based on this information, `create_tool_from_function` can automatically infer the parameters and generate a JSON schema, so you don't need to define `parameters` separately." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "id": "XaQgnmPoJD13" }, "outputs": [], "source": [ "from typing import Annotated, Literal\n", "from haystack.tools import create_tool_from_function\n", "\n", "WEATHER_INFO = {\n", " \"Berlin\": {\"weather\": \"mostly sunny\", \"temperature\": 7, \"unit\": \"celsius\"},\n", " \"Paris\": {\"weather\": \"mostly cloudy\", \"temperature\": 8, \"unit\": \"celsius\"},\n", " \"Rome\": {\"weather\": \"sunny\", \"temperature\": 14, \"unit\": \"celsius\"},\n", " \"Madrid\": {\"weather\": \"sunny\", \"temperature\": 10, \"unit\": \"celsius\"},\n", " \"London\": {\"weather\": \"cloudy\", \"temperature\": 9, \"unit\": \"celsius\"},\n", "}\n", "\n", "\n", "def get_weather(\n", " city: Annotated[str, \"the city for which to get the weather\"] = \"Berlin\",\n", " unit: Annotated[Literal[\"Celsius\", \"Fahrenheit\"], \"the unit for the temperature\"] = \"Celsius\",\n", "):\n", " \"\"\"A simple function to get the current weather for a location.\"\"\"\n", " if city in WEATHER_INFO:\n", " return WEATHER_INFO[city]\n", " else:\n", " return {\"weather\": \"sunny\", \"temperature\": 21.8, \"unit\": \"fahrenheit\"}\n", "\n", "\n", "weather_tool = create_tool_from_function(get_weather)" ] }, { "cell_type": "markdown", "metadata": { "id": "bkRPp3JKpZgf" }, "source": [ "## Running OpenAIChatGenerator with Tools\n", "\n", "To use the tool calling feature, you need to pass the list of tools to `OpenAIChatGenerator` as `tools`. \n", "\n", "Instruct the model to use provided tools with a system message and then provide a query that requires a tool call as a user message:" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "id": "OEScMyqctzFN" }, "outputs": [], "source": [ "from haystack.dataclasses import ChatMessage\n", "from haystack.components.generators.chat import OpenAIChatGenerator\n", "from haystack.components.generators.utils import print_streaming_chunk\n", "\n", "user_messages = [\n", " ChatMessage.from_system(\n", " \"Use the tool that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\"\n", " ),\n", " ChatMessage.from_user(\"Can you tell me where Mark lives?\"),\n", "]\n", "\n", "chat_generator = OpenAIChatGenerator(model=\"gpt-4o-mini\", streaming_callback=print_streaming_chunk)\n", "response = chat_generator.run(messages=user_messages, tools=[rag_pipeline_tool, weather_tool])" ] }, { "cell_type": "markdown", "metadata": { "id": "zYwb-SqaGL3O" }, "source": [ "As a response, you'll get a `ChatMessage` with information about the tool name and arguments as a [ToolCall](https://docs.haystack.deepset.ai/reference/data-classes-api#toolcall) object:" ] }, { "cell_type": "markdown", "metadata": { "id": "EHAs6kU5OPWe" }, "source": [ "```python\n", "{'replies': [\n", " ChatMessage(\n", " _role=,\n", " _content=[TextContent(text=''), ToolCall(tool_name='rag_pipeline_tool', arguments={'query': 'Where does Mark live?'}, id='call_xHEPMFrkHEKsi7tFB8FItkFC')],\n", " _name=None,\n", " meta={'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'tool_calls', 'usage': {}}\n", " )\n", " ]\n", "}\n", "```" ] }, { "cell_type": "markdown", "metadata": { "id": "_ulFfHcnGfsw" }, "source": [ "Next, use [ToolInvoker](https://docs.haystack.deepset.ai/docs/toolinvoker) to process `ChatMessage` object containing tool calls, invoke the corresponding tools and return the results as a list of `ChatMessage`. " ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 86, "referenced_widgets": [ "466461528ca1456481e860f6202df70c", "30d9aba7738f4d79a5c4d0593d9b6faa", "27424a72441e465ab03699702bf08e75", "e2046da9ab2a4f14b81ac130abf43393", "85ad1c88c416446fb1df56789c2c65a9", "c29df52ee29d4602878f502f14ec121d", "23b57a518b2a4724a807e7d87d84ed6f", "b29f84659637486c97bf97023d2d9c3a", "68457ab6ef80409abc692c2591e282c9", "f548b43c34f14120b1f5344a2b5216b7", "ce7ddd5617a44e6eb8fbaf510717d1a6" ] }, "id": "sijMc9fhR7ej", "outputId": "bd126d0f-3204-4089-a786-f0b233621149" }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "466461528ca1456481e860f6202df70c", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Batches: 0%| | 0/1 [00:00, _content=[ToolCallResult(result=\"{'reply': 'Mark lives in Berlin.'}\", origin=ToolCall(tool_name='rag_pipeline_tool', arguments={'query': 'Where does Mark live?'}, id='call_kbdd34iZhkVAdIj2ZThTbZdV'), error=False)], _name=None, _meta={})]\n" ] } ], "source": [ "from haystack.components.tools import ToolInvoker\n", "\n", "tool_invoker = ToolInvoker(tools=[rag_pipeline_tool, weather_tool])\n", "\n", "if response[\"replies\"][0].tool_calls:\n", " tool_result_messages = tool_invoker.run(messages=response[\"replies\"])[\"tool_messages\"]\n", " print(f\"tool result messages: {tool_result_messages}\")" ] }, { "cell_type": "markdown", "metadata": { "id": "I6zam1cfbuZv" }, "source": [ "As the last step, run the `OpenAIChatGenerator` again with the tool call results and get the final answer." ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "TdwkbMPLPO_L", "outputId": "0ee96a46-16f3-4a03-ce54-a69bedde24ae" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Mark lives in Berlin." ] } ], "source": [ "# Pass all the messages to the ChatGenerator with the correct order\n", "messages = user_messages + response[\"replies\"] + tool_result_messages\n", "final_replies = chat_generator.run(messages=messages, tools=[rag_pipeline_tool, weather_tool])[\"replies\"]" ] }, { "cell_type": "markdown", "metadata": { "id": "5UkjGYEfwPzS" }, "source": [ "## Building the Chat Agent\n", "\n", "As you notice above, OpenAI Chat Completions API does not call the tool; instead, the model generates JSON that you can use to call the tool in your code. That's why, to build an end-to-end chat agent, you need to check if the OpenAI response is a `tool_calls` for every message. If so, you need to call the corresponding tool with the provided arguments and send the tool response back to OpenAI. Otherwise, append both user and messages to the `messages` list to have a regular conversation with the model. Let's build an application that handles all cases.\n", "\n", "To build a nice UI for your application, you can use [Gradio](https://www.gradio.app/) that comes with a chat interface. Install `gradio`, run the code cell below and use the input box to interact with the chat application that has access to two tools you've created above. \n", "\n", "Example queries you can try:\n", "* \"***What is the capital of Sweden?***\": A basic query without any tool calls\n", "* \"***Can you tell me where Giorgio lives?***\": A basic query with one tool call\n", "* \"***What's the weather like in Berlin?***\", \"***Is it sunny there?***\": To see the messages are being recorded and sent\n", "* \"***What's the weather like where Jean lives?***\": To force two tool calls\n", "* \"***What's the weather like today?***\": To force OpenAI to ask more clarification\n", "\n", "> Keep in mind that OpenAI models can sometimes hallucinate answers or tools and might not work as expected." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "mmOBVDvmOPWe" }, "outputs": [], "source": [ "%%bash\n", "\n", "pip install -U gradio pytz" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "sK_JeKZLhXcy" }, "outputs": [], "source": [ "import gradio as gr\n", "\n", "from haystack.dataclasses import ChatMessage\n", "from haystack.components.generators.chat import OpenAIChatGenerator\n", "from haystack.components.tools import ToolInvoker\n", "\n", "tool_invoker = ToolInvoker(tools=[rag_pipeline_tool, weather_tool])\n", "chat_generator = OpenAIChatGenerator(model=\"gpt-4o-mini\", tools=[rag_pipeline_tool, weather_tool])\n", "response = None\n", "messages = [\n", " ChatMessage.from_system(\n", " \"Use the tool that you're provided with. Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\"\n", " )\n", "]\n", "\n", "\n", "def chatbot_with_tc(message, history):\n", " global messages\n", " messages.append(ChatMessage.from_user(message))\n", " response = chat_generator.run(messages=messages)\n", "\n", " while True:\n", " # if OpenAI response is a function call\n", " if response and response[\"replies\"][0].tool_calls:\n", " tool_result_messages = tool_invoker.run(messages=response[\"replies\"])[\"tool_messages\"]\n", " messages = messages + response[\"replies\"] + tool_result_messages\n", " response = chat_generator.run(messages=messages)\n", "\n", " # Regular Conversation\n", " else:\n", " messages.append(response[\"replies\"][0])\n", " break\n", " return response[\"replies\"][0].text\n", "\n", "\n", "demo = gr.ChatInterface(\n", " fn=chatbot_with_tc,\n", " examples=[\n", " \"Can you tell me where Giorgio lives?\",\n", " \"What's the weather like in Madrid?\",\n", " \"Who lives in London?\",\n", " \"What's the weather like where Mark lives?\",\n", " ],\n", " title=\"Ask me about weather or where people live!\",\n", ")\n", "\n", "## Uncomment the line below to launch the chat app with UI\n", "# demo.launch()" ] }, { "cell_type": "markdown", "metadata": { "id": "rgGKjm7cpv1_" }, "source": [ "## What's next\n", "\n", "🎉 Congratulations! You've learned how to build chat applications that demonstrate agent-like behavior using OpenAI function calling and Haystack Pipelines.\n", "\n", "If you liked this tutorial, there's more to learn about Haystack:\n", "- [Create a Swarm of Agents](https://haystack.deepset.ai/cookbook/swarm)\n", "- [Evaluating RAG Pipelines](https://haystack.deepset.ai/tutorials/35_evaluating_rag_pipelines)\n", "\n", "To stay up to date on the latest Haystack developments, you can [sign up for our newsletter](https://landing.deepset.ai/haystack-community-updates) or [join Haystack discord community](https://discord.gg/Dr63fr9NDS).\n", "\n", "Thanks for reading!" ] } ], "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.6" } }, "nbformat": 4, "nbformat_minor": 0 }