{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HJsHqaTksDHt"
      },
      "source": [
        "# Function Calling with OpenAIChatGenerator 📞\n",
        "\n",
        "> ⚠️ As of Haystack 2.9.0, this recipe has been deprecated. For the same example, follow [Tutorial: Building a Chat Agent with Function Calling](https://haystack.deepset.ai/tutorials/40_building_chat_application_with_function_calling)\n",
        "\n",
        "*Notebook by Bilge Yucel ([LI](https://www.linkedin.com/in/bilge-yucel/) & [X (Twitter)](https://twitter.com/bilgeycl))*\n",
        "\n",
        "A guide to understand function calling and how to use OpenAI function calling feature with [Haystack](https://github.com/deepset-ai/haystack).\n",
        "\n",
        "📚 Useful Sources:\n",
        "* [OpenAIChatGenerator Docs](https://docs.haystack.deepset.ai/v2.0/docs/openaichatgenerator)\n",
        "* [OpenAIChatGenerator API Reference](https://docs.haystack.deepset.ai/v2.0/reference/generator-api#openaichatgenerator)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PWXXqq_MPn7y"
      },
      "source": [
        "## Overview\n",
        "\n",
        "Here are some use cases of function calling from [OpenAI Docs](https://platform.openai.com/docs/guides/function-calling):\n",
        "* **Create assistants that answer questions by calling external APIs** (e.g. like ChatGPT Plugins)\n",
        "e.g. define functions like send_email(to: string, body: string), or get_current_weather(location: string, unit: 'celsius' | 'fahrenheit')\n",
        "* **Convert natural language into API calls**\n",
        "e.g. convert \"Who are my top customers?\" to get_customers(min_revenue: int, created_before: string, limit: int) and call your internal API\n",
        "* **Extract structured data from text**\n",
        "e.g. define a function called extract_data(name: string, birthday: string), or sql_query(query: string)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K04cnh_IleMV"
      },
      "source": [
        "## Set up the Development Environment"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zNyqNVFaPN1A",
        "outputId": "d8c49055-1340-4328-b184-43b94abbbf34"
      },
      "outputs": [],
      "source": [
        "%%bash\n",
        "\n",
        "pip install haystack-ai==2.8.1"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "WM-sVkYonutA"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "from getpass import getpass\n",
        "from google.colab import userdata\n",
        "\n",
        "os.environ[\"OPENAI_API_KEY\"] = userdata.get('OPENAI_API_KEY') or getpass(\"OPENAI_API_KEY: \")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0_mGdadLrcNr"
      },
      "source": [
        "## Learn about the OpenAIChatGenerator\n",
        "\n",
        "`OpenAIChatGenerator` is a component that supports the function calling feature of OpenAI.\n",
        "\n",
        "The way to communicate with `OpenAIChatGenerator` is through [`ChatMessage`](https://docs.haystack.deepset.ai/v2.0/docs/data-classes#chatmessage) list. Therefore, create a `ChatMessage` with \"USER\" role using `ChatMessage.from_user()` and send it to OpenAIChatGenerator:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6ZiH3ZUCPZfH",
        "outputId": "f3684af3-2948-4743-f46d-27a1e5a33053"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'replies': [ChatMessage(content='Natural Language Processing (NLP) is a branch of artificial intelligence that deals with the interaction between computers and humans in natural language. It focuses on the understanding, interpretation, and generation of human language to enable machines to process and analyze textual data efficiently.', role=<ChatRole.ASSISTANT: 'assistant'>, name=None, meta={'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'stop', 'usage': {'completion_tokens': 50, 'prompt_tokens': 16, 'total_tokens': 66}})]}\n"
          ]
        }
      ],
      "source": [
        "from haystack.dataclasses import ChatMessage\n",
        "from haystack.components.generators.chat import OpenAIChatGenerator\n",
        "\n",
        "client = OpenAIChatGenerator()\n",
        "response = client.run(\n",
        "    [ChatMessage.from_user(\"What's Natural Language Processing? Be brief.\")]\n",
        ")\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cbl0xs0MJ76Z"
      },
      "source": [
        "### Basic Streaming\n",
        "\n",
        "OpenAIChatGenerator supports streaming, provide a `streaming_callback` function and run the client again to see the difference."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "D0nEEd5PJ1X2",
        "outputId": "b845ce78-962c-4d74-e7a3-132f8bcf795c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Natural Language Processing (NLP) is a field of artificial intelligence that focuses on the interaction between humans and computers using natural language. It involves the development of algorithms and methods to enable computers to understand, interpret, and generate human language in a manner that is meaningful and useful."
          ]
        }
      ],
      "source": [
        "from haystack.dataclasses import ChatMessage\n",
        "from haystack.components.generators.chat import OpenAIChatGenerator\n",
        "from haystack.components.generators.utils import print_streaming_chunk\n",
        "\n",
        "client = OpenAIChatGenerator(streaming_callback=print_streaming_chunk)\n",
        "response = client.run(\n",
        "    [ChatMessage.from_user(\"What's Natural Language Processing? Be brief.\")]\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HeR1XyNytCBY"
      },
      "source": [
        "## Function Calling with OpenAIChatGenerator\n",
        "\n",
        "We'll try to recreate the [example on OpenAI docs](https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FWkDXKbeoNqZ"
      },
      "source": [
        "### Define a Function\n",
        "\n",
        "We'll define a `get_current_weather` function that mocks a Weather API call in the response:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "yV_LM-b7O6KI"
      },
      "outputs": [],
      "source": [
        "def get_current_weather(location: str, unit: str = \"celsius\"):\n",
        "  ## Do something\n",
        "  return {\"weather\": \"sunny\", \"temperature\": 21.8, \"unit\": unit}\n",
        "\n",
        "available_functions = {\n",
        "  \"get_current_weather\": get_current_weather\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zJt-mzb4oHxj"
      },
      "source": [
        "### Create the `tools`\n",
        "\n",
        "We'll then add information about this function to our `tools` list by following [OpenAI's tool schema](https://platform.openai.com/docs/api-reference/chat/create#chat-create-tools)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "1C-BzvPQntmd"
      },
      "outputs": [],
      "source": [
        "tools = [\n",
        "    {\n",
        "        \"type\": \"function\",\n",
        "        \"function\": {\n",
        "            \"name\": \"get_current_weather\",\n",
        "            \"description\": \"Get the current weather\",\n",
        "            \"parameters\": {\n",
        "                \"type\": \"object\",\n",
        "                \"properties\": {\n",
        "                    \"location\": {\n",
        "                        \"type\": \"string\",\n",
        "                        \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
        "                    },\n",
        "                    \"unit\": {\n",
        "                        \"type\": \"string\",\n",
        "                        \"enum\": [\"celsius\", \"fahrenheit\"],\n",
        "                        \"description\": \"The temperature unit to use. Infer this from the users location.\",\n",
        "                    },\n",
        "                },\n",
        "                \"required\": [\"location\", \"unit\"],\n",
        "            },\n",
        "        }\n",
        "    }\n",
        "]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bkRPp3JKpZgf"
      },
      "source": [
        "### Run OpenAIChatGenerator with tools\n",
        "\n",
        "We'll pass the list of tools in the `run()` method as `generation_kwargs`.\n",
        "\n",
        "Let's define messages and run the generator:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "OEScMyqctzFN"
      },
      "outputs": [],
      "source": [
        "from haystack.dataclasses import ChatMessage\n",
        "\n",
        "messages = []\n",
        "messages.append(ChatMessage.from_system(\"Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\"))\n",
        "messages.append(ChatMessage.from_user(\"What's the weather like in Berlin?\"))\n",
        "\n",
        "client = OpenAIChatGenerator(streaming_callback=print_streaming_chunk)\n",
        "response = client.run(\n",
        "    messages=messages,\n",
        "    generation_kwargs={\"tools\":tools}\n",
        ")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OvPWnfpnQjJa"
      },
      "source": [
        "It's a function call! 📞 The response gives us information about the function name and arguments to use to call that function:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8dd2QOOYvIJB",
        "outputId": "bfd50f26-be16-4229-bd83-d8f90038e0dc"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'replies': [ChatMessage(content='[{\"index\": 0, \"id\": \"call_fFQKCAUba8RRu2BZ4v8IVYPH\", \"function\": {\"arguments\": \"{\\\\n  \\\\\"location\\\\\": \\\\\"Berlin\\\\\",\\\\n  \\\\\"unit\\\\\": \\\\\"celsius\\\\\"\\\\n}\", \"name\": \"get_current_weather\"}, \"type\": \"function\"}]', role=<ChatRole.ASSISTANT: 'assistant'>, name=None, meta={'model': 'gpt-4o-mini-2024-07-18', 'index': 0, 'finish_reason': 'tool_calls', 'usage': {}})]}"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "response"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bbxk3gEbvJ_m"
      },
      "source": [
        "Optionally, add the message with function information to the message list"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "VO0kAxGRQmM-"
      },
      "outputs": [],
      "source": [
        "messages.append(response[\"replies\"][0])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0mSQij2-OyoV"
      },
      "source": [
        "See how we can extract the `function_name` and `function_args` from the message"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ADK925C6o3uO",
        "outputId": "cbd2d908-1242-4506-82e6-876a9842b362"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "function_name: get_current_weather\n",
            "function_args: {'location': 'Berlin', 'unit': 'celsius'}\n"
          ]
        }
      ],
      "source": [
        "import json\n",
        "\n",
        "function_call = json.loads(response[\"replies\"][0].content)[0]\n",
        "function_name = function_call[\"function\"][\"name\"]\n",
        "function_args = json.loads(function_call[\"function\"][\"arguments\"])\n",
        "print(\"function_name:\", function_name)\n",
        "print(\"function_args:\", function_args)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R1HWsY5_tZ6d"
      },
      "source": [
        "### Make a Tool Call\n",
        "\n",
        "Let's locate the corresponding function for `function_name` in our `available_functions` dictionary and use `function_args` when calling it. Once we receive the response from the tool, we'll append it to our `messages` for later sending to OpenAI."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "y2rF5JcytaMX"
      },
      "outputs": [],
      "source": [
        "function_to_call = available_functions[function_name]\n",
        "function_response = function_to_call(**function_args)\n",
        "function_message = ChatMessage.from_function(content=json.dumps(function_response), name=function_name)\n",
        "messages.append(function_message)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oEB3e_PcZ11n"
      },
      "source": [
        "Make the last call to OpenAI with response coming from the function and see how OpenAI uses the provided information"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c1p6w-uxZ09K",
        "outputId": "97973938-7d43-4216-98b2-23f75c6c99a4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The current weather in Berlin is sunny with a temperature of 21.8°C."
          ]
        }
      ],
      "source": [
        "response = client.run(\n",
        "    messages=messages,\n",
        "    generation_kwargs={\"tools\":tools}\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_WTrLQr-X1Y2"
      },
      "source": [
        "## Improve the Example\n",
        "\n",
        "Let's add more tool to our example and improve the user experience 👇\n",
        "\n",
        "We'll add one more tool `use_haystack_pipeline` for OpenAI to use when there's a question about countries and capitals:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "rfYYSytSVtPm"
      },
      "outputs": [],
      "source": [
        "tools = [\n",
        "    {\n",
        "        \"type\": \"function\",\n",
        "        \"function\": {\n",
        "            \"name\": \"get_current_weather\",\n",
        "            \"description\": \"Get the current weather\",\n",
        "            \"parameters\": {\n",
        "                \"type\": \"object\",\n",
        "                \"properties\": {\n",
        "                    \"location\": {\n",
        "                        \"type\": \"string\",\n",
        "                        \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
        "                    },\n",
        "                    \"unit\": {\n",
        "                        \"type\": \"string\",\n",
        "                        \"enum\": [\"celsius\", \"fahrenheit\"],\n",
        "                        \"description\": \"The temperature unit to use. Infer this from the users location.\",\n",
        "                    },\n",
        "                },\n",
        "                \"required\": [\"location\", \"unit\"],\n",
        "            },\n",
        "        }\n",
        "    },\n",
        "    {\n",
        "        \"type\": \"function\",\n",
        "        \"function\": {\n",
        "            \"name\": \"use_haystack_pipeline\",\n",
        "            \"description\": \"Use for search about countries and capitals\",\n",
        "            \"parameters\": {\n",
        "                \"type\": \"object\",\n",
        "                \"properties\": {\n",
        "                    \"query\": {\n",
        "                        \"type\": \"string\",\n",
        "                        \"description\": \"The query to use in the search. Infer this from the user's message\",\n",
        "                    },\n",
        "                },\n",
        "                \"required\": [\"query\"]\n",
        "            },\n",
        "        }\n",
        "    },\n",
        "]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "VHxeFwAhrNZw"
      },
      "outputs": [],
      "source": [
        "def get_current_weather(location: str, unit: str = \"celsius\"):\n",
        "  return {\"weather\": \"sunny\", \"temperature\": 21.8, \"unit\": unit}\n",
        "\n",
        "def use_haystack_pipeline(query: str):\n",
        "  # It returns a mock response\n",
        "  return {\"documents\": \"Cutopia is the capital of Utopia\", \"query\": query}\n",
        "\n",
        "available_functions = {\n",
        "  \"get_current_weather\": get_current_weather,\n",
        "  \"use_haystack_pipeline\": use_haystack_pipeline,\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5UkjGYEfwPzS"
      },
      "source": [
        "### Start the Application\n",
        "\n",
        "Have fun having a chat with OpenAI 🎉\n",
        "\n",
        "Example queries you can try:\n",
        "* \"***What's the capital of Utopia***\", \"***Is it sunny there?***\": To test the messages are being recorded and sent\n",
        "* \"***What's the weather like in the capital of Utopia?***\": To force two function calls\n",
        "* \"***What's the weather like today?***\": To force OpenAI to ask more clarification"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sK_JeKZLhXcy",
        "outputId": "8ffaaa59-bae1-4ca1-fd90-eedf768ec94b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\n",
            "INFO: Type 'exit' or 'quit' to stop\n",
            "What's the weather like today?\n",
            "Sure, can you please tell me your current location?INFO: Type 'exit' or 'quit' to stop\n",
            "utopia\n",
            "The weather in Utopia today is sunny with a temperature of 21.8 degrees Celsius.INFO: Type 'exit' or 'quit' to stop\n",
            "exit\n"
          ]
        }
      ],
      "source": [
        "import json\n",
        "from haystack.dataclasses import ChatMessage, ChatRole\n",
        "\n",
        "messages = []\n",
        "messages.append(ChatMessage.from_system(\"Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\"))\n",
        "\n",
        "print(messages[-1].content)\n",
        "\n",
        "while True:\n",
        "  # if this is a tool call\n",
        "  if response and response[\"replies\"][0].meta[\"finish_reason\"] == 'tool_calls':\n",
        "    function_calls = json.loads(response[\"replies\"][0].content)\n",
        "    for function_call in function_calls:\n",
        "      function_name = function_call[\"function\"][\"name\"]\n",
        "      function_to_call = available_functions[function_name]\n",
        "      function_args = json.loads(function_call[\"function\"][\"arguments\"])\n",
        "\n",
        "      function_response = function_to_call(**function_args)\n",
        "      function_message = ChatMessage.from_function(content=json.dumps(function_response), name=function_name)\n",
        "      messages.append(function_message)\n",
        "\n",
        "  # Regular Conversation\n",
        "  else:\n",
        "    # If it's not user's first message and there's an assistant message\n",
        "    if not messages[-1].is_from(ChatRole.SYSTEM):\n",
        "      messages.append(ChatMessage.from_assistant(response[\"replies\"][0].content))\n",
        "\n",
        "    user_input = input(\"INFO: Type 'exit' or 'quit' to stop\\n\")\n",
        "    if user_input.lower() == \"exit\" or user_input.lower() == \"quit\":\n",
        "      break\n",
        "    else:\n",
        "      messages.append(ChatMessage.from_user(user_input))\n",
        "\n",
        "  response = client.run(\n",
        "    messages=messages,\n",
        "    generation_kwargs={\"tools\":tools}\n",
        "  )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XgiXu5bltJJN"
      },
      "source": [
        "### Print the summary of the conversation\n",
        "\n",
        "This part can help you understand the message order"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tMom7ESfT_R_",
        "outputId": "5c883070-0e9f-42d2-9afc-d0821f0244bb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "=== SUMMARY ===\n",
            "\n",
            " - Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.\n",
            "\n",
            " - What's the weather like today?\n",
            "\n",
            " - Sure, can you please tell me your current location?\n",
            "\n",
            " - utopia\n",
            "\n",
            " - {\"weather\": \"sunny\", \"temperature\": 21.8, \"unit\": \"celsius\"}\n",
            "\n",
            " - The weather in Utopia today is sunny with a temperature of 21.8 degrees Celsius.\n"
          ]
        }
      ],
      "source": [
        "print(\"\\n=== SUMMARY ===\")\n",
        "for m in messages:\n",
        "  print(f\"\\n - {m.content}\")"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "authorship_tag": "ABX9TyMyWvdrsAJ9LGk5Lfb7YZGB",
      "include_colab_link": true,
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}