{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "2OvkPji9O-qX" }, "source": [ "# Tutorial: Creating Your First QA Pipeline with Retrieval-Augmentation\n", "\n", "- **Level**: Beginner\n", "- **Time to complete**: 10 minutes\n", "- **Components Used**: [`InMemoryDocumentStore`](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore), [`SentenceTransformersDocumentEmbedder`](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder), [`SentenceTransformersTextEmbedder`](https://docs.haystack.deepset.ai/docs/sentencetransformerstextembedder), [`InMemoryEmbeddingRetriever`](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), [`PromptBuilder`](https://docs.haystack.deepset.ai/docs/promptbuilder), [`OpenAIChatGenerator`](https://docs.haystack.deepset.ai/docs/openaichatgenerator)\n", "- **Prerequisites**: You must have an [OpenAI API Key](https://platform.openai.com/api-keys).\n", "- **Goal**: After completing this tutorial, you'll have learned the new prompt syntax and how to use PromptBuilder and OpenAIChatGenerator to build a generative question-answering pipeline with retrieval-augmentation." ] }, { "cell_type": "markdown", "metadata": { "id": "LFqHcXYPO-qZ" }, "source": [ "## Overview\n", "\n", "This tutorial shows you how to create a generative question-answering pipeline using the retrieval-augmentation ([RAG](https://www.deepset.ai/blog/llms-retrieval-augmentation)) approach with Haystack. The process involves four main components: [SentenceTransformersTextEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformerstextembedder) for creating an embedding for the user query, [InMemoryBM25Retriever](https://docs.haystack.deepset.ai/docs/inmemorybm25retriever) for fetching relevant documents, [PromptBuilder](https://docs.haystack.deepset.ai/docs/promptbuilder) for creating a template prompt, and [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator) for generating responses.\n", "\n", "For this tutorial, you'll use the Wikipedia pages of [Seven Wonders of the Ancient World](https://en.wikipedia.org/wiki/Wonders_of_the_World) as Documents, but you can replace them with any text you want.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "Kww5B_vXO-qZ" }, "source": [ "## Installing Haystack\n", "\n", "Install Haystack and other required packages with `pip`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "UQbU8GUfO-qZ", "outputId": "c33579e9-5557-43bd-a3c5-63b8373770c7" }, "outputs": [], "source": [ "%%bash\n", "\n", "pip install haystack-ai\n", "pip install \"datasets>=2.6.1\"\n", "pip install \"sentence-transformers>=4.1.0\"" ] }, { "cell_type": "markdown", "metadata": { "id": "_lvfew16O-qa" }, "source": [ "## Fetching and Indexing Documents\n", "\n", "You'll start creating your question answering system by downloading the data and indexing the data with its embeddings to a DocumentStore. \n", "\n", "In this tutorial, you will take a simple approach to writing documents and their embeddings into the DocumentStore. For a full indexing pipeline with preprocessing, cleaning and splitting, check out our tutorial on [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline).\n", "\n", "\n", "### Initializing the DocumentStore\n", "\n", "Initialize a DocumentStore to index your documents. A DocumentStore stores the Documents that the question answering system uses to find answers to your questions. In this tutorial, you'll be using the `InMemoryDocumentStore`." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "id": "CbVN-s5LO-qa" }, "outputs": [], "source": [ "from haystack.document_stores.in_memory import InMemoryDocumentStore\n", "\n", "document_store = InMemoryDocumentStore()" ] }, { "cell_type": "markdown", "metadata": { "id": "yL8nuJdWO-qa" }, "source": [ "> `InMemoryDocumentStore` is the simplest DocumentStore to get started with. It requires no external dependencies and it's a good option for smaller projects and debugging. But it doesn't scale up so well to larger Document collections, so it's not a good choice for production systems. To learn more about the different types of external databases that Haystack supports, see [DocumentStore Integrations](https://haystack.deepset.ai/integrations?type=Document+Store)." ] }, { "cell_type": "markdown", "metadata": { "id": "XvLVaFHTO-qb" }, "source": [ "The DocumentStore is now ready. Now it's time to fill it with some Documents." ] }, { "cell_type": "markdown", "metadata": { "id": "HryYZP9ZO-qb" }, "source": [ "### Fetch the Data\n", "\n", "You'll use the Wikipedia pages of [Seven Wonders of the Ancient World](https://en.wikipedia.org/wiki/Wonders_of_the_World) as Documents. We preprocessed the data and uploaded to a Hugging Face Space: [Seven Wonders](https://huggingface.co/datasets/bilgeyucel/seven-wonders). Thus, you don't need to perform any additional cleaning or splitting.\n", "\n", "Fetch the data and convert it into Haystack Documents:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "INdC3WvLO-qb", "outputId": "1af43d0f-2999-4de4-d152-b3cca9fb49e6" }, "outputs": [], "source": [ "from datasets import load_dataset\n", "from haystack import Document\n", "\n", "dataset = load_dataset(\"bilgeyucel/seven-wonders\", split=\"train\")\n", "docs = [Document(content=doc[\"content\"], meta=doc[\"meta\"]) for doc in dataset]" ] }, { "cell_type": "markdown", "metadata": { "id": "czMjWwnxPA-3" }, "source": [ "### Initalize a Document Embedder\n", "\n", "To store your data in the DocumentStore with embeddings, initialize a [SentenceTransformersDocumentEmbedder](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder) with the model name and call `warm_up()` to download the embedding model.\n", "\n", "> If you'd like, you can use a different [Embedder](https://docs.haystack.deepset.ai/docs/embedders) for your documents." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "EUmAH9sEn3R7", "outputId": "ee54b59b-4d4a-45eb-c1a9-0b7b248f1dd4" }, "outputs": [], "source": [ "from haystack.components.embedders import SentenceTransformersDocumentEmbedder\n", "\n", "doc_embedder = SentenceTransformersDocumentEmbedder(model=\"sentence-transformers/all-MiniLM-L6-v2\")\n", "doc_embedder.warm_up()" ] }, { "cell_type": "markdown", "metadata": { "id": "9y4iJE_SrS4K" }, "source": [ "### Write Documents to the DocumentStore\n", "\n", "Run the `doc_embedder` with the Documents. The embedder will create embeddings for each document and save these embeddings in Document object's `embedding` field. Then, you can write the Documents to the DocumentStore with `write_documents()` method." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 66, "referenced_widgets": [ "7d482188c12d4a7886f20a65d3402c59", "2a3ec74419ae4a02ac0210db66133415", "ddeff9a822404adbbc3cad97a939bc0c", "36d341ab3a044709b5af2e8ab97559bc", "88fc33e1ab78405e911b5eafa512c935", "91e5d4b0ede848319ef0d3b558d57d19", "d2428c21707d43f2b6f07bfafbace8bb", "7fdb2c859e454e72888709a835f7591e", "6b8334e071a3438397ba6435aac69f58", "5f5cfa425cac4d37b2ea29e53b4ed900", "3c59a82dac5c476b9a3e3132094e1702" ] }, "id": "ETpQKftLplqh", "outputId": "b9c8658c-90c8-497c-e765-97487c0daf8e" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Batches: 0%| | 0/5 [00:00 ⚠️ Notice that you used `sentence-transformers/all-MiniLM-L6-v2` model to create embeddings for your documents before. This is why you need to use the same model to embed the user queries." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "id": "LyJY2yW628dl" }, "outputs": [], "source": [ "from haystack.components.embedders import SentenceTransformersTextEmbedder\n", "\n", "text_embedder = SentenceTransformersTextEmbedder(model=\"sentence-transformers/all-MiniLM-L6-v2\")" ] }, { "cell_type": "markdown", "metadata": { "id": "0_cj-5m-O-qb" }, "source": [ "### Initialize the Retriever\n", "\n", "Initialize a [InMemoryEmbeddingRetriever](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever) and make it use the InMemoryDocumentStore you initialized earlier in this tutorial. This Retriever will get the relevant documents to the query." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "id": "-uo-6fjiO-qb" }, "outputs": [], "source": [ "from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever\n", "\n", "retriever = InMemoryEmbeddingRetriever(document_store)" ] }, { "cell_type": "markdown", "metadata": { "id": "6CEuQpB7O-qb" }, "source": [ "### Define a Template Prompt\n", "\n", "Create a custom prompt for a generative question answering task using the RAG approach. The prompt should take in two parameters: `documents`, which are retrieved from a document store, and a `question` from the user. Use the Jinja2 looping syntax to combine the content of the retrieved documents in the prompt.\n", "\n", "Next, initialize a [PromptBuilder](https://docs.haystack.deepset.ai/docs/promptbuilder) instance with your prompt template. The PromptBuilder, when given the necessary values, will automatically fill in the variable values and generate a complete prompt. This approach allows for a more tailored and effective question-answering experience." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "id": "ObahTh45FqOT" }, "outputs": [], "source": [ "from haystack.components.builders import ChatPromptBuilder\n", "from haystack.dataclasses import ChatMessage\n", "\n", "template = [\n", " ChatMessage.from_user(\n", " \"\"\"\n", "Given the following information, answer the question.\n", "\n", "Context:\n", "{% for document in documents %}\n", " {{ document.content }}\n", "{% endfor %}\n", "\n", "Question: {{question}}\n", "Answer:\n", "\"\"\"\n", " )\n", "]\n", "\n", "prompt_builder = ChatPromptBuilder(template=template)" ] }, { "cell_type": "markdown", "metadata": { "id": "HR14lbfcFtXj" }, "source": [ "### Initialize a ChatGenerator\n", "\n", "\n", "ChatGenerators are the components that interact with large language models (LLMs). Now, set `OPENAI_API_KEY` environment variable and initialize a [OpenAIChatGenerator](https://docs.haystack.deepset.ai/docs/openaichatgenerator) that can communicate with OpenAI GPT models. As you initialize, provide a model name:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "SavE_FAqfApo", "outputId": "1afbf2e8-ae63-41ff-c37f-5123b2103356" }, "outputs": [], "source": [ "import os\n", "from getpass import getpass\n", "from haystack.components.generators.chat import OpenAIChatGenerator\n", "\n", "if \"OPENAI_API_KEY\" not in os.environ:\n", " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter OpenAI API key:\")\n", "chat_generator = OpenAIChatGenerator(model=\"gpt-4o-mini\")" ] }, { "cell_type": "markdown", "metadata": { "id": "nenbo2SvycHd" }, "source": [ "> You can replace `OpenAIChatGenerator` in your pipeline with another `ChatGenerator`. Check out the full list of chat generators [here](https://docs.haystack.deepset.ai/docs/generators)." ] }, { "cell_type": "markdown", "metadata": { "id": "1bfHwOQwycHe" }, "source": [ "### Build the Pipeline\n", "\n", "To build a pipeline, add all components to your pipeline and connect them. Create connections from `text_embedder`'s \"embedding\" output to \"query_embedding\" input of `retriever`, from `retriever` to `prompt_builder` and from `prompt_builder` to `llm`. Explicitly connect the output of `retriever` with \"documents\" input of the `prompt_builder` to make the connection obvious as `prompt_builder` has two inputs (\"documents\" and \"question\").\n", "\n", "For more information on pipelines and creating connections, refer to [Creating Pipelines](https://docs.haystack.deepset.ai/docs/creating-pipelines) documentation." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "id": "f6NFmpjEO-qb", "outputId": "89fd1b48-5189-4401-9cf8-15f55c503676" }, "outputs": [], "source": [ "from haystack import Pipeline\n", "\n", "basic_rag_pipeline = Pipeline()\n", "# Add components to your pipeline\n", "basic_rag_pipeline.add_component(\"text_embedder\", text_embedder)\n", "basic_rag_pipeline.add_component(\"retriever\", retriever)\n", "basic_rag_pipeline.add_component(\"prompt_builder\", prompt_builder)\n", "basic_rag_pipeline.add_component(\"llm\", chat_generator)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", "🚅 Components\n", " - text_embedder: SentenceTransformersTextEmbedder\n", " - retriever: InMemoryEmbeddingRetriever\n", " - prompt_builder: ChatPromptBuilder\n", " - llm: OpenAIChatGenerator\n", "🛤️ Connections\n", " - text_embedder.embedding -> retriever.query_embedding (List[float])\n", " - retriever.documents -> prompt_builder.documents (List[Document])\n", " - prompt_builder.prompt -> llm.messages (List[ChatMessage])" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Now, connect the components to each other\n", "basic_rag_pipeline.connect(\"text_embedder.embedding\", \"retriever.query_embedding\")\n", "basic_rag_pipeline.connect(\"retriever\", \"prompt_builder\")\n", "basic_rag_pipeline.connect(\"prompt_builder.prompt\", \"llm.messages\")" ] }, { "cell_type": "markdown", "metadata": { "id": "6NqyLhx7O-qc" }, "source": [ "That's it! Your RAG pipeline is ready to generate answers to questions!" ] }, { "cell_type": "markdown", "metadata": { "id": "DBAyF5tVO-qc" }, "source": [ "## Asking a Question\n", "\n", "When asking a question, use the `run()` method of the pipeline. Make sure to provide the question to both the `text_embedder` and the `prompt_builder`. This ensures that the `{{question}}` variable in the template prompt gets replaced with your specific question." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 86, "referenced_widgets": [ "4e6e97b6d54f4f80bb7e8b25aba8e616", "1a820c06a7a049d8b6c9ff300284d06e", "58ff4e0603a74978a134f63533859be5", "8bdb8bfae31d4f4cb6c3b0bf43120eed", "39a68d9a5c274e2dafaa2d1f86eea768", "d0cfe5dacdfc431a91b4c4741123e2d0", "e7f1e1a14bb740d18827dd78bbe7b2e3", "3fda06f905b445a488efdd2dd08c0939", "2bc341a780f7498ba9cd475468841bb5", "d7218475e23b420a8c03d00ca4ab8718", "a694abaf765f4d1b82fa0138e59c6793" ] }, "id": "Vnt283M5O-qc", "outputId": "d2843a73-3ad5-4daa-8d1e-a58de7aa2bb0" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Batches: 100%|██████████| 1/1 [00:00<00:00, 1.77it/s]\n", "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n", "To disable this warning, you can either:\n", "\t- Avoid using `tokenizers` before the fork if possible\n", "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "The Colossus of Rhodes was a statue of the Greek sun-god Helios, standing approximately 70 cubits (about 33 meters or 108 feet) tall. Although no complete descriptions of its appearance exist, scholars believe it featured the following characteristics:\n", "\n", "1. **Facial Features**: The head of the statue likely had curly hair, with spikes resembling bronze or silver flames radiating outward. This style is similar to depictions found on contemporary Rhodian coins.\n", "\n", "2. **Posture**: While the exact pose is uncertain, it is suggested that the statue may have been constructed in a pose where Helios is depicted shielding his eyes with one hand, a common representation of someone looking toward the sun.\n", "\n", "3. **Construction Materials**: The structure was built using iron tie bars and brass plates, which formed the skin of the statue. The interior was filled with stone blocks.\n", "\n", "4. **Height and Scale**: The Colossus was positioned on a 15-metre-high (49-foot) pedestal, making it one of the tallest statues of the ancient world, towering over the harbor entrance.\n", "\n", "5. **Symbolic Representation**: The statue was meant to symbolize the victory and freedom of the Rhodians after successfully defending their city against an invader.\n", "\n", "Overall, the Colossus of Rhodes was an impressive and monumental statue designed to celebrate and symbolize the strength and resilience of the city of Rhodes.\n" ] } ], "source": [ "question = \"What does Rhodes Statue look like?\"\n", "\n", "response = basic_rag_pipeline.run({\"text_embedder\": {\"text\": question}, \"prompt_builder\": {\"question\": question}})\n", "\n", "print(response[\"llm\"][\"replies\"][0].text)" ] }, { "cell_type": "markdown", "metadata": { "id": "IWQN-aoGO-qc" }, "source": [ "Here are some other example questions to test:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "id": "_OHUQ5xxO-qc" }, "outputs": [], "source": [ "examples = [\n", " \"Where is Gardens of Babylon?\",\n", " \"Why did people build Great Pyramid of Giza?\",\n", " \"What does Rhodes Statue look like?\",\n", " \"Why did people visit the Temple of Artemis?\",\n", " \"What is the importance of Colossus of Rhodes?\",\n", " \"What happened to the Tomb of Mausolus?\",\n", " \"How did Colossus of Rhodes collapse?\",\n", "]" ] }, { "cell_type": "markdown", "metadata": { "id": "XueCK3y4O-qc" }, "source": [ "## What's next\n", "\n", "🎉 Congratulations! You've learned how to create a generative QA system for your documents with the RAG approach.\n", "\n", "If you liked this tutorial, you may also enjoy:\n", "- [Filtering Documents with Metadata](https://haystack.deepset.ai/tutorials/31_metadata_filtering)\n", "- [Preprocessing Different File Types](https://haystack.deepset.ai/tutorials/30_file_type_preprocessing_index_pipeline)\n", "- [Creating a Hybrid Retrieval Pipeline](https://haystack.deepset.ai/tutorials/33_hybrid_retrieval)\n", "\n", "To stay up to date on the latest Haystack developments, you can [subscribe to our newsletter](https://landing.deepset.ai/haystack-community-updates) and [join Haystack discord community](https://discord.gg/haystack).\n", "\n", "Thanks for reading!" ] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.6" }, "orig_nbformat": 4, "vscode": { "interpreter": { "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6" } } }, "nbformat": 4, "nbformat_minor": 0 }