UseCases/Chat_wth_docs_GenAI/Chat_with_docs_OpenSource_GenAI_Python.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "id": "38bc532d-c78c-4c89-b191-242da0733f39",
            "metadata": {},
            "source": [
                "<header>\n",
                "   <p  style='font-size:36px;font-family:Arial; color:#F0F0F0; background-color: #00233c; padding-left: 20pt; padding-top: 20pt;padding-bottom: 10pt; padding-right: 20pt;'>\n",
                "       Chat with documents using Generative AI with Vantage\n",
                "  <br>\n",
                "       <img id=\"teradata-logo\" src=\"https://storage.googleapis.com/clearscape_analytics_demo_data/DEMO_Logo/teradata.svg\" alt=\"Teradata\" style=\"width: 125px; height: auto; margin-top: 20pt;\">\n",
                "    </p>\n",
                "</header>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "fff71661-19b4-423a-867a-7c815b064c81",
            "metadata": {},
            "source": [
                "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>Introduction:</b></p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In the Chat with documentation system using Generative AI demo, the combination of <b>RAG, Langchain, and LLM models</b> allows users to ask queries in layman's terms, retrieve relevant information from the Vector store, and generate accurate and concise answers based on the retrieved data. This integration of retrieval-based and generative-based approaches provides a powerful tool for extracting knowledge from structured sources and delivering user-friendly responses.</p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In this demo we will build Chatbot using Panel (for chat UI), LangChain, a powerful library for working with LLMs like GPT-3.5, GPT-4, Bloom, etc. and JumpStart in ClearScape notebooks, a system is built where users can ask business questions in natural English and receive answers with data drawn from the relevant databases.</p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The following diagram illustrates the architecture.</p>\n",
                "\n",
                "<center><img src=\"images/header2.png\" alt=\"architecture\"  width=800 height=800/></center>\n",
                "\n",
                "\n",
                "<br>\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Before going any farther, let's get a better understanding of RAG, LangChain, and LLM.</p>\n",
                "\n",
                "<ol style = 'font-size:16px;font-family:Arial;color:#00233C'><b><li> Retrieval-Augmented Generation (RAG):</li></b></ol>\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'> &emsp;  &emsp;RAG is a framework that combines the strengths of retrieval-based and generative-based approaches in question-answering systems.It utilizes both a retrieval model and a generative model to generate high-quality answers to user queries. The retrieval model is responsible for retrieving relevant information from a knowledge source, such as a database or documents. The generative model then takes the retrieved information as input and generates concise and accurate answers in natural language.</p>\n",
                "\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>A typical RAG (Retrieval-and-Generation) application has two main components:</p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Indexing:</b> a pipeline for ingesting data from a source and indexing it. This usually happens offline. The indexing process involves several steps, including loading the data, splitting it into smaller chunks, and storing and indexing the splits. This is often done using a VectorStore and Embeddings model.</p>\n",
                "    \n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Retrieval and generation:</b> the actual RAG chain, which takes the user query at run time and retrieves the relevant data from the index, then passes that to the model. The retrieval process involves searching the index for the most relevant data based on the user query, and then passing that data to the model for generation.</p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The most common full sequence from raw data to answer looks like:</p>\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Indexing</b></p>\n",
                "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
                "    <li><b>Load:</b> Load: First we need to load our data. We'll use <code>PyPDFLoader</code> for this.</li>\n",
                "    <li><b>Split:</b> Text splitters break large Documents into smaller chunks. This is useful both for indexing data and for passing it in to a model, since large chunks are harder to search over and won't in a model's finite context window. Here, our pdf document will be splits into pages.</li>\n",
                "    <li><b>Store:</b> We need somewhere to store and index our splits, so that they can later be searched over. This is often done using a VectorStore and Embeddings model</li>\n",
                "    </ul>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The following diagram illustrates the architecture of load, split and store.</p>\n",
                "\n",
                "<center><img src=\"images/rag_load_store.png\" alt=\"rag indexing architecture\" width=800 height=600/></center>\n",
                "<center>image source: <a href=\"https://python.langchain.com/docs/use_cases/question_answering/\">langchain.com</a></center>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Retrieval and generation</b></p>\n",
                "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
                "    <li><b>Retrieval:</b> At run time, the user enters a query, which is passed to the model for retrieval. The model searches the index for the most relevant data based on the user query, and returns the results.</li>\n",
                "    <li><b>Generation:</b> Finally, the model generates an answer based on the retrieved data. The answer is then presented to the user.</li>\n",
                "    </ul>\n",
                "    \n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The following diagram illustrates the architecture of retrieval and generation.</p>\n",
                "<center><img src=\"images/rag_retrieval_generation.png\" alt=\"retrieval generation architecture\" width=800 height=600/></center>\n",
                "<center>image source: <a href=\"https://python.langchain.com/docs/use_cases/question_answering/\">langchain.com</a></center>\n",
                "\n",
                "<ol style = 'font-size:16px;font-family:Arial;color:#00233C' start=\"2\"><b><li> Langchain:</li></b></ol>\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'> &emsp;  &emsp; LangChain is a framework that facilitates the integration and chaining of large language models with other tools and sources to build more sophisticated AI applications. LangChain does not serve its own LLMs; instead, it provides a standard way of communicating with a variety of LLMs, including those from OpenAI and HuggingFace. LangChain accelerates the development of AI applications with building blocks. We learn the leverage the following building blocks in this notebook:</p>\n",
                " \n",
                "<ol style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
                "    <li> <b> LLMs</b> – LangChain's <code>llm</code> class is designed to provide a standard interface for all LLM it supports.   </li>\n",
                "    <li> <b> PromptTemplate</b>  - LangChain’s <code>PromptTemplate</code> class are predefined structures for generating prompts for LLM’s. They can be reused across different LLM's.</li>\n",
                "    <li> <b> Chains</b> – When we build complex AI applications, we may need to combine multiple calls to LLM’s and to other components  LangChain’s <code>chain</code> class allows us to link calls to LLM’s and components. The most common type of chaining in any LLM application is combining a prompt template with an LLM and optionally an output parser. </li>\n",
                "</ol>\n",
                "\n",
                "<ol style = 'font-size:16px;font-family:Arial;color:#00233C' start=\"3\"><b><li> LLM Models (Large Language Models):</li></b></ol>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'> &emsp;  &emsp; LLM models refer to the large-scale language models that are trained on vast amounts of text data.\n",
                "These models, such as GPT-3 (Generative Pre-trained Transformer 3),  GPT-3.5, GPT-4, HuggingFace BLOOM, LLaMA, Google's FLAN-T5, etc. are capable of generating human-like text responses. LLM models have been pre-trained on diverse sources of text data, enabling them to learn patterns, grammar, and context from a wide range of topics. They can be fine-tuned for specific tasks, such as question-answering, natural language understanding, and text generation.\n",
                "LLM models have achieved impressive results in various natural language processing tasks and are widely used in AI applications for generating human-like text responses.</p>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "2ca49b9d-df0b-400a-acf8-0252ab8a2618",
            "metadata": {},
            "source": [
                "<p style = 'font-size:16px;font-family:Arial;color:#00233c'><b>Steps in the analysis:</b></p>\n",
                "<ol style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
                "    <li>Configuring the environment</li>\n",
                "    <li>Connect to Vantage</li>\n",
                "    <li>Data Exploration</li>\n",
                "    <li>LLM</li>\n",
                "    <li>Chat with documents</li>\n",
                "</ol>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "a833b5bf-74af-42be-8543-3782e1da95dc",
            "metadata": {},
            "source": [
                "<hr style='height:2px;border:none;background-color:#00233C;'>\n",
                "<b style = 'font-size:28px;font-family:Arial;color:#00233c'>1. Configuring the environment</b>"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "2f6027a7-888d-441f-abc7-a6ea1c45f0a6",
            "metadata": {},
            "outputs": [],
            "source": [
                "%%capture\n",
                "# '%%capture' suppresses the display of installation steps of the following packages\n",
                "!pip install -r requirements.txt --quiet"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "86b160ce-5ace-4116-86b6-394d6502553b",
            "metadata": {},
            "source": [
                "<div class=\"alert alert-block alert-info\">\n",
                "    <p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Note: </b><i>The above statements will install the required libraries to run this demo. Be sure to restart the kernel after executing the above lines to bring the installed libraries into memory. The simplest way to restart the Kernel is by typing zero zero: <b> 0 0</b></i></p>\n",
                "    </div>\n",
                "    \n",
                "<div class=\"alert alert-block alert-info\">\n",
                "    <p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Note: </b><i>To ensure that the Chatbot interface reflects the latest changes, please reload the page by clicking the 'Reload' button or pressing F5 on your keyboard for <b>first-time only</b> This will update the notebook with the latest modifications, and you'll be able to interact with the Chatbot using the new libraries.</i></p>\n",
                "    </div>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "61067c88-2e9a-4c92-985b-34dc4ab74a13",
            "metadata": {},
            "source": [
                "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
                "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>1.1 Import the required libraries</b></p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Here, we import the required libraries, set environment variables and environment paths (if required).</p>"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "ea50a4aa-1211-44fc-8166-317c35253207",
            "metadata": {},
            "outputs": [],
            "source": [
                "import os\n",
                "\n",
                "\n",
                "# teradata lib\n",
                "from teradataml import *\n",
                "\n",
                "# LLM\n",
                "from operator import itemgetter\n",
                "from langchain.chat_models import ChatOpenAI\n",
                "from langchain.prompts import ChatPromptTemplate\n",
                "from langchain.schema.output_parser import StrOutputParser\n",
                "from langchain.schema.runnable import RunnablePassthrough\n",
                "from langchain.prompts.prompt import PromptTemplate\n",
                "from langchain.vectorstores import FAISS\n",
                "from langchain.embeddings.openai import OpenAIEmbeddings\n",
                "from langchain.schema import format_document\n",
                "from langchain.schema.runnable import RunnableMap\n",
                "from typing import List, Tuple\n",
                "\n",
                "# Suppress warnings\n",
                "warnings.filterwarnings(\"ignore\")\n",
                "display.max_rows = 5"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "a59718f8-7af4-4d1a-abc7-a860eb7cbae3",
            "metadata": {},
            "source": [
                "<hr style='height:2px;border:none;background-color:#00233C;'>\n",
                "<b style = 'font-size:28px;font-family:Arial;color:#00233c'>2. Connect to Vantage and OpenAI</b>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "4153b889-0924-4e0f-acc8-6b0d440c77b3",
            "metadata": {},
            "source": [
                "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>2.1 Get the OpenAI API key</b></p>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "73e10add-4b3c-4fc5-9359-0b44737bba0f",
            "metadata": {},
            "source": [
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In order to utilize this demo, you will need an OpenAI API key. If you do not have one, please refer to the instructions provided in this guide to obtain your OpenAI API key: </p>\n",
                "\n",
                "[Openai_setup_api_key_guide](..//Openai_setup_api_key/Openai_setup_api_key.md)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "afd64269-9393-434a-ac26-0b8386fc0a38",
            "metadata": {},
            "outputs": [],
            "source": [
                "# enter your openai api key\n",
                "api_key = input(prompt=\"\\n Please Enter OpenAI API key: \")"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "b22db506-84a7-406b-be9f-9fe69d268ba4",
            "metadata": {},
            "source": [
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Begin running steps with Shift + Enter keys. </p>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "490f3fe2-ed63-4838-bb19-4c0d0157453d",
            "metadata": {},
            "source": [
                "<hr style='height:2px;border:none;background-color:#00233C;'>\n",
                "<b style = 'font-size:28px;font-family:Arial;color:#00233c'>3. Data Exploration</b>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The Chat with documentation demo aims to demonstrate how users can interact with documents such as insurance policy wordings, invoices, and other similar documents through a conversational interface.</p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The Traveler Easy Single Trip - International insurance policy is a comprehensive travel insurance plan that provides cover for a wide range of risks, including medical expenses, trip cancellation, loss of luggage, and personal accident. The policy is designed to be affordable and flexible, and it can be purchased online or over the phone.<p/>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "5c1f35d2-fc39-4a0a-9008-3850cd58e50e",
            "metadata": {},
            "source": [
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The source data from <a href=\"https://axa-com-my.cdn.axa-contento-118412.eu/axa-com-my/3d2f84a5-42b9-459b-911a-710546df0633_Policy+wording+-+SmartTraveller+Easy+Single+Trip+-+International+%280820%29.pdf\">AXA</a> is loaded in FAISS as Vector Database.</p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Now, let's use <code>PyPDFLoader</code> library to read the pdf document and split it into pages.</p>"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "b7718aa2-2787-471d-85f4-fd592b151d08",
            "metadata": {},
            "outputs": [],
            "source": [
                "from langchain.document_loaders import PyPDFLoader\n",
                "\n",
                "loader = PyPDFLoader(\"data/SmartTraveller_International.pdf\")\n",
                "pages = loader.load_and_split()"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "b0d6d954-b72f-41c3-9f8c-bc15cf74d672",
            "metadata": {},
            "source": [
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>There are 24 pages that describe the policy in detail.</p>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "89367d27-22ca-4a0e-b299-0f8e1a669157",
            "metadata": {},
            "source": [
                "<hr style='height: 2px;border:none;background-color:#00233C;'>\n",
                "<b style = 'font-size:28px;font-family:Arial;color:#00233c'>4. LLM </b>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "680228a2-3bfa-44f4-8a02-b73fa8bf8e95",
            "metadata": {},
            "source": [
                "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>4.1 Define LLM model</b></p>  \n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In OpenAI's language models, the <b>temperature</b> parameter controls the randomness of the generated text. It affects the diversity and creativity of the model's responses. It is always a number between 0 and 1. A temperature of 0 means the responses will be very straightforward, almost deterministic (meaning you almost always get the same response to a given prompt). A temperature of 1 means the responses can vary wildly.</p>\n",
                "\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>A higher temperature value, such as 1.0, increases the randomness and diversity of the generated output. This can lead to more varied and surprising responses, but it may also result in less coherence and occasional nonsensical outputs. A higher temperature means that the model might select a word with slightly lower probability, leading to more variation, randomness and creativity. A very high temperature therefore increases the risk of <b>hallucination</b>, meaning that the model starts selecting words that will make no sense or be off-topic.</p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>On the other hand, a lower temperature value, such as 0.2 or below, reduces randomness and makes the model's output more focused and deterministic. The generated text is likely to be more conservative, sticking closely to patterns observed in the training data. A temperature of 0 means roughly that the model will always select the highest probability word.</p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Choosing an appropriate temperature value depends on the desired output. Higher temperatures can be useful for creative tasks or brainstorming, while lower temperatures are preferred when you need more control over the output, such as when generating specific responses or following a particular style.</p>"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "fbceafb4-793b-4d62-ab22-c9d49ac9a759",
            "metadata": {},
            "outputs": [],
            "source": [
                "from langchain.llms import OpenAI\n",
                "\n",
                "# OpenAI API\n",
                "os.environ[\"OPENAI_API_KEY\"] = api_key\n",
                "\n",
                "# call open AI model - api\n",
                "llm = OpenAI(temperature=0)"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "5753a719-44da-460e-91c4-a17b665b25b2",
            "metadata": {},
            "source": [
                "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
                "\n",
                "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b> 4.2 Create a Vector store</b></p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Vector stores are a powerful tool for storing and searching unstructured data in Language Large Models (LLMs). By embedding unstructured data into numerical vectors, vector stores allow you to easily store and search through vast amounts of data, such as text documents, images, or videos. </p>\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>One of the key benefits of using a vector store in LLM applications is that it allows you to perform near-instant searches. This means you can quickly find the information you need, without having to manually sift through each piece of data.</p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Vector stores also use advanced similarity algorithms to retrieve the most relevant embedding vectors for your query. This means you can be sure that you're getting the most relevant results, even when dealing with large amounts of data.But that's not all. Vector stores also allow you to store and search through unstructured data in a way that is highly scalable. This means you can easily handle large amounts of data, without having to worry about performance issues.</p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>One of the key aspects of working with vector stores is creating the vectors to put in them. This is usually done via embeddings, which are numerical representations of text or other unstructured data. Therefore, it is recommended that you familiarize yourself with the text embedding model interfaces before diving into this.</p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In this demo, we will use the OpenAI embeddings method to generate the embeddings. OpenAI embeddings are a type of word embedding that can be used to represent products in a way that captures their semantic meaning. To generate embeddings for a insurance document, we will use the pages collected in the previous step by <code>PyPDFLoader</code>. We will use the OpenAI Embeddings API to generate embeddings for entire document. Please refer to the <a href=\"https://platform.openai.com/docs/guides/embeddings\"> Embeddings documentation</a> for more information about embeddings and types of models available.</p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The OpenAI Embeddings API takes a text string as input and returns a vector of numbers that represent the embedding. The length of the vector depends on the model that you are using. For example, the text-embedding-ada-002 model returns a vector of 1536 numbers.</p>"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "8be9fa81-82de-43c4-a44c-4a1f0df7dd2d",
            "metadata": {},
            "outputs": [],
            "source": [
                "vectorstore = FAISS.from_documents(\n",
                "    pages, OpenAIEmbeddings(model=\"text-embedding-ada-002\")\n",
                ")\n",
                "\n",
                "vectorstore.save_local(\"./data/faiss_index\")\n",
                "\n",
                "retriever = vectorstore.as_retriever(search_type=\"similarity\", search_kwargs={\"k\": 2})"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "b6ea91ff-1832-43a6-96aa-6c0cd7e28f53",
            "metadata": {},
            "source": [
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>A <code>retriever</code>  is an interface that allows you to retrieve documents based on an unstructured query. It is more general than a vector store, as it does not need to store documents, only retrieve them. Vector stores can be used as the backbone of a retriever, but there are other types of retrievers as well.One of the most common types of search used in vector stores is similarity search, which compares the similarity between the query and the stored documents to retrieve the most relevant results. This is the type of search used in our vector store retriever.</p>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "3a32ecf7-575b-460b-ab16-773a7d7b9e8c",
            "metadata": {},
            "source": [
                "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
                "\n",
                "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b> 4.3 Create a Prompt templates and Chain</b></p>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><code>PromptTemplate</code> in Language Large Models (LLMs) are pre-defined templates that guide the user in generating prompts for the model. These templates provide a structure for the user to input specific information, such as <b>topic, tone,</b> and <b>style</b>, to help the model generate more accurate and relevant responses. By using prompt templates, users can create more effective prompts and improve the quality of the model's responses. A template may include instructions, few-shot examples, and specific context and questions appropriate for a given task.</p>"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "ee40e7af-b23b-456e-be13-e9a3a382d249",
            "metadata": {},
            "outputs": [],
            "source": [
                "que_template = \"\"\"Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question, in its original language.\n",
                "\n",
                "If user will start by greeting, just say, \"Hey there! 😊 Welcome to our chatbot!\"\n",
                "\n",
                "Chat History:\n",
                "{chat_history}\n",
                "Follow Up Input: {question}\n",
                "Standalone question:\"\"\"\n",
                "QUESTION_PROMPT = PromptTemplate.from_template(que_template)\n",
                "\n",
                "ans_template = \"\"\"Answer the question based only on the following context:\n",
                "{context}\n",
                "\n",
                "If user will start by greeting, just say, \"Hey there! 😊 Welcome to our chatbot!\"\n",
                "\n",
                "Question: {question}\n",
                "\"\"\"\n",
                "ANSWER_PROMPT = ChatPromptTemplate.from_template(ans_template)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "3126a0a5-ea9a-40f2-a3d5-7cce0e0e54dc",
            "metadata": {},
            "outputs": [],
            "source": [
                "DEFAULT_DOCUMENT_PROMPT = PromptTemplate.from_template(template=\"{page_content}\")\n",
                "\n",
                "\n",
                "def combine_documents(\n",
                "    docs, document_prompt=DEFAULT_DOCUMENT_PROMPT, document_separator=\"\\n\\n\"\n",
                "):\n",
                "    doc_strings = [format_document(doc, document_prompt) for doc in docs]\n",
                "    return document_separator.join(doc_strings)\n",
                "\n",
                "\n",
                "def format_chat_history(chat_history: List[Tuple]) -> str:\n",
                "    buffer = \"\"\n",
                "    for dialogue_turn in chat_history:\n",
                "        human = \"Human: \" + dialogue_turn[0]\n",
                "        ai = \"Assistant: \" + dialogue_turn[1]\n",
                "        buffer += \"\\n\" + \"\\n\".join([human, ai])\n",
                "    return buffer"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "c41ea900-7703-4646-a3dd-fdbf93f85a0f",
            "metadata": {},
            "source": [
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We have designed this demo to handle follow-up questions with ease, even when they contain references to past chat history. </p>\n",
                "    \n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>For example, if you ask <b>\"Does this policy cover  Loss of or Damage to the Insured’s Articles?\"</b> and then follow up with <b>\"what is the reimbursement limit per Baggage?\"</b>, our system will understand the context and provide an appropriate response.</p>\n",
                "    \n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Here's how it works: we add an additional step prior to retrieval that combines the chat history and the question into a standalone question. This allows us to perform the standard retrieval steps of looking up relevant documents from the retriever and passing those documents and the question into a question answering chain to return a response.</p>"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "2efd2e71-b3c5-4675-96b8-dac85fdb4b75",
            "metadata": {},
            "outputs": [],
            "source": [
                "inputs = RunnableMap(\n",
                "    standalone_question=RunnablePassthrough.assign(\n",
                "        chat_history=lambda x: format_chat_history(x[\"chat_history\"])\n",
                "    )\n",
                "    | QUESTION_PROMPT\n",
                "    | ChatOpenAI(temperature=0)\n",
                "    | StrOutputParser(),\n",
                ")\n",
                "context = {\n",
                "    \"context\": itemgetter(\"standalone_question\") | retriever | combine_documents,\n",
                "    \"question\": lambda x: x[\"standalone_question\"],\n",
                "}\n",
                "conversational_qa_chain = inputs | context | ANSWER_PROMPT | ChatOpenAI()"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "a0fe1c6a-191a-4dc7-9b3f-7c282ec94541",
            "metadata": {},
            "source": [
                "<hr style='height: 2px;border:none;background-color:#00233C;'>\n",
                "<b style = 'font-size:28px;font-family:Arial;color:#00233c'>5. Chat with documents</b>\n",
                "\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Our chatbot is now ready. You can add questions to ask from the insurance document in the chatbot user interface (UI) that opens in the cell below.</p>"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "2150c0cb-e9b5-4ec5-a521-19d47819996e",
            "metadata": {},
            "outputs": [],
            "source": [
                "import panel as pn\n",
                "\n",
                "pn.extension(design=\"material\")\n",
                "\n",
                "# keep chat history\n",
                "chat_history = []\n",
                "\n",
                "\n",
                "# keep last k chats\n",
                "def get_chat_window(k=3):\n",
                "    return chat_history[-k:]\n",
                "\n",
                "\n",
                "# panel callback function\n",
                "def callback(contents, user, instance):\n",
                "    result = conversational_qa_chain.invoke(\n",
                "        {\"question\": contents, \"chat_history\": get_chat_window()}\n",
                "    ).content\n",
                "    chat_history.append((contents, result))\n",
                "    return result\n",
                "\n",
                "\n",
                "pn.chat.ChatInterface(\n",
                "    callback=callback,\n",
                "    show_rerun=False,\n",
                "    show_undo=False,\n",
                "    show_clear=False,\n",
                "    width=800,\n",
                "    height=400,\n",
                ")"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "c8490cae-814e-4e83-a615-7e511ac33a33",
            "metadata": {},
            "source": [
                "<div class=\"alert alert-block alert-info\">\n",
                "    <p style = 'font-size:16px;font-family:Arial;color:#00233C'>If the chatbot didn't work when you pressed ENTER, on your first time using this demo on your environment, did you use F5 to reload the site? See instructions at the top of the notebook.\n",
                "If you asked a question and got no response after a few minutes, it is possible that you will need to type 0 0 to restart the kernel and re-run the demo. Questions outside the model seem to confuse the chatbot.</p></div>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "50b8b817-c0ff-42b3-a651-c8268ac40942",
            "metadata": {},
            "source": [
                "<hr style='height: 2px;border:none;background-color:#00233C;'>\n",
                "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In conclusion, the Chat with documentation demo is important because it showcases the potential of conversational interfaces to facilitate user interaction with various types of documents. This technology can make it easier for users to access and understand complex information, such as insurance policy wordings and invoices. </p>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "e5463848-592f-4321-b852-287e133872dd",
            "metadata": {},
            "source": [
                "<footer style=\"padding-bottom:35px; background:#f9f9f9; border-bottom:3px solid #00233C\">\n",
                "    <div style=\"float:left;margin-top:14px\">ClearScape Analytics™</div>\n",
                "    <div style=\"float:right;\">\n",
                "        <div style=\"float:left; margin-top:14px\">\n",
                "            Copyright © Teradata Corporation - 2023. All Rights Reserved\n",
                "        </div>\n",
                "    </div>\n",
                "</footer>"
            ]
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3 (ipykernel)",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.9.10"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 5
}