UseCases/Natural_Language_Processing/Natural_Language_Processing_Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9abfac3c-6662-4b57-b5a8-82f0f6505d71",
   "metadata": {},
   "source": [
    "<header>\n",
    "   <p  style='font-size:36px;font-family:Arial; color:#F0F0F0; background-color: #00233c; padding-left: 20pt; padding-top: 20pt;padding-bottom: 10pt; padding-right: 20pt;'>\n",
    "       Natural Languauge Processing in Vantage\n",
    "  <br>\n",
    "       <img id=\"teradata-logo\" src=\"https://storage.googleapis.com/clearscape_analytics_demo_data/DEMO_Logo/teradata.svg\" alt=\"Teradata\" style=\"width: 125px; height: auto; margin-top: 20pt;\">\n",
    "    </p>\n",
    "</header>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b093809e-594f-4167-b142-d79d345ca4a8",
   "metadata": {},
   "source": [
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>This is a Beta Release of this notebook!</b></p>   \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>   \n",
    "This should be considered a Beta Release.  Every effort has been made to ensure it's functionality and completeness.  If you discover any issue with any part of this notebook, your suggestions will be reviewed and, if approved, merged in the next round of notebook commits.   \n",
    "<br>   \n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>How to submit a Review of this Notebook</b></p>   \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>   \n",
    "First, please read through and execute the notebook.   \n",
    "<br> \n",
    "If you think the notebook is fine as it is, great.  Please send us a quick email and let us know. When the notebook is taken out of beta, this paragraph will be removed and the Beta designation the index page will be removed.   \n",
    "<br> \n",
    "If you find something that you question or you think needs to be changed or if you think it's great as it is:   \n",
    "</p>       \n",
    "<ol style = 'font-size:16px;font-family:Arial;color:#00233C'>   \n",
    "  <li>Send us an email: <a href=\"mailto:ClearScapeAnalyticsNotebookReviews@Teradata.com?subject=Beta Notebook Review\">Click here.</a></li>   \n",
    "  <li>Please include the name of the notebook in email.</li>   \n",
    "  <li>Include a screen shot or a clear description of the section in the notebook you would like us to look at.</li>   \n",
    "  <li>Paste the screen shot or enter the description of the section into the email.</li>   \n",
    "  <li>Describe how you would change it.</li>   \n",
    "  <li>Send It!</li>   \n",
    "</ol>   \n",
    "</p>   \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>If we have any follow-up questions, we'll reach out to you at the return email address. </p>   \n",
    "<hr style=\"height:4px;border:none;background-color:#00233C;\"> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "507bfa51-059d-49a3-91d0-6a1de1d9f510",
   "metadata": {},
   "source": [
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>Introduction</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "Natural Language Processing (NLP) involves teaching computers to understand, interpret, and generate human language, just like people do. It's about enabling computers to read and understand text, so they can perform tasks that involve language, such as answering questions, understanding customer feedback, or even generating human-like responses.<br>Think of NLP as a translator between humans and computers. It allows computers to analyze and make sense of text data in a way that's meaningful for businesses. There are many uses of NLP that can be used in business for example <br><b>Customer Insights</b>: NLP helps businesses understand what their customers are saying across different channels like emails, reviews, or social media. It can analyze this text to identify trends, sentiments, and common issues, helping companies tailor their products and services to meet customer needs better.<br><b>Automated Support</b>: NLP powers chatbots and virtual assistants that can understand and respond to customer queries in real-time. These assistants can handle routine inquiries, provide product recommendations, or even troubleshoot problems, freeing up human agents for more complex tasks.<br><b>Information Extraction</b>: NLP can extract valuable information from unstructured text data, such as contracts, legal documents, or research papers. It helps businesses quickly find relevant information, identify key insights, and make informed decisions based on this data.<br><b>Personalization</b>: By analyzing customer interactions and preferences expressed in text, NLP enables businesses to personalize their marketing messages, offers, and user experiences. This personalized approach can lead to higher customer engagement and loyalty.<br><br>In essence, NLP empowers businesses to leverage the power of language to improve customer experiences, streamline operations, and drive better decision-making. \n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Business Values</b></p>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>Efficiency: Automating sentiment analysis saves time and resources compared to manual analysis.</li>\n",
    "    <li>Insights: Gain valuable insights into customer sentiment and preferences to drive strategic decision-making.</li>\n",
    "    <li>Proactive Response: Identify and address customer concerns and issues in real-time to improve customer satisfaction and loyalty.</li>\n",
    "    <li>Competitive Advantage: Stay ahead of competitors by continuously monitoring and adapting to changing customer sentiments and market trends.</li>\n",
    " </ul>\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Why Vantage?</b></p>  \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Natural Language Processing deals with huge amounts of data; doing pre-processing and using Vantage's InDB text processing functions saves time and can be easily scaled to meet business needs. Moreover, using Clearscape Analytics it is very easy to integrate widely used 3rd party LLM models like GPT with trusted business data.<br>In this demo we will process the comments received by a retail store using Vantage's InDb functions.  </p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a047ba1-e8ce-4228-83c8-fe3a6f8379fc",
   "metadata": {},
   "source": [
    "<hr style='height:2px;border:none;background-color:#00233C;'>\n",
    "<b style = 'font-size:20px;font-family:Arial;color:#00233c'>1. Configuring the environment</b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9290b9db-a1cc-43e3-8639-3a7dd8dfd200",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "# '%%capture' suppresses the display of installation steps of the following packages\n",
    "!pip install -r requirements.txt --quiet"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef6f671b-a24c-4911-87a8-10cd17fadfb1",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "    <p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Note: </b><i>The above statements will install the required libraries to run this demo. Be sure to restart the kernel after executing the above lines to bring the installed libraries into memory. The simplest way to restart the Kernel is by typing zero zero: <b> 0 0</b></i></p>\n",
    "    </div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63fd7658-b33c-481f-80ee-0beff2ee2663",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>2. Connect to Vantage</b></p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10130700-f95c-4832-8db7-39f21a679f9e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Suppress warnings\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\", category=DeprecationWarning) \n",
    "\n",
    "\n",
    "#import libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt \n",
    "import getpass\n",
    "\n",
    "import timeit\n",
    "import tqdm\n",
    "from tqdm.notebook import *\n",
    "tqdm_notebook.pandas()\n",
    "\n",
    "# teradata lib\n",
    "from teradataml import *\n",
    "import teradataml\n",
    "from teradataml import configure\n",
    "from teradataml.analytics.valib import *\n",
    "configure.val_install_location = \"val\"\n",
    "\n",
    "from utils.sql_helper_func import *\n",
    "from utils.opensource_helper_func import *\n",
    "\n",
    "display.max_rows = 5\n",
    "display.print_sqlmr_query=False\n",
    "display.suppress_vantage_runtime_warnings=True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0969ef7-3e8d-4e11-a6ea-e6e5dcf11ff6",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We will be prompted to provide the password. We will enter the password, press the Enter key, and then use the down arrow to go to the next cell.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0f334bfe-a214-49ec-86d8-06b027a5a2c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "%run -i ../startup.ipynb\n",
    "eng = create_context(host = 'host.docker.internal', username = 'demo_user', password = password)\n",
    "print(eng)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5f822383-bb3c-447f-9a7e-730f98e74872",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "execute_sql('''SET query_band='DEMO=Natural_Language_Processing_Python.ipynb;' UPDATE FOR SESSION; ''')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e0175c5-820e-4a8f-9353-6308c63f7715",
   "metadata": {},
   "source": [
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'> <b>Getting Data for This Demo</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We have provided data for this demo on cloud storage. We have the option of either running the demo using foreign tables to access the data without using any storage on our environment or downloading the data to local storage, which may yield somewhat faster execution. However, we need to consider available storage. There are two statements in the following cell, and one is commented out. We may switch which mode we choose by changing the comment string.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b1a4c01-edd1-4c06-aa47-5451df2d0f42",
   "metadata": {},
   "outputs": [],
   "source": [
    "%run -i ../run_procedure.py \"call get_data('DEMO_Retail_local');\"\n",
    " # takes about 2 minute 30 seconds, estimated space: 90 MB\n",
    "#%run -i ../run_procedure.py \"call get_data('DEMO_Retail_cloud');\" \n",
    "# takes about 30 seconds, estimated space: 0 MB"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6451ecbf-6b2f-4193-b44f-eb472a0dbc7f",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Optional step – We should execute the below step only if we want to see the status of databases/tables created and space used.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d650f5c-006a-4fb9-b6d7-1e705511b0da",
   "metadata": {},
   "outputs": [],
   "source": [
    "%run -i ../run_procedure.py \"call space_report();\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1d38a9c-a891-45be-90b1-164085dd4597",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>3. Data Exploration</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2266bc7e-138a-408d-a772-66f0ca5ddbea",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Please check the db version should be above 17.20.03.21 for the functions to work correctly. If the database version is less than that, please create a new VM in Clearscape Analytics Experience.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a8c997c-804b-4112-aa2a-6c1f6994cef7",
   "metadata": {},
   "outputs": [],
   "source": [
    "configure.database_version"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1c0c277-c6f0-4365-bc51-c498f1bf771b",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Let us start by creating a \"Virtual DataFrame\" that points directly to the dataset in Vantage.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49f69153-cd20-43bf-bd67-a55c888a75a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "tdf_reviews = DataFrame('\"DEMO_Retail\".\"Web_Comment\"')\n",
    "tdf_reviews"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f24ad14a-ebf9-4937-94e2-d72d8f59f8d1",
   "metadata": {},
   "outputs": [],
   "source": [
    "tdf_reviews.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b6b76d1-eb8a-4992-8159-e1d8f10bf337",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We have around 23k comments in our dataset. Let us first remove the null comments.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b394a7f4-5a97-4978-a1bd-20710b937bcb",
   "metadata": {},
   "outputs": [],
   "source": [
    "tdf_nonull = tdf_reviews[tdf_reviews.comment_text.isnull() == 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "702529d6-fa8c-4630-9b8f-c16d7b2977fd",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'> For demo purpose we will use 5k comments for our analysis.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0f4fb627-0fa4-4f60-af10-5db210c85b61",
   "metadata": {},
   "outputs": [],
   "source": [
    "tdf_sample = tdf_nonull.iloc[:5000, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ea8d513-49b7-412e-a6bd-f2e7a4d0a204",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>4. Sentiment Extraction</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18ba237e-bb17-44c0-9a23-bce43a0dbcb2",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Sentiment Extraction is the process of analyzing large volumes of text to determine whether it expresses a positive, negative, or neutral sentiment.<br> Clearscape Analytics SentimentExtractor \n",
    "uses a dictionary model to extract the sentiment (positive, negative, or neutral) of each input document or sentence. The  dictionary model consists of WordNet( a lexical database of the English language).The function handles negated sentiments as follows:<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>-1 if the sentiment is negated. For example, I am not happy.</li>\n",
    "    <li>-1 if one word separates the sentiment and a negation word. For example, I am not very happy.</li>\n",
    "    <li>+1 if two or more words separate the sentiment and a negation word. For example, I am not saying I am happy.</li></ul>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Let us use this function to see the overall sentiments of comments received in our sample dataset.</p>     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a7a4f72-7a87-4ca8-b304-c1c4fefcc01d",
   "metadata": {},
   "outputs": [],
   "source": [
    "sentimentextractor_out = SentimentExtractor(text_column=\"comment_text\",\n",
    "                                                data=tdf_sample,\n",
    "                                                accumulate=['comment_id', 'comment_text']\n",
    "                                                )\n",
    "\n",
    "senti = sentimentextractor_out.result\n",
    "senti"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be4b8fab-465b-4463-bc54-227803b52474",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>As we can see that the function outputs the polarity , sentiment score and the sentiment words on which it calculated the score. In the sentiment_words output column the function returns the overall positive and negative scores, the words which are used for soring the sentiment -1 for negative and +1 for positive sentiment and in the brackets it also displays how many times the word is repeated in the comment. e.g beautiful 1 (2) means beautiful is positive sentiment word and has occured twice in the comment. We also have an option of providing the custom dictonary to the function.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4ef16958-285e-4682-9b3b-058cc32bc7bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "d1=senti.select(['comment_id','polarity']).groupby('polarity').count()\n",
    "d1 = d1.assign(drop_columns=True,\n",
    "          Polarity=d1.polarity,\n",
    "          Count=d1.count_comment_id)\n",
    "d1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "043b7451-d83a-4ea0-be3e-9e41dc1afb94",
   "metadata": {},
   "outputs": [],
   "source": [
    "plot1 = d1.to_pandas()\n",
    "# Create a bar plot\n",
    "ax = plot1.plot(kind='bar', x='Polarity', y='Count', color='skyblue', edgecolor='black', figsize=(8, 6))\n",
    "\n",
    "# Add count labels on top of the bars\n",
    "for p in ax.patches:\n",
    "    ax.annotate(str(p.get_height()), (p.get_x() + p.get_width() / 2., p.get_height()),\n",
    "                ha='center', va='center', xytext=(0, 10), textcoords='offset points')\n",
    "\n",
    "# Rotate x-labels by 45 degrees\n",
    "ax.set_xticklabels(ax.get_xticklabels(), rotation=45, ha='right')\n",
    "\n",
    "# Add labels and title\n",
    "plt.title('Polarity of comments')\n",
    "plt.xlabel('Polarity')\n",
    "plt.ylabel('Counts')\n",
    "\n",
    "# Show the plot\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "503ab3bf-be96-463a-94cb-7bcd08882d58",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>From the above chart we can see that the comments are largely postive in sentiment.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40afb547-9610-4d9c-82d8-2e1ab1aa0e71",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>5. Integrating with OpenSource LLM and create Word Embeddings</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14f793dc-c5af-4fd2-9c96-856892567ca9",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "Word embedding is a technique used in natural language processing (NLP) to represent words as dense vectors. This allows words with similar meanings to have similar representations. Word embeddings capture semantic relationships between words, enabling NLP models to better understand and process human language.<br><br>Traditional methods of representing words, such as one-hot encoding or bag-of-words, represent each word as a sparse vector where most elements are zero and only one element is one (for one-hot encoding) or a count of occurrences (for bag-of-words). These representations do not capture semantic similarity between words and can result in high-dimensional and sparse feature spaces.<br> <br>Word embeddings represent words as dense vectors of fixed dimensionality (e.g., 100, 200, or 300 dimensions) where each dimension represents a different aspect of the word's meaning. These vectors are learned from large corpora of text using techniques like Word2Vec, GloVe, or FastText.The key idea behind word embeddings is that words that occur in similar contexts tend to have similar meanings. By training word embeddings on large text corpora, the model learns to map words with similar meanings to nearby points in the vector space. For example, in a well-trained word embedding model, the vectors for \"male\" and \"female\" are expected to be closer to each other than to the vector for \"apple\".\n",
    "</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93c32045-7331-4fa6-811a-9a2046c91a03",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'> Large Language Models are trained on huge amounts of data enabling them to learn patterns, grammar, and context from a wide range of topics. They can be fine-tuned for specific tasks, such as question-answering, natural language understanding, and text generation etc and have a wide range of uses across various domains due to their ability to understand and generate human-like text.<br>In this demo we will use opensource 'GIST-small-Embedding-v0' model for our embeddings generation. Clearscape analytics can integrate with any opensource, OpenAI or cloud provider specific LLM (AWS Sagemaker/AWS Bedrock). Please refer demo index for demos on other integrations with TDApiClient. </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aac9013d-b38a-4d64-a83e-1c9f5c76dbcb",
   "metadata": {},
   "outputs": [],
   "source": [
    "df1=tdf_sample.drop(['comment_summary'],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06dfad04-ff27-4631-a5b4-cbc3360bc9ca",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We will first create function which will create embeddings from the opensouce model we are using. We are using GIST-small-Embedding-v0 model you are free to change the model as per your needs.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45f4cfa3-5a95-4146-878b-b5c28fb4355b",
   "metadata": {},
   "outputs": [],
   "source": [
    "def recursive_emb_generator(table_name, file_name, chunksize=100):\n",
    "    wallclock_time_start = timeit.default_timer()\n",
    "    \n",
    "    # delete the records     \n",
    "    delete_emb_from_sql(table_name, eng)\n",
    "    \n",
    "    # Read the data in chunks of 1000 rows\n",
    "    temp_df = pd.read_csv(file_name, chunksize=chunksize)\n",
    "    \n",
    "    # Iterate over the chunks\n",
    "    for chunk in tqdm(temp_df, desc=\"Overall progress \",):\n",
    "        print(\"Data size in current chunk: \", chunk.shape)\n",
    "        df_chunk = get_embeddings_hf(chunk)\n",
    "        \n",
    "        copy_emb_to_sql(table_name=table_name, tdf=df_chunk)\n",
    "        copy_to_sql(df_chunk, table_name=table_name,primary_index='comment_id', if_exists='append')\n",
    "        print(f\"{df_chunk.shape[0]} reviews saved to sql \\n\")\n",
    "\n",
    "    wallclock_time_end = timeit.default_timer()\n",
    "    wallclock_time = wallclock_time_end - wallclock_time_start\n",
    "    print('wallclock time:\\t', wallclock_time)\n",
    "    print('-'*50,' complete ', '-'*50)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44335f15-6957-4abe-ad80-35d30e14cd14",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Below will generate embeddings on our data and store the result in database. The code runs for approx 5min for the sample data we are using. In the interest of time we have preloaded the embeddings in the table, you can run the code or use the preloaded table.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f0baed8-da7c-4ad3-8204-f9b6ec2a7bdc",
   "metadata": {},
   "source": [
    "<div style=\"background-color: #ffdddd; padding: 10px; border: 1px solid #f44336; border-radius: 5px; margin-bottom: 10px;\">\n",
    "    <p><strong>Warning:</strong> Running the code might take 5+ minutes.</p>\n",
    "    <p>Please confirm if you want to proceed.</p>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ef47cfd-54f2-476e-ab8b-fcc70f49caa9",
   "metadata": {},
   "outputs": [],
   "source": [
    "user_conformation = input(\"Do you want to run the code? (yes/no): \")\n",
    "\n",
    "\n",
    "if user_conformation.lower() == 'yes':\n",
    "    # export df to csv\n",
    "    df1.to_csv('./df_rev.csv')\n",
    "    recursive_emb_generator(table_name=\"comment_embeddings\", file_name='df_rev.csv', chunksize=1000)\n",
    "    comment_embeddings = DataFrame(\"comment_embeddings\")\n",
    "    print(\"Embeddings generated successfully.\")\n",
    "else:\n",
    "    comment_embeddings = DataFrame(in_schema('DEMO_Retail', 'Comment_Embeddings'))  \n",
    "    print(\"Existing embeddings generated loaded.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21336cea-cbd4-4702-9d1a-40ecb3d1835f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(\"Data information: \\n\",comment_embeddings.shape)\n",
    "comment_embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8d84c7c-6df4-4a61-8b66-b9af97d83e3b",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>6. Kmeans clustering using the embeddings</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7f094f7-2ec9-488d-bd17-b1ab11853e10",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In the previous step we have created embeddings from the text data we have. The generated embeddings are the features that can be used in machine learning algorithms. We will use Kmeans clustering to categorize the comments in different clusters.<br>First let us start by generating columnlist to be used in KMeans function.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28d79e8e-3e7f-4346-8931-73912c584b9e",
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_column_list = comment_embeddings.columns\n",
    "embedding_column_list.remove(\"comment_id\")\n",
    "embedding_column_list.remove(\"customer_id\")\n",
    "embedding_column_list.remove(\"comment_text\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc5ab58d-6dcc-47eb-a585-419300568c36",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run KMeans to find the clustering based on embeddings.\n",
    "kmeans_out = KMeans(\n",
    "    id_column=\"comment_id\",\n",
    "    data=comment_embeddings,\n",
    "    target_columns=embedding_column_list,\n",
    "    output_cluster_assignment=True,\n",
    "    num_clusters=7\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f875e0b2-c664-4c48-9330-4e4a44e65139",
   "metadata": {},
   "outputs": [],
   "source": [
    "kmeans_df=kmeans_out.result\n",
    "kmeans_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3b4254ba-bb23-43d6-8b83-b90fdb399cbf",
   "metadata": {},
   "outputs": [],
   "source": [
    "d2 = kmeans_df.groupby('td_clusterid_kmeans').count()\n",
    "d2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a262d769-7f7e-4619-b237-f566fe4df279",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert to pandas DataFrame \n",
    "plot2 = d2.to_pandas()\n",
    "\n",
    "# Plotting the bar chart\n",
    "ax = plot2.plot(kind='bar', x='td_clusterid_kmeans', y='count_comment_id', color='skyblue', edgecolor='black', figsize=(8, 6))\n",
    "\n",
    "# Add labels and title\n",
    "plt.title('Comments in each cluster')\n",
    "plt.xlabel('Cluster_Id')\n",
    "plt.ylabel('Counts')\n",
    "\n",
    "# Show count on top of bars\n",
    "for p in ax.patches:\n",
    "    ax.annotate(str(p.get_height()), (p.get_x() + p.get_width() / 2., p.get_height()), ha='center', va='center', xytext=(0, 10), textcoords='offset points')\n",
    "\n",
    "# Rotate x-labels by 45 degrees\n",
    "plt.xticks(rotation=360)\n",
    "\n",
    "# Show the plot\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a863299a-231b-429b-a7e3-7a4d49013682",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>7. PCA</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b5bc2e1-4654-4e66-b615-7c3a025a9604",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Principal Component Analysis (PCA) is a technique used for dimensionality reduction in data analysis and machine learning. It works by transforming the original high-dimensional data into a lower-dimensional space while retaining as much of the original variance as possible. PCA achieves this by identifying the principal components, which are the directions in feature space along which the data varies the most. These principal components are computed as the eigenvectors of the covariance matrix of the standardized data, and they represent the most significant sources of variation in the data. By selecting a subset of the principal components that capture the most variance, PCA allows for a more compact representation of the data while preserving its essential structure and relationships. The transformed data can be used for visualization, feature extraction, noise reduction, and other analysis tasks, making PCA a powerful tool for data exploration and dimensionality reduction.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a32c236-a419-43b1-8612-216b0fcd9153",
   "metadata": {},
   "outputs": [],
   "source": [
    "pca_df=comment_embeddings.join(other = kmeans_df, on = [\"comment_id\"], how = \"inner\",lprefix = \"emb\", rprefix = \"kmeans\")\n",
    "pca_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fd982e8a-6d46-4acf-950c-6e95b6669b24",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "pca_obj = valib.PCA(data=pca_df,\n",
    "                        columns=embedding_column_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b4a1754a-04d1-4be1-87f7-02b202464893",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get PCA scores using the model generated above\n",
    "obj = valib.PCAPredict(data=pca_df,\n",
    "                           model=pca_obj.result,\n",
    "                           index_columns=\"emb_comment_id\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "084b6803-ce54-4330-9faf-d5f01d0c2141",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print the results\n",
    "obj.result"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6139e1f9-e563-4344-8cf4-fb18b4eecb2a",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>As we see from above the PCA has reduced 300+ embeddings to 80+ embeddings. We will plot the first 2 factors to see how our clusters looks.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4e207d5e-17bd-4e5b-bbda-a06ced5a3214",
   "metadata": {},
   "outputs": [],
   "source": [
    "out_reduced_df = obj.result.select(['emb_comment_id','Factor 1','Factor 2'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bb1e213c-b534-4826-9bcc-0775e9dd4239",
   "metadata": {},
   "outputs": [],
   "source": [
    "out_reduced_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6e0003d-1f35-4f5d-9e07-b2b18c790ce2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Join the KMeans output with dataframe with reduced number of columns.\n",
    "final_df=kmeans_df.join(other = out_reduced_df, on = [\"comment_id = emb_comment_id\"], how = \"inner\",lprefix = \"l\", rprefix = \"r\")\n",
    "final_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd9d467e-fc89-4bb8-b39b-15614cdbe5c9",
   "metadata": {},
   "outputs": [],
   "source": [
    "plot3=final_df.to_pandas().reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c9c857c-f3da-440f-aace-a7294335dafd",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(10, 8))\n",
    "scatter = plt.scatter(plot3['Factor 1'], plot3['Factor 2'], c=plot3['td_clusterid_kmeans'], cmap='viridis')\n",
    "plt.title('PCA Visualization of Clusters')\n",
    "plt.legend(*scatter.legend_elements(), title='Clusters')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ee75544-f3cf-4a82-ae02-501d69f2031b",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>8. Term Frequency-Inverse Document Frequency</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39fece81-f82c-4e95-8efa-73fe39fc94e8",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "TF-IDF (Term Frequency-Inverse Document Frequency) is a numerical statistic used in NLP to evaluate the importance of a word in a document relative to a collection of documents. It is calculated by multiplying two factors: the term frequency (TF), which measures how frequently a word occurs in a document, and the inverse document frequency (IDF), which penalizes words that are common across multiple documents in the collection. TF-IDF assigns higher weights to words that are frequent in a document but rare in other documents, allowing it to capture the discriminative power of words in distinguishing documents. This technique is commonly used for text mining, document classification, search engine ranking, and other tasks where the relevance of words needs to be assessed within a corpus of text data.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c175d66c-b15e-4363-9a19-45bbb9920f3a",
   "metadata": {},
   "outputs": [],
   "source": [
    "title_input=tdf_sample.join(other = kmeans_df, on = [\"comment_id\"],how = \"inner\",lprefix = \"l\", rprefix = \"r\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b865ed50-8c1f-4f11-b4e1-2f39f09c584d",
   "metadata": {},
   "outputs": [],
   "source": [
    "title_df = title_input.assign(\n",
    "    drop_columns=True,\n",
    "    comment_id=title_input.l_comment_id,\n",
    "    customer_id=title_input.customer_id,\n",
    "    comment_text=title_input.comment_text,\n",
    "    comment_summary=title_input.comment_summary,\n",
    "    cluster_id=title_input.td_clusterid_kmeans\n",
    " )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55946d49-562b-4615-a9eb-7d45b4b716e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "title_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4636d93e-b095-4b50-858a-c4a09e0eb08e",
   "metadata": {},
   "outputs": [],
   "source": [
    "copy_to_sql(df = title_df, table_name = 'title_comments', if_exists = 'replace',primary_index = \"comment_id\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcc9c2c1-a810-4d51-827b-8463fddb917e",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "A text parser, also known as a text tokenizer, breaks a text into its constituent parts, such as words, phrases, sentences, or other meaningful units. The <b>TD_TextParser</b> function performs the following operations:\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'><li>Tokenizes the text in the specified column</li>\n",
    "    <li>Removes the punctuations from the text and converts the text to lowercase</li>\n",
    "    <li>Removes stop words from the text and converts the text to their root forms</li>\n",
    "    <li>Creates a row for each word in the output table</li>\n",
    "    <li>Performs stemming; that is, the function identifies the common root form of a word by removing or replacing word suffixes</li>\n",
    "    </ul>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'> The output table generated from the TD_TextParser is fed to the <b>TD_TFIDF</b> function. TD_TFIDF function represents each document as an N-dimensional vector, where N is the number of terms in the document set (therefore, the document vector is sparse). Each entry in the document vector is the TF-IDF score of a term.</p>    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3208bc3d-1967-4e47-8c67-0b99ffcbd647",
   "metadata": {},
   "outputs": [],
   "source": [
    "qry1='''\n",
    "CREATE MULTISET TABLE tfidf_input_tokenized AS (\n",
    "SELECT comment_id, cast(token as varchar(15)) as token, cluster_id FROM TD_TextParser (\n",
    "ON title_comments AS InputTable\n",
    "USING\n",
    "TextColumn ('comment_text')\n",
    "ConvertToLowerCase ('true')\n",
    "OutputByWord ('true')\n",
    "Punctuation ('\\[.,-?\\!\\]')\n",
    "RemoveStopWords ('true')\n",
    "StemTokens ('true')\n",
    "Accumulate ('comment_id','cluster_id')\n",
    ") AS dt ) WITH DATA;\n",
    "'''\n",
    "\n",
    "qry2='''CREATE MULTISET TABLE tfidf_comments AS (\n",
    "SELECT * FROM TD_TFIDF (\n",
    "   ON tfidf_input_tokenized  AS InputTable\n",
    "   USING\n",
    "   DocIdColumn ('cluster_id')\n",
    "   TokenColumn ('token')\n",
    "   TFNormalization ('LOG')\n",
    "   IDFNormalization ('SMOOTH')\n",
    "   Regularization ('L2')\n",
    "   --Accumulate ('cluster_id')\n",
    ") AS dt ) WITH DATA;\n",
    "'''\n",
    "\n",
    "# Execute the query\n",
    "execute_sql(qry1)\n",
    "execute_sql(qry2)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5cd1fa0-7194-4f95-b977-888116b9029a",
   "metadata": {},
   "outputs": [],
   "source": [
    "tfidf_comments = DataFrame(\"tfidf_comments\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ce69281-a675-46ce-8090-836a66e70cd9",
   "metadata": {},
   "outputs": [],
   "source": [
    "tfidf_comments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d89f4ed2-cb87-4a47-8606-e2ac7ecb7811",
   "metadata": {},
   "outputs": [],
   "source": [
    "window = tfidf_comments.window(partition_columns=\"cluster_id\",\n",
    "                               order_columns=\"TD_TF_IDF\"\n",
    "                              )\n",
    "\n",
    "# Execute rank() on a window.\n",
    "df = window.rank()\n",
    "df.sort('col_rank')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40c3b77a-de5b-4a39-ab5b-707519b6bf72",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>From above we can see the frequency and importance of each word in the cluster.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cc47e14-a7e5-41c9-a637-e0fbc240250f",
   "metadata": {},
   "source": [
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>Conclusion</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ecd35fef-d012-418f-bb61-9e9619ca50ee",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In this demo we have seen how we can do analysis and pre-processing of the text data in Vantage using InDb functions and integrating with 3rd party LLM models. </p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f78894af-5444-4d7b-afc9-93b0c139fd20",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>9. Cleanup</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b9fda39-aa2d-4b70-8fa6-027c7aac735d",
   "metadata": {},
   "source": [
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C;color:#00233C'><b>Work Tables</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C;'>\n",
    "We need to clean up our work tables to prevent errors next time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "825f607d-db0a-46c4-b463-31f82fbec57f",
   "metadata": {},
   "outputs": [],
   "source": [
    "tables = ['tfidf_comments','tfidf_input_tokenized','title_comments','comment_embeddings']\n",
    "\n",
    "# Loop through the list of tables and execute the drop table command for each table\n",
    "for table in tables:\n",
    "    try:\n",
    "        db_drop_table(table_name = table)\n",
    "    except:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6cd94a98-d3e7-40d3-84a2-57d3931c1e8d",
   "metadata": {},
   "source": [
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C;color:#00233C'><b>Databases and Tables</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We will use the following code to clean up tables and databases created for this demonstration.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d1284f1-8b92-4e65-a681-04ee478247ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "%run -i ../run_procedure.py \"call remove_data('DEMO_Retail');\" \n",
    "#Takes 20 seconds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4c470f7-5747-4777-b1c3-8ae4a2433218",
   "metadata": {},
   "outputs": [],
   "source": [
    "remove_context()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "767393be-e89c-4d6d-9e8f-fc440fc28901",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0788928e-d332-4e24-984b-e88bb05fb8c1",
   "metadata": {},
   "source": [
    "<b style = 'font-size:20px;font-family:Arial;color:#00233C'>Required Materials</b>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Let’s look at the elements we have available for reference for this notebook:</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60a109c3-cf1d-4419-ba7e-34e989a5c7ac",
   "metadata": {},
   "source": [
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Filters:</b></p>\n",
    "    <ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li><b>Industry:</b> Retail</li>\n",
    "    <li><b>Functionality:</b> Text Analysis</li>\n",
    "    <li><b>Use Case:</b> Natural Language Processing</li>\n",
    "    </ul>\n",
    "    <p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Related Resources:</b></p>\n",
    "    <ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li><a href = 'https://www.teradata.com/Blogs/NPS-is-a-metric-not-the-goal'>·In the fight to improve customer experience, NPS is a metric, not the goal</a></li>\n",
    "    <li><a href = 'https://www.teradata.com/insights/ai-and-machine-learning/using-natural-language-to-query-teradata-vantagecloud-with-llms'>·Using Natural Language to query Teradata Vantage Cloud with LLMs</a></li>\n",
    "    </ul>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fa577ba-8825-428c-8e90-686e900e9574",
   "metadata": {},
   "source": [
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Reference Links:</b></p>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'> \n",
    "       <li>Teradata Vantage™ - Analytics Database Analytic Functions - 17.20: <a href = 'https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/Teradata-VantageTM-Analytics-Database-Analytic-Functions-17.20/Introduction-to-Analytics-Database-Analytic-Functions '>https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/Teradata-VantageTM-Analytics-Database-Analytic-Functions-17.20/Introduction-to-Analytics-Database-Analytic-Functions </a></li>    \n",
    "  <li>Teradata® Package for Python User Guide - 17.20: <a href = 'https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/Teradata-Package-for-Python-User-Guide-17.20/Introduction-to-Teradata-Package-for-Python'>https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/Teradata-Package-for-Python-User-Guide-17.20/Introduction-to-Teradata-Package-for-Python</a></li>\n",
    "  <li>Teradata® Package for Python Function Reference - 17.20: <a href = 'https://docs.teradata.com/r/Enterprise/Teradata-Package-for-Python-Function-Reference-17.20/Teradata-Package-for-Python-Function-Reference'>https://docs.teradata.com/r/Enterprise/Teradata-Package-for-Python-Function-Reference-17.20/Teradata-Package-for-Python-Function-Reference</a></li>      \n",
    "  <li>Teradata® API Integration Guide for Cloud Machine Learning: <a href = 'https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/Teradata-VantageTM-API-Integration-Guide-for-Cloud-Machine-Learning/Teradata-Partner-API'>https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/Teradata-VantageTM-API-Integration-Guide-for-Cloud-Machine-Learning/Teradata-Partner-API</a></li>    \n",
    "</ul>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91320552-fa4b-4378-8d87-8b8477800766",
   "metadata": {},
   "source": [
    "<footer style=\"padding-bottom:35px; background:#f9f9f9; border-bottom:3px solid #00233C\">\n",
    "    <div style=\"float:left;margin-top:14px\">ClearScape Analytics™</div>\n",
    "    <div style=\"float:right;\">\n",
    "        <div style=\"float:left; margin-top:14px\">\n",
    "            Copyright © Teradata Corporation - 2024. All Rights Reserved\n",
    "        </div>\n",
    "    </div>\n",
    "</footer>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}