Getting_Started/Intro_to_Pandas_For_Python/Intro_to_Pandas_For_Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c5b28e52-43c8-4ead-846f-52f48a437959",
   "metadata": {},
   "source": [
    "<header>\n",
    "   <p  style='font-size:36px;font-family:Arial; color:#F0F0F0; background-color: #00233c; padding-left: 20pt; padding-top: 20pt;padding-bottom: 10pt; padding-right: 20pt;'>\n",
    "       Getting Started with Jupyter Notebook, Python, and Pandas\n",
    "  <br>\n",
    "       <img id=\"teradata-logo\" src=\"https://storage.googleapis.com/clearscape_analytics_demo_data/DEMO_Logo/teradata.svg\" alt=\"Teradata\" style=\"width: 125px; height: auto; margin-top: 20pt;\">\n",
    "    </p>\n",
    "</header>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97604e7e-984e-4415-8d83-49f939525bc5",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><i>This is a basic instructional notebook on how to use Jupyter and Python Pandas. It will show the basics of data frames, perform some simple queries against system tables, and then show how to load and save data from existing files in this notebook's directories.</i></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b099a0a",
   "metadata": {},
   "source": [
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Links:</b></p>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>Jupyter Notebook Reference: <a href = 'https://jupyter.org/documentation'>https://jupyter.org/documentation</a></li>\n",
    "    <li>Python Pandas Reference: <a href = 'https://pandas.pydata.org/docs/user_guide/index.html'>https://pandas.pydata.org/docs/user_guide/index.html</a></li>\n",
    "</ul>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Contents</b></p>\n",
    "<ol style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>Overview of Jupyter Notebook and Python Pandas</li>\n",
    "<li>Overview of Python Pandas\n",
    "    <ul><li>Series</li>\n",
    "    <li>DataFrames</li>\n",
    "    <li>Accessing Data</li>\n",
    "    <li>Modifying DataFrames</li></ul></li>\n",
    "<li>Loading Data From Files\n",
    "    <ul><li>DataFrame metadata</li>\n",
    "    <li>Grouping, Aggregation, and Pivot Tables</li>\n",
    "        <li>Data Cleansing and Transformation</li></ul></li>\n",
    "<li>Loading Data From databases and remote systems\n",
    "    <ul><li>Combining DataFrames - joins and concatenation</li></ul></li>\n",
    "    <li>Persisting Data - saving to files</li>\n",
    "    <li>Cleanup</li>\n",
    "</ol>\n",
    "\n",
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d52c666",
   "metadata": {},
   "source": [
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>1. Jupyter Notebook</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Jupyter Notebook is an open-source browser-based application that allows users to create and share interactive documents that contain live code, equations, visualizations, and/or narrative text.  Common uses include:</p>\n",
    "\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>Data Cleansing and Transformation</li>\n",
    "    <li>Numerical Simulation</li>\n",
    "    <li>Statistical Modeling</li>\n",
    "    <li>Data Visualization</li>\n",
    "    <li>Machine Learning</li>\n",
    "    <li>And much more</li>\n",
    "</ul>\n",
    "    \n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Anatomy of a Notebook</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Notebooks consist of individual cells that can contain text, images, or executable code</p>\n",
    "    <br>\n",
    "\n",
    "**Text Cells** use a formatting syntax called markdown that uses special character sequences to format output.  For example, to make text **bold** one would wrap the text in two asterisks, like this: ```**bold**```\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Note that the above text doesn't look quite like the rest of the text.  Markdown has very simple styling; for more advanced control over text presentation, one can use html</p>\n",
    "\n",
    "```html\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Note that the above text doesn't look quite like the rest of the text.  Markdown has very simple styling; for more advanced control over text presentation, one can use html</p>\n",
    "```\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Code Cells</b> contain executable code.  The code can be in different languages; it is most common to see Python, although R and Julia is common, and Teradata provides a dedicated SQL plugin for Vantage</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Magic Statements</b> are special directives that aren't in the underlying language of the notebook.  These statements direct the underlying notebook application to perform some function.  For Python notebooks, the IPython engine supports magic statements documented <a href = 'https://ipython.readthedocs.io/en/stable/interactive/magics.html'>here</a>.  Usually these statements are preceeded by a special character such as '%' or '!' and will instruct the environment to do something such as execute commandline statements, display images inline, save output to a file, etc.</p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6d5fdd21",
   "metadata": {},
   "outputs": [],
   "source": [
    "# An Example of a Code Cell.\n",
    "# To Run this Python code, execute the cell by clicking the \"play\" button\n",
    "# Or pressing SHIFT-ENTER\n",
    "\n",
    "print('hello world')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7121d371",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Some python code preceeded by a magic statement\n",
    "# that times the execution of the cell\n",
    "\n",
    "%time\n",
    "print('How long did it take?')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23703f8a",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><i><b>Note</b>: You can find a helpful resource on keyboard shortcuts in the JupyterLab documentation. Simply follow the link provided: <a href = 'https://jupyterlab.readthedocs.io/en/stable/user/interface.html#keyboard-shortcuts'>Keyboard Shortcuts</a>. If you're using the Classic Jupyter Notebook interface, you can access the keyboard shortcuts by selecting Help->Keyboard Shortcuts from the menu. These shortcuts can greatly assist you in working with code, executing code blocks, and more. It's worth exploring and familiarizing yourself with these shortcuts to enhance your productivity.</i></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11b3dab0",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>2. Python Libraries/Packages</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Python is a programming language that allows developers to add new functionalities by using libraries. These libraries contain pre-written code contributed by various individuals and companies. To use these libraries, we use the \"pip\" command to install them. When we install a library, it may also require other libraries to be installed along with it.\n",
    "<br>\n",
    "<br>\n",
    "One such library is <a href = 'https://pandas.pydata.org'>Python Pandas</a>. It is the most popular and powerful Python packaeges for working with data. Pandas also provides the functional and linguistic basis for the Teradata \"teradataml\" package. This notebook barely scratches the surface of what's available in Pandas and the documentation and community resources are a great place to continue learning.\n",
    "<br>\n",
    "<br>\n",
    "After installing the libraries, it is important to restart Python Kernel so that it recognizes the newly installed libraries and their functions. Restarting Python Kernel means closing and reopening it. There are a couple of ways to restart Python Kernel: you can simply type \"0 0\" (zero space zero) as a command, or you can use the command line menus and choose \"Kernel/Restart Kernel...\" or \"Kernel/Restart Kernel and Clear All Outputs\".\n",
    "<br>\n",
    "<br>\n",
    "To make things easier for users and to save time during startup, we have already pre-installed and tested the libraries used in the ClearScape Analytics Experience Jupyter notebooks with Python. The pip commands for installing these libraries are commented out, which means they are not executed by default. You can uncomment these commands if you want to install newer versions of the libraries or if you encounter any issues and need to reinstall them.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e5bd620",
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install --upgrade pandas --user\n",
    "# !pip install --upgrade numpy --user"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a14b9891",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\"> \n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>2.1 Import Statements</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Common convention is to load the entire pandas\\ library with the alias of 'pd'.  Most online resources will use this convention.  Run the following cell:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c4b81a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# commonly, all general import statements will be\n",
    "# written near the top of a notebook or file\n",
    "\n",
    "import getpass as gp\n",
    "import io\n",
    "import getpass\n",
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from teradataml import *"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "630f5463",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>2.2 Pandas Series</b></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>A Pandas Series is an object that contains list of objects and a labeled index.  Think about it as a single column in an Excel worksheet, but with the added flexibility of the row number alternatively having a label.  This is very powerful, since items can be referenced using this label value instead of just position</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Items in a Series can consist of any Python object; strings, numbers, or even functions or programs.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "abc22d03",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a simple series from a list of values\n",
    "\n",
    "# This is a simple Python list object with some numbers and strings as items\n",
    "d = [1, 2, 'foo', 'bar']\n",
    "\n",
    "# Create a Pandas Series by passing the list above as the data parameter to the Series() Module Function.\n",
    "# Note the convention 'pd.function_name()':\n",
    "my_series = pd.Series(data = d)\n",
    "\n",
    "#Show the Series - the index has been auto-assigned as a positional number\n",
    "my_series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6f6aba59",
   "metadata": {},
   "outputs": [],
   "source": [
    "# This Series can be enhanced by adding a list of labels for the index\n",
    "labels = ['row1', 'row2', 'row3', 'row4']\n",
    "\n",
    "# Re-create the Series with the labels as an index by adding a new parameter:\n",
    "my_series = pd.Series(data = d, index = labels)\n",
    "\n",
    "my_series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "797e583e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Index value can be used to reference the item\n",
    "my_series['row2']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ba71f1b",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>2.3 Pandas DataFrame</b></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Pandas DataFrames form the core of Pandas Data Management capabilities.  Most analytical and other operations are performed on DataFrames using many different method calls.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>A Pandas DataFrame closely resembles a table; with rows and columns.  A more Pandas-native way to understand a DataFrame is that it consists of multiple labeled Series that share an index; and each Series is a column.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "40c326df",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create lists of values and an index of labels as source data\n",
    "Age = [23, 33, 55, 78]\n",
    "FName = ['Bob', 'Sue', 'Jen', 'Jack']\n",
    "ID = ['p1', 'p2', 'p3', 'p4']\n",
    "\n",
    "# Create two series using the same index values:\n",
    "series_1 = pd.Series(data = Age, index = ID)\n",
    "series_2 = pd.Series(data = FName, index = ID)\n",
    "\n",
    "# Create a DataFrame by concatenating the two Series\n",
    "# concat() is a Pandas Module Function (note pd.\"\") that will\n",
    "# combine multiple Series or DataFrames together long either axis\n",
    "# These series are passed as a List object:\n",
    "df_1 = pd.concat([series_1, series_2], axis = 1) # axis = 1 means combine columns\n",
    "\n",
    "df_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ebd7683c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Modifications can be performed on DataFrames\n",
    "# This is an example of renaming the column labels using a dictionary object as a parameter\n",
    "# the dictionary contains keys based on the current column label, and a value of the new one.\n",
    "# This method call RETURNS a new DataFrame with the columns renamed - more on this below\n",
    "df_1.rename(columns = {0:'Age', 1:'First Name'})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8de0c5cb",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>2.4 Methods that return an object vs. modify the object</b></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Most Pandas Methods will <b>Return</b> an object (DataFrame or Series) and <b>not</b> modify the underlying object.  In the cell above, the 'rename()' method returns a DataFrame as a response to the method call - that's why the output is displayed the way we expect to see it.  However, the original DataFrame has not been modified:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f94e14dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this to see that df_1 hasn't been modified\n",
    "df_1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51e669c3",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>If the user wishes to modify the original DataFrame or Series, one can assign the object equivalent to the method call.  Many methods also have a Boolean parameter 'inplace' which controls whether the original object is modified \"in place\".  However, it is considered not best practice to use inplace - it is more explicit to use equivalence.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7957673e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Assign equivalence:\n",
    "df_1 = df_1.rename(columns = {0:'Age', 1:'First Name'})\n",
    "df_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d7fd5e33",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Inplace:\n",
    "df_1.rename(columns = {'Age':'age', 'First Name':'FName'}, inplace = True)\n",
    "df_1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc3d4afc",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>2.5 Accessing DataFrame Contents</b></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>DataFrame contents can be accessed by row, column, or by using Boolean expressions.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "790182b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Return the Series represented by the column label 'age'\n",
    "df_1['age']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6e266031",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Or a row:\n",
    "df_1.loc['p1']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0e659b21",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Or a single position (row/column):\n",
    "df_1.loc['p3', 'age']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb083a16",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Or an expression.\n",
    "df_1[df_1['age'] > 50]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae3d1d35",
   "metadata": {},
   "outputs": [],
   "source": [
    "# the expression above actually creates a Series\n",
    "# that consists of Boolean values that acts as a \"mask\"\n",
    "# applied to the DataFrame:\n",
    "df_1['age'] > 50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96c806ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "# so when multiple expressions are combined:\n",
    "df_1[(df_1['age'] > 50) & (df_1['FName'] == 'Jen')]\n",
    "# We're actually creating a mask using Boolean operators."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1f044439",
   "metadata": {},
   "outputs": [],
   "source": [
    "# in this case OR\n",
    "# Is applied when combining two Series\n",
    "(df_1['age'] > 50) | (df_1['FName'] == 'Bob')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a2b615a",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_1[(df_1['age'] > 50) | (df_1['FName'] == 'Bob')]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "495f4d35",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>2.6 DataFrame from a Dictionay Object</b></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Python Dictionaries are very powerful objects, and contain most of the information needed to construct a Pandas DataFrame.  DataFrames can be constructed directly from dictionary objects:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34131d3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a dictionary consisting of two keys (A and B)\n",
    "# each with a list of values\n",
    "my_dict = {'A':[1,2,3,4], 'B':[5,6,7,8]}\n",
    "print('Original Dictionary:')\n",
    "print(my_dict)\n",
    "\n",
    "df_from_dict = pd.DataFrame(my_dict)\n",
    "print('New DataFrame:')\n",
    "df_from_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d92f409",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>2.7 Adding and Removing Columns</b></p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "431440e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# New columns can be created using simple declaration:\n",
    "df_from_dict['C'] = df_from_dict['A'] + df_from_dict['B']\n",
    "df_from_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "59fe87bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Or using the assign() method\n",
    "# note this method returns the object vs. modifying it, so to modify it, assign equivalence\n",
    "\n",
    "df_from_dict.assign(D = df_from_dict['A'] * df_from_dict['B'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73e0d3dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Drop method removes rows or columns depending on axis type\n",
    "# again, it returns an object, so assign equivalence or inplace = True:\n",
    "\n",
    "df_from_dict.drop('C', axis = 1, inplace = True)\n",
    "df_from_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64340cc4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Drop a row using the index:\n",
    "df_from_dict.drop(3, axis = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c1a1a8d",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>3. Populating DataFrames using Files</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The section above provided examples of constructing Series and DataFrames from other Python Object that we created manually in code.  More commonly, users will construct DataFrames from objects created from reading files or source data vs. manual entry.  Structured data such as excel files, csv, JSON, etc. as well as unstructured text and images can all be used to construct DataFrames directly.</p>\n",
    "\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>A sampling of the available methods are illustrated below.  The documentation will provide invaluable for understanding the variety and flexibility available in Pandas.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bcb4bb05",
   "metadata": {},
   "outputs": [],
   "source": [
    "# CSV is a common structured data format\n",
    "# Reading in this data can be trivially simple by using\n",
    "# one of pandas' module functions - read_csv()\n",
    "\n",
    "df_1 = pd.read_csv('data/example.csv')\n",
    "df_1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ef3be73",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>openpyxl is a separate library that must be installed and loaded into your environment prior to use. openpyxl is a Python library to read/write Excel 2010 xlsx/xlsm/xltx/xltm files.<br>\n",
    "openpyxl can be installed via a magic command that will execute the command-line tool 'pip' - the python package management utlity. Run the following \"!pip install\" command if necessary.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64f70716",
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install openpyxl --user"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6d2cb897",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Excel is another common format\n",
    "# Provide a file path and a sheet number\n",
    "# Instruct which column to use as an index - in this case the first column\n",
    "# finally - declare an \"engine\" - basically the underlying interpreter that reads the data\n",
    "\n",
    "df_calories = pd.read_excel('data/Calorie_Log.xlsx', sheet_name = 'Sheet1', index_col = 0, engine = 'openpyxl')\n",
    "df_calories"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5af08d1a",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>3.1 Investigating DataFrames</b></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Pandas provides many methods and attributes that are used to understand the content and structure of DataFrames</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Common attributes include:</p>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>shape. Returns a Tuple consisting of the length of rows, columns</li>\n",
    "    <li>dtypes. Returns a Series containing the Pandas datatype of the columns</li>\n",
    "    <li>index.  Returns a Series of index values.</li>\n",
    "    <li>columns.  Returns a Series of column labels</li>\n",
    "</ul>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Common methods include:</p>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>head().  Returns a Dataframe consisting of the first five (or pass an integer for more or less) rows of the DataFrame</li>\n",
    "    <li>info(). Returns an object displaying summary information about the DataFrame.</li>\n",
    "    <li>describe().  Returns a DataFrame which contains statistical information of the numeric columns</li>\n",
    "    <li>count()/min()/max()/mean()/std().  Returns the respective aggregation of numeric columns.</li>\n",
    "</ul>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9989b1ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_calories.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ddb8faf2",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_calories.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd540b54",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>3.2 GroupBy</b></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>groupby is a powerful method that will group rows of data together by one or more columns.  Grouped objects can have aggregations and other calculations performed on them.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b9f63ebc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Group the DataFrame by the column Exercise, and calculate the mean\n",
    "# of the numeric columns\n",
    "df_calories.groupby('Exercise').mean(numeric_only = True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af7625a1",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>3.3 Pivot Tables</b></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The pivot_table() method allows for very flexible control over the creation of pivot table DataFrames; including multiple groupings, aggregations, and formatting.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>For a comprehensive tutorial, see the article <a href = 'https://pbpython.com/pandas-pivot-table-explained.html'>\"Practical Business Python\"</a>.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>For the example below, this method call will use the same groupings as above, but add multiple aggregations and a totals row:</p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ae3cc35",
   "metadata": {},
   "outputs": [],
   "source": [
    "# numpy is a powerful scientific/mathematic library that Pandas leverages.\n",
    "# in this case, numpy provides us the mean and sum functions that are passed to the\n",
    "# pivot_table method.\n",
    "\n",
    "my_pivot = df_calories.drop('Dinner', axis = 1).pivot_table(index = 'Exercise', aggfunc = ['mean', 'sum'], margins = True)\n",
    "\n",
    "my_pivot"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76909a3a",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>3.4 Data Transformation/Manipulation</b></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Note the column \"Dinner\" is missing in the above aggregation examples.  This is because the column \"Dinner\" is a string of format '### calories', and therefore isn't able to be numerically formulated.  Further above, we reviewed how to create a new column using simple assignment (df['new_column'] = EXPRESSION).  Another option is self-assignment; we can assign an existing column a new value based on an expression.  Here's the process of converting a string column to numeric:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dad89928",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's do a quick manual example with a piece of text first:\n",
    "\n",
    "test_string = '234 calories'\n",
    "\n",
    "# Python string manipulation makes this conversion/extraction easy:\n",
    "\n",
    "new_values = test_string.split(' ') #split the string into a tuple using the space as the delimiter\n",
    "\n",
    "print(new_values[0]) # The first item in the tuple is the string '234'\n",
    "\n",
    "# However, it is still a string, so we need to cast it to a number:\n",
    "print(type(new_values[0]))\n",
    "\n",
    "print(type(int(new_values[0])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "01d097d7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# using the process above, we can convert the column\n",
    "# There are some slight modifications since we're dealing with a DataFrame\n",
    "# and not just a single string\n",
    "#  expand = True will return the split string as a DataFrame instead of a tuple\n",
    "#  this allows us to take the first item [0]\n",
    "#  we use astype(int) instead of simple cast int() to create a numeric column type:\n",
    "\n",
    "df_calories['Dinner'] = df_calories['Dinner'].str.split(' ', expand = True)[0].astype(int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba634cd4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# check the work:\n",
    "df_calories.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a39aa89c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Rerun the pivot_table method call, note Dinner appears:\n",
    "\n",
    "my_pivot = df_calories.pivot_table(index='Exercise', aggfunc=['mean', 'sum'], margins=True)\n",
    "\n",
    "my_pivot"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe9eed71",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>3.5 Other File Formats and file-like objects</b></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Pandas has read_ methods that support an extensive list of file formats:</p>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>read_json</li>\n",
    "    <li>read_html</li>\n",
    "    <li>read_parquet</li>\n",
    "    <li>etc.</li>\n",
    "    </ul>\n",
    "\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Most of these follow the same convention of passing a source file as a \"path\" or a file-like object.  Paths can be local filesystem paths, or they can be URIs (s3://bucket/path/to/object, ftp://user:pass@host/path, etc.).  Files can also be string or binary-serializable objects that can be read into a memory buffer using io.StringIO or io.BytesIO:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1f892b25",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Construct a CSV formatted string\n",
    "my_string = 'col1,col2\\n1,2'\n",
    "\n",
    "# read it into a memory buffer\n",
    "my_buffer = io.StringIO(my_string)\n",
    "\n",
    "# pass it to read_csv\n",
    "df = pd.read_csv(my_buffer)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31c95091",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>4. Populating DataFrames using Database connections</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Pandas and databases are a natural fit.  By combining these two technologies, users can leverage the power of large, server-based data management systems with the flexibility and innovation available in the Python ecosystem.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Common relational database management systems use table structures as storage, and SQL (Structured Query Language) as the syntax used to interact in reading, operating on, and writing data.</p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In order to access this data, users need to take some additional steps to connect to the database, and then execute SQL to operate on, or retrieve data to the client.</p>\n",
    "\n",
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>4.1 Establishing a Database connection</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The python ecosystem supports a common database interface called <a href = 'https://www.python.org/dev/peps/pep-0249/'>DBAPI2</a>.  One of the objects defined in this standard is the \"connection\" object type.  Pandas can use this connection object to perform the underlying communication with the target system.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Connection Libraries</b> Are ususally required and installed separately using pip or other means.  Some of these include:</p>\n",
    "\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>pyodbc for ODBC connections (requires underlying ODBC drivers)</li>\n",
    "    <li>jaydebeapi for JDBC connections (requires underlying java and JDBC drivers)</li>\n",
    "    <li>vendor-specific native drivers (that usually don't require underlying configuration):\n",
    "        <ul><li>teradatasql for Teradata Vantage</li>\n",
    "            <li>pymssql for MS SQL Server</li>\n",
    "            <li>cx_Oracle for Oracle</li>\n",
    "            <li>others</li></ul></li>\n",
    "    </ul>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>These libraries all have their system-specific requirements and syntax for connecting tot he target system. See the vendor/provider documentation.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cda040cb",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>4.2 Make changes for your execution</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The Jupyter Module for Teradata provides a helper library called tdconnect - this can use the underlying client configs and pass a JWT token for SSO. Establish connection to Teradata Vantage server (uses the Teradata SQL Driver for Python). Before you execute the following statement, replace the variables &ltHOSTNAME&gt, &ltUID&gt and &ltPWD&gt with your target Vantage system hostname (or IP address), and your database user ID(QLID) and password, respectively.</p>\n",
    "    \n",
    "```python\n",
    "td_context = create_context(host=\"tdprdX.td.teradata.com\", username=\"xy123456\", password=gp.getpass(prompt='Password:'), logmech=\"LDAP\")\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8242733d",
   "metadata": {},
   "outputs": [],
   "source": [
    "eng = create_context(host = 'host.docker.internal', username = 'demo_user', password = getpass.getpass(prompt = 'Password:'))\n",
    "print(eng)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6917425",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>4.3 Retrieving data using DBAPI connections</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Pandas provides a read_sql() module function for returning the results of a SQL query as a DataFrame.  Construct the query as a string, and pass it and the connection to the function:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b389ce9c",
   "metadata": {},
   "outputs": [],
   "source": [
    "qry = 'SELECT TOP 5 * FROM DBC.TablesV;'\n",
    "\n",
    "df_sql = pd.read_sql(qry, eng)\n",
    "\n",
    "df_sql"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f44b1356",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>4.4 SQLAlchemy Connections</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The Python ecosystem has continued to evolve database integration capabilities, and is now supporting a high-level architectural pattern based on a project called <a href = 'https://www.sqlalchemy.org/'>SQLALchemy</a>.  SQLAlchemy provides a more object-oriented approach to interacting with RDBMS, essentially providing a more \"pythonic\" pattern for interacting with databases instead of relying wholly on SQL.</p>\n",
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>4.5 Teradataml Package</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The teradataml package relies heavily on SQLAlchemy architecture, and provides significant capabilities to the user and the user's interaction with Vantage.  Please see the relevant notebooks in this directory and in the <a href = 'https://docs.teradata.com/search/all?query=Teradata%25C2%25AE+Python+Package+User+Guide&content-lang=en-US'>Getting Started Guide</a> for more information.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf38b9e7",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>4.6 Combining DataFrames</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Pandas provides multiple techniques for combining data contained in DataFrames and Series.  Recall the beginning of this document, an example was shown using the concat() function to comine Series into a DataFrame.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Objects can be combined using <b>Methods</b> and <b>Module Functions</b>.  Methods are functions applied to an object, and Module functions are called under the Pandas module (usually pd)</p>\n",
    "\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>The <b>Method call</b> will combine itself with another object provided as a parameter.  These include:\n",
    "        <ul><li>DataFrame.join() will combine the calling DataFrame to another DataFrame using the calling DataFrame's index.</li>\n",
    "            <li>DataFrame.merge() will combine the calling DataFrame to another using arbitrary columns.</li>\n",
    "            <li>DataFrame.append() will add rows to the DataFrame using another Series or DataFrame.</li></ul></li>\n",
    "    <li>The <b>Module Function</b> will take multiple standalone objects, typically as a List, and return a new DataFrame or Series.  These include:\n",
    "        <ul><li>Pandas merge() will take two DataFrames, Series, or other objects and combine them using indices or column values</li>\n",
    "            <li>Pandas concat() will take a List of DataFrames, Series, or other objects and combine them across an axis, and not attempt to match up column values.  In the case of mismatched indices, rows will be appended.</li></ul></li>\n",
    "          </ul>  \n",
    "          \n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>join() and merge() Methods, and the merge() Module Function all behave in similar ways, but syntax and parameters differ for each.  Other important concepts include:</p>\n",
    "\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>Declaring how to join objects.  Usually this parameter is one of 'left', 'right', 'outer', 'inner', 'cross'.  These are roughly equivalent to SQL-style JOIN directives.</li>\n",
    "    <li>Which column or columns to use as keys to join on.  This parameter usually defaults to index, but can also be any column.</li>\n",
    "    <li>Suffix.  Under conditions where the resulting object has duplicate column names, the functions will append a suffix to enforce uniqueness.</li>\n",
    "    </ul>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "83990425-b05d-4952-b400-ca29f5082f1b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# An example of merge.\n",
    "# take the cleaned up DataFrame from above, and combine it to the original one:\n",
    "\n",
    "df_calories.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14f41257-e708-4d09-9322-454fff2d68aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# re-read the original data with the old string column:\n",
    "df_orig = pd.read_excel('data/Calorie_Log.xlsx', sheet_name = 'Sheet1', index_col = 0, engine = 'openpyxl')\n",
    "\n",
    "df_orig.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac19ee28-a53c-4210-b799-1565dbee893d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Call the Merge function:\n",
    "df_merge = pd.merge(left = df_calories,\n",
    "                    right = df_orig, \n",
    "                    left_index = True,\n",
    "                    right_index = True,\n",
    "                    how = 'inner')\n",
    "\n",
    "df_merge.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad52b3b0",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>5. Persisting DataFrames</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The end result of most workflows is the output; the resulting analytic, visualization, data representation, etc.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Pandas provides native capabilities for persisting data into human-readable, compressed, and/or SQL database formats. Note that many third-party libraries including <b>teradatasql/teradataml</b> provide methods for writing data to the target database system.</p>\n",
    "\n",
    "<b style = 'font-size:18px;font-family:Arial;color:#00233C'>File Persistence</b>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Some common human-readable formats include:</p>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>Excel</li>\n",
    "    <li>CSV</li>\n",
    "    <li>JSON (Debatable)</li>\n",
    "    <li>HTML</li>\n",
    "    </ul>\n",
    "    \n",
    "<b style = 'font-size:18px;font-family:Arial;color:#00233C'>Optimized Formats</b>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Pandas can also write binary formats that are optimized for storage space, read/scan performance, etc.  Some of these include:</p>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>Apache Parquet</li>\n",
    "    <li>Feather</li>\n",
    "    <li>Pickle</li>\n",
    "    </ul>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Pickle</b> is a common and useful Python file format.  Python will store the object itself to a file; including metadata, dependent package information, etc.  Any arbitrary object can be pickled and unpickled.  Have care, as there is a big security risk in that a pickle could contain malicious code that can execute upon object instantiation.  Never unpickle from an untrusted source.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35d98dee",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pickle the merged dataframe:\n",
    "\n",
    "df_merge.to_pickle('data/df_merge.zip') # Adding a zip extension will automatically compress the file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "841b4f78",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read the pickle and display the data:\n",
    "df_unpickle = pd.read_pickle('data/df_merge.zip')\n",
    "df_unpickle.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3baec598",
   "metadata": {},
   "source": [
    "<b style = 'font-size:18px;font-family:Arial;color:#00233C'>Excel</b>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Excel continues to be the defacto standard for working with data in a desktop tool.  Pandas provides extremely flexible capabilities for outputting Excel format, including style/formatting, layout, tabs, etc.  Here is a simple example:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f868f490",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_merge.to_excel('my_excel.xlsx', sheet_name = 'Sheet1', header = True, engine = 'openpyxl')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a3fc9a1",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>6. Cleanup</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>It is a good practice to remove the context that we created to connect to Vantage. The remove_context function removes the current context associated with the Vantage connection.remove_context() not only closes the connection but also garbage collects the intermediate views and tables created by teradataml. Teradata recommends calling remove_context() to end a session, so that intermediate views and tables created by teradataml are garbage collected.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4354f8f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "remove_context()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22c4b97d-32bb-41f0-8c71-2bd7e0947343",
   "metadata": {},
   "source": [
    "<footer style=\"padding-bottom:35px; background:#f9f9f9; border-bottom:3px solid #00233C\">\n",
    "    <div style=\"float:left;margin-top:14px\">ClearScape Analytics™</div>\n",
    "    <div style=\"float:right;\">\n",
    "        <div style=\"float:left; margin-top:14px\">\n",
    "            Copyright © Teradata Corporation - 2023. All Rights Reserved\n",
    "        </div>\n",
    "    </div>\n",
    "</footer>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}