UseCases/Vantage_Analytics_Library/VAL_Overview_SQL.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e575b8d0-c385-4c77-87a6-f2d258be9fee",
   "metadata": {},
   "source": [
    "<header>\n",
    "   <p  style='font-size:36px;font-family:Arial; color:#F0F0F0; background-color: #00233c; padding-left: 20pt; padding-top: 20pt;padding-bottom: 10pt; padding-right: 20pt;'>\n",
    "       Overview: Vantage Analytic Library (VAL)\n",
    "  <br>\n",
    "       <img id=\"teradata-logo\" src=\"https://storage.googleapis.com/clearscape_analytics_demo_data/DEMO_Logo/teradata.svg\" alt=\"Teradata\" style=\"width: 125px; height: auto; margin-top: 20pt;\">\n",
    "    </p>\n",
    "</header>\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>Introduction</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Vantage Analytics Library (VAL) is a set of over 50 functions for analyzing data content, performing hypothesis tests and executing advanced analytics. Teradata has been evolving the sophistication of these tools for decades and has now packaged the functionality into a library that is accessed for in-database execution by a stored procedure call.   The stored procedure is passed the name of a function, a list of columns and a set of parameters for the function.</p>\n",
    " \n",
    "<img id=\"Calling-VAL\" src=\"images/VAL_Calling.png\" alt=\"VAL\" width=\"420\" align=\"right\" />    \n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The stored procedure \"td_analyze( ... )\"  is in the VAL database on Vantage. It can be called by:</p>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'> A SQL statement that you write</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Using the Python teradataml object valib</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'> Using the R package tdplyr</li>\n",
    "   \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>With VAL, you perform complex processing of the data in the database where it is stored instead of the time and complexity involved in exporting the data out of the database and having a client platform with enough storage and processing capacity to perform the analysis. </p>\n",
    "    \n",
    "[![View Video](images/Movie_icon.png)](https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwjO4cKK-u77AhUhM0QIHbyXAT8QwqsBegQIGRAG&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DqxNw6sM0amg&usg=AOvVaw2e6UgLGXgWImNTIIPvARhV)\n",
    "[... Video Introduction to VAL](https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwjO4cKK-u77AhUhM0QIHbyXAT8QwqsBegQIGRAG&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DqxNw6sM0amg&usg=AOvVaw2e6UgLGXgWImNTIIPvARhV)\n",
    "    \n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>Try a simple example</b></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>There are many descriptions and sample executions in other notebooks (listed at the end of this notebook), but it would help to show a simple example to understand the concept of using VAL.</p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Click in the following cell, press shift/enter to provide your password.  Then click the next cell to continue</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c3466ef-e60f-4741-b9cc-501eba8e60bf",
   "metadata": {},
   "outputs": [],
   "source": [
    "%connect local, hidewarnings=true"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4edfba07-d529-4501-b87f-6abe4b3094fb",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The following will use the 'Values' function to analyze one of the built-in sets of data in Vantage's DBC database showing space used by tables. </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bb1e30f2-77a3-4467-8eb9-176e2ed4e269",
   "metadata": {},
   "outputs": [],
   "source": [
    "call val.td_analyze('Values',\n",
    "                         'database=dbc;\n",
    "                          tablename=tablesize;\n",
    "                          columns=all;\n",
    "                          uniques=true');"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04bd5008-7380-470d-9c52-0a7b8502f810",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The above output shows there are 6 columns in dbc.tablesize. The statistics will vary depending on how you've used your environment so far, but there are usually about 6700 rows. There may be 1600-1700 unique values of TableName. Every table on the system is spread across 4 AMPs on this tiny demo system which is why there are 4 unique values for Vproc.</p>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10b8d278-9bd9-4e1a-a02f-683d429a9563",
   "metadata": {},
   "source": [
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>List of Vantage Analytics Library Functions</b></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>There are more functions available than just 'Values'.  Here is a list:</p>\n",
    "\n",
    "<img id=\"VAL-Functions\" src=\"images/VAL_Functions.png\" alt=\"VAL\" width=\"1000\" />   \n",
    "\n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>Workbooks providing more details</b></p>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>VAL Descriptive Statistics and Transformation SQL function</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>This notebook demonstrates the use Descriptive Statistics and Data Transformation Functions which are useful for the initial processing of data to understand its makeup</p>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>VAL Hypothesis Test SQL functions</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Statistical tests provide a means of testing whether the outcome of an experiment could have been accidental. Numerous tests are provided in-database such as Parametric, Binomial, Contingency, Kolmogorov/Smirnoff and Rank tests.</p>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>VAL Analytic and Machine Learning Algorithms and Scoring</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>This notebook demonstrates the use of Vantage Analytic Library for Regression, Factor Analysis, Decision Trees, K-Means, and Association Rules/Sequence Analysis. </p>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>VAL teradataml Demo</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>This notebook shows the power of Vantage and the Analytic Library through a series of functions that use the Teradata SQLE to perform descriptive statistics, data engineering and transformation, model building, model evaluation and model scoring. All Analytic Library functions are called through Python using the teradataml interfaces for VAL. This notebook covers a subset of the function demonstrated in the other 3 VAL  SQL notebooks, but implemented with Python.</p>\n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>Vantage Analytics Library Reference Material</b></p>\n",
    "\n",
    "[Search for Vantage Analytics Library Documentation](https://docs.teradata.com/search/all?query=Vantage+Analytics+Library&content-lang=en-US)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "971b93b2-daf1-42e8-bf6d-fe6bdb097e9b",
   "metadata": {},
   "source": [
    "<footer style=\"padding-bottom:35px; background:#f9f9f9; border-bottom:3px solid #00233C\">\n",
    "    <div style=\"float:left;margin-top:14px\">ClearScape Analytics™</div>\n",
    "    <div style=\"float:right;\">\n",
    "        <div style=\"float:left; margin-top:14px\">\n",
    "            Copyright © Teradata Corporation - 2023. All Rights Reserved\n",
    "        </div>\n",
    "    </div>\n",
    "</footer>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Teradata SQL",
   "language": "Teradata SQL",
   "name": "teradatasql"
  },
  "language_info": {
   "codemirror_mode": "Teradata SQL",
   "file_extension": ".tdrs",
   "mimetype": "application/vnd.teradata.resultset",
   "name": "Teradata SQL",
   "nbconvert_exporter": "",
   "pygments_lexer": "",
   "version": "16.20"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}