00_Intro/bedrock_boto3_setup.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "88a5ab2f-d044-4956-b75b-7408d9c3e323",
   "metadata": {},
   "source": [
    "# Amazon Bedrock boto3 Setup\n",
    "\n",
    "> *This notebook should work well with the **`Data Science 3.0`** kernel in SageMaker Studio*\n",
    "\n",
    "---\n",
    "\n",
    "In this demo notebook, we demonstrate how to use the [`boto3` Python SDK](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) to work with [Amazon Bedrock](https://aws.amazon.com/bedrock/) Foundation Models.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6aeedd9f-f0a3-4f8e-934d-22f6f7a89de5",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "\n",
    "Run the cells in this section to install the packages needed by the notebooks in this workshop. ⚠️ You will see pip dependency errors, you can safely ignore these errors. ⚠️\n",
    "\n",
    "IGNORE ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "108c611c-7246-45c4-9f1e-76888b5076eb",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install --no-build-isolation --force-reinstall \\\n",
    "    \"boto3==1.28.57\" \\\n",
    "    \"awscli==1.29.57\" \\\n",
    "    \"botocore==1.31.57\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f1c8940",
   "metadata": {},
   "source": [
    "This notebook demonstrates invoking Bedrock models directly using the AWS SDK, but for later notebooks in the workshop you'll also need to install other packages.\n",
    "\n",
    "Here are links for some packages you might not be familiar with:\n",
    "\n",
    "- [FAISS](https://github.com/facebookresearch/faiss), to store vector embeddings\n",
    "- [sqlalchemy](https://www.sqlalchemy.org/), SQLAlchemy is the Python SQL toolkit \n",
    "- [Pinecone](http://pinecone.io), to store vector embeddings\n",
    "- [PyPDF](https://pypi.org/project/pypdf/), for handling PDF files\n",
    "- [IPyWidgets](https://ipywidgets.readthedocs.io/en/stable/), for interactive UI widgets in the notebook\n",
    "- [NeMo-Guardrails](https://github.com/NVIDIA/NeMo-Guardrails) an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd4a2efa",
   "metadata": {},
   "source": [
    "### text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12a834cf",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install --quiet \\\n",
    "    \"langchain==0.0.350\" \\\n",
    "    \"transformers==4.24\" \\\n",
    "    sqlalchemy -U \\\n",
    "    \"faiss-cpu==1.7.4\" \\\n",
    "    \"pypdf==3.8\" \\\n",
    "    pinecone-client \\\n",
    "    apache-beam \\\n",
    "    datasets \\\n",
    "    tiktoken \\\n",
    "    \"ipywidgets==7\" \\\n",
    "    matplotlib \\\n",
    "    anthropic \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75fabac3",
   "metadata": {},
   "source": [
    "### agents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fbe0a18f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install --quiet \\\n",
    "    xmltodict==0.13.0  \\\n",
    "    duckduckgo-search  \\\n",
    "    yfinance  \\\n",
    "    pandas_datareader  \\\n",
    "    langchain_experimental \\\n",
    "    pysqlite3-binary \\\n",
    "    google-search-results\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fcb0708",
   "metadata": {},
   "source": [
    "### entity extraction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4ef71a19",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install --quiet beautifulsoup4\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23c0b4a7",
   "metadata": {},
   "source": [
    "### image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7bce99bd",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": "%pip install --quiet \"pillow==9.5\"\n"
  },
  {
   "cell_type": "markdown",
   "id": "0bc989d0",
   "metadata": {},
   "source": [
    "### guardrails"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d12e5c9a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "apt-get update && apt-get install g++ -y\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fbf0fde6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "\n",
    "%pip install -qU --no-cache-dir nemoguardrails==0.5.0\n",
    "\n",
    "%pip install -qU \"faiss-cpu==1.7.4\" \\\n",
    "                      \"langchain==0.0.350\" \\\n",
    "                      \"pypdf==3.8\" \\\n",
    "                      \"ipywidgets==7\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "298bf02a",
   "metadata": {},
   "source": [
    "### Restart Kernel "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f645bbbb-43ef-460a-8e8e-7b06b3b3b102",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# restart kernel\n",
    "from IPython.core.display import HTML\n",
    "HTML(\"<script>Jupyter.notebook.kernel.restart()</script>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27610c0f-7de6-4440-8f76-decf30e3c5ca",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Create the boto3 client\n",
    "\n",
    "Interaction with the Bedrock API is done via the AWS SDK for Python: [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html).\n",
    "\n",
    "Depending on your environment, you might need to customize the setup when creating your Bedrock service client. To help with this, we've provided a `get_bedrock_client()` utility method that supports passing in different options. You can find the implementation in [../utils/bedrock.py](../utils/bedrock.py)\n",
    "\n",
    "#### Use different clients\n",
    "The boto3 provides different clients for Amazon Bedrock to perform different actions. The actions for [`InvokeModel`](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) and [`InvokeModelWithResponseStream`](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html) are supported by Amazon Bedrock Runtime where as other operations, such as [ListFoundationModels](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_ListFoundationModels.html), are handled via [Amazon Bedrock client](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_Operations_Amazon_Bedrock.html).\n",
    "\n",
    "The `get_bedrock_client()` method accepts `runtime` (default=True) parameter to return either `bedrock` or `bedrock-runtime` client.\n",
    "\n",
    "#### Use the default credential chain\n",
    "\n",
    "If you are running this notebook from [Amazon Sagemaker Studio](https://aws.amazon.com/sagemaker/studio/) and your Sagemaker Studio [execution role](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html) has permissions to access Bedrock you can just run the cells below as-is. This is also the case if you are running these notebooks from a computer whose default AWS credentials have access to Bedrock.\n",
    "\n",
    "#### Use a different AWS Region\n",
    "\n",
    "If you're running this notebook from your own computer or a SageMaker notebook in a different AWS Region from where Bedrock is set up, you can un-comment the `os.environ['AWS_DEFAULT_REGION']` line below and specify the region to use.\n",
    "\n",
    "#### Use a specific profile\n",
    "\n",
    "In case you're running this notebook from your own computer where you have setup the AWS CLI with multiple profiles, and the profile which has access to Bedrock is not the default one, you can un-comment the `os.environ['AWS_PROFILE']` line below and specify the profile to use.\n",
    "\n",
    "#### Use a different role\n",
    "\n",
    "In case you or your company has setup a specific, separate [IAM Role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) to access Bedrock, you can specify it by un-commenting the `os.environ['BEDROCK_ASSUME_ROLE']` line below. Ensure that your current user or role have permissions to [assume](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) such role.\n",
    "\n",
    "#### A note about `langchain`\n",
    "\n",
    "The Bedrock classes provided by `langchain` create a Bedrock boto3 client by default. To customize your Bedrock configuration, we recommend to explicitly create the Bedrock client using the method below, and pass it to the [`langchain.Bedrock`](https://python.langchain.com/docs/integrations/llms/bedrock) class instantiation method using `client=boto3_bedrock`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae2b2a05-78a9-40ca-9b5e-121030f9ede1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import os\n",
    "import sys\n",
    "\n",
    "import boto3\n",
    "import botocore\n",
    "\n",
    "module_path = \"..\"\n",
    "sys.path.append(os.path.abspath(module_path))\n",
    "from utils import bedrock, print_ww\n",
    "\n",
    "\n",
    "# ---- ⚠️ Un-comment and edit the below lines as needed for your AWS setup ⚠️ ----\n",
    "\n",
    "# If you are running SageMaker in another region than us-east-1, and Bedrock models are enabled in us-east-1, \n",
    "# you need to uncomment the below line:\n",
    "\n",
    "# os.environ[\"AWS_DEFAULT_REGION\"] = \"us-east-1\" \n",
    "\n",
    "# Below lines should be used in local environment only:\n",
    "\n",
    "# os.environ[\"AWS_PROFILE\"] = \"<YOUR_PROFILE>\"\n",
    "# os.environ[\"BEDROCK_ASSUME_ROLE\"] = \"<YOUR_ROLE_ARN>\"  # E.g. \"arn:aws:...\"\n",
    "\n",
    "\n",
    "boto3_bedrock = bedrock.get_bedrock_client(\n",
    "    assumed_role=os.environ.get(\"BEDROCK_ASSUME_ROLE\", None),\n",
    "    region=os.environ.get(\"AWS_DEFAULT_REGION\", None),\n",
    "    runtime=False\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e9174c4-326a-463e-92e1-8c7e47111269",
   "metadata": {},
   "source": [
    "#### Validate the connection\n",
    "\n",
    "We can check the client works by trying out the `list_foundation_models()` method, which will tell us all the models available for us to use "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f67b4466-12ff-4975-9811-7a19c6206604",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "boto3_bedrock.list_foundation_models()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f690043-df45-448f-8fa6-1ea8b06f1087",
   "metadata": {
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "## `InvokeModel` body and output\n",
    "\n",
    "The `invoke_model()` method of the Amazon Bedrock runtime client (`InvokeModel` API) will be the primary method we use for most of our Text Generation and Processing tasks - whichever model we're using.\n",
    "\n",
    "Although the method is shared, the format of input and output varies depending on the foundation model used - as described below:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4650fa3-a831-4039-9fd6-749926d35979",
   "metadata": {},
   "source": [
    "### Amazon Titan Large\n",
    "\n",
    "#### Input\n",
    "```json\n",
    "{   \n",
    "    \"inputText\": \"<prompt>\",\n",
    "    \"textGenerationConfig\" : { \n",
    "        \"maxTokenCount\": 512,\n",
    "        \"stopSequences\": [],\n",
    "        \"temperature\": 0.1,  \n",
    "        \"topP\": 0.9\n",
    "    }\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"inputTextTokenCount\": 613,\n",
    "    \"results\": [{\n",
    "        \"tokenCount\": 219,\n",
    "        \"outputText\": \"<output>\"\n",
    "    }]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ca0c329-ff58-483e-8abd-63867950c14b",
   "metadata": {},
   "source": [
    "### AI21 Jurassic (Grande and Jumbo) \n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"prompt\": \"<prompt>\",\n",
    "    \"maxTokens\": 200,\n",
    "    \"temperature\": 0.5,\n",
    "    \"topP\": 0.5,\n",
    "    \"stopSequences\": [],\n",
    "    \"countPenalty\": {\"scale\": 0},\n",
    "    \"presencePenalty\": {\"scale\": 0},\n",
    "    \"frequencyPenalty\": {\"scale\": 0}\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"id\": 1234,\n",
    "    \"prompt\": {\n",
    "        \"text\": \"<prompt>\",\n",
    "        \"tokens\": [\n",
    "            {\n",
    "                \"generatedToken\": {\n",
    "                    \"token\": \"\\u2581who\\u2581is\",\n",
    "                    \"logprob\": -12.980147361755371,\n",
    "                    \"raw_logprob\": -12.980147361755371\n",
    "                },\n",
    "                \"topTokens\": null,\n",
    "                \"textRange\": {\"start\": 0, \"end\": 6}\n",
    "            },\n",
    "            //...\n",
    "        ]\n",
    "    },\n",
    "    \"completions\": [\n",
    "        {\n",
    "            \"data\": {\n",
    "                \"text\": \"<output>\",\n",
    "                \"tokens\": [\n",
    "                    {\n",
    "                        \"generatedToken\": {\n",
    "                            \"token\": \"<|newline|>\",\n",
    "                            \"logprob\": 0.0,\n",
    "                            \"raw_logprob\": -0.01293118204921484\n",
    "                        },\n",
    "                        \"topTokens\": null,\n",
    "                        \"textRange\": {\"start\": 0, \"end\": 1}\n",
    "                    },\n",
    "                    //...\n",
    "                ]\n",
    "            },\n",
    "            \"finishReason\": {\"reason\": \"endoftext\"}\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7adb6bee-7654-4269-9127-9afa4e823454",
   "metadata": {},
   "source": [
    "### Anthropic Claude\n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"prompt\": \"\\n\\nHuman:<prompt>\\n\\nAnswer:\",\n",
    "    \"max_tokens_to_sample\": 300,\n",
    "    \"temperature\": 0.5,\n",
    "    \"top_k\": 250,\n",
    "    \"top_p\": 1,\n",
    "    \"stop_sequences\": [\"\\n\\nHuman:\"]\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"completion\": \"<output>\",\n",
    "    \"stop_reason\": \"stop_sequence\"\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb94aa2f-30da-499b-b3f5-02f102dbb1ea",
   "metadata": {},
   "source": [
    "### Stability AI Stable Diffusion XL\n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"text_prompts\": [\n",
    "        {\"text\": \"this is where you place your input text\"}\n",
    "    ],\n",
    "    \"cfg_scale\": 10,\n",
    "    \"seed\": 0,\n",
    "    \"steps\": 50\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{ \n",
    "    \"result\": \"success\", \n",
    "    \"artifacts\": [\n",
    "        {\n",
    "            \"seed\": 123, \n",
    "            \"base64\": \"<image in base64>\",\n",
    "            \"finishReason\": \"SUCCESS\"\n",
    "        },\n",
    "        //...\n",
    "    ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80f4adca-cfc4-439b-84b7-e528398684e3",
   "metadata": {
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "## Common inference parameter definitions\n",
    "\n",
    "### Randomness and Diversity\n",
    "\n",
    "Foundation models support the following parameters to control randomness and diversity in the \n",
    "response.\n",
    "\n",
    "**Temperature** – Large language models use probability to construct the words in a sequence. For any \n",
    "given next word, there is a probability distribution of options for the next word in the sequence. When \n",
    "you set the temperature closer to zero, the model tends to select the higher-probability words. When \n",
    "you set the temperature further away from zero, the model may select a lower-probability word.\n",
    "\n",
    "In technical terms, the temperature modulates the probability density function for the next tokens, \n",
    "implementing the temperature sampling technique. This parameter can deepen or flatten the density \n",
    "function curve. A lower value results in a steeper curve with more deterministic responses, and a higher \n",
    "value results in a flatter curve with more random responses.\n",
    "\n",
    "**Top K** – Temperature defines the probability distribution of potential words, and Top K defines the cut \n",
    "off where the model no longer selects the words. For example, if K=50, the model selects from 50 of the \n",
    "most probable words that could be next in a given sequence. This reduces the probability that an unusual \n",
    "word gets selected next in a sequence.\n",
    "In technical terms, Top K is the number of the highest-probability vocabulary tokens to keep for Top-\n",
    "K-filtering - This limits the distribution of probable tokens, so the model chooses one of the highest-\n",
    "probability tokens.\n",
    "\n",
    "**Top P** – Top P defines a cut off based on the sum of probabilities of the potential choices. If you set Top \n",
    "P below 1.0, the model considers the most probable options and ignores less probable ones. Top P is \n",
    "similar to Top K, but instead of capping the number of choices, it caps choices based on the sum of their \n",
    "probabilities.\n",
    "For the example prompt \"I hear the hoof beats of ,\" you may want the model to provide \"horses,\" \n",
    "\"zebras\" or \"unicorns\" as the next word. If you set the temperature to its maximum, without capping \n",
    "Top K or Top P, you increase the probability of getting unusual results such as \"unicorns.\" If you set the \n",
    "temperature to 0, you increase the probability of \"horses.\" If you set a high temperature and set Top K or \n",
    "Top P to the maximum, you increase the probability of \"horses\" or \"zebras,\" and decrease the probability \n",
    "of \"unicorns.\"\n",
    "\n",
    "### Length\n",
    "\n",
    "The following parameters control the length of the generated response.\n",
    "\n",
    "**Response length** – Configures the minimum and maximum number of tokens to use in the generated \n",
    "response.\n",
    "\n",
    "**Length penalty** – Length penalty optimizes the model to be more concise in its output by penalizing \n",
    "longer responses. Length penalty differs from response length as the response length is a hard cut off for \n",
    "the minimum or maximum response length.\n",
    "\n",
    "In technical terms, the length penalty penalizes the model exponentially for lengthy responses. 0.0 \n",
    "means no penalty. Set a value less than 0.0 for the model to generate longer sequences, or set a value \n",
    "greater than 0.0 for the model to produce shorter sequences.\n",
    "\n",
    "### Repetitions\n",
    "\n",
    "The following parameters help control repetition in the generated response.\n",
    "\n",
    "**Repetition penalty (presence penalty)** – Prevents repetitions of the same words (tokens) in responses. \n",
    "1.0 means no penalty. Greater than 1.0 decreases repetition."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce22c308-ebbf-4ef5-a823-832b7c236e31",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Try out the models\n",
    "\n",
    "With some theory out of the way, let's see the models in action! Run the cells below to see basic, synchronous example invocations for each model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6a0a79b9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "bedrock_runtime = bedrock.get_bedrock_client(\n",
    "    assumed_role=os.environ.get(\"BEDROCK_ASSUME_ROLE\", None),\n",
    "    region=os.environ.get(\"AWS_DEFAULT_REGION\", None)\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "893872fe-04fa-4f09-9736-6c6173ec1fc2",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Amazon Titan Large"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7df55eed-a3cf-426c-95ea-ec60dade6477",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# If you'd like to try your own prompt, edit this parameter!\n",
    "prompt_data = \"\"\"Command: Write me a blog about making strong business decisions as a leader.\n",
    "\n",
    "Blog:\n",
    "\"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd2bb671-6b10-4948-9e5e-95d6ced3b86f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "try:\n",
    "\n",
    "    body = json.dumps({\"inputText\": prompt_data})\n",
    "    modelId = \"amazon.titan-tg1-large\"\n",
    "    accept = \"application/json\"\n",
    "    contentType = \"application/json\"\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    print(response_body.get(\"results\")[0].get(\"outputText\"))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d7c0fe6-576a-4380-89aa-726bab5d65ff",
   "metadata": {},
   "source": [
    "### Anthropic Claude"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a725de2-bdea-4d86-b12d-d1d7cdda010b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# If you'd like to try your own prompt, edit this parameter!\n",
    "prompt_data = \"\"\"Human: Write me a blog about making strong business decisions as a leader.\n",
    "\n",
    "Assistant:\n",
    "\"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ba33ac0-fa16-4c4f-b882-e838d0cb5830",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "body = json.dumps({\"prompt\": prompt_data, \"max_tokens_to_sample\": 500})\n",
    "modelId = \"anthropic.claude-instant-v1\"  # change this to use a different version from the model provider\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    print(response_body.get(\"completion\"))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed0e3144-c6df-400d-aab1-1540614dbbde",
   "metadata": {},
   "source": [
    "### AI21 Jurassic Grande"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c02d1585-945e-45d1-99d2-171e956138f8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "body = json.dumps({\"prompt\": prompt_data, \"maxTokens\": 200})\n",
    "modelId = \"ai21.j2-mid-v1\"  # change this to use a different version from the model provider\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    print(response_body.get(\"completions\")[0].get(\"data\").get(\"text\"))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc498bea",
   "metadata": {},
   "source": [
    "### Stability Stable Diffusion XL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "173e51a2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "prompt_data = \"a landscape with trees\"\n",
    "body = json.dumps({\n",
    "    \"text_prompts\": [{\"text\": prompt_data}],\n",
    "    \"cfg_scale\": 10,\n",
    "    \"seed\": 20,\n",
    "    \"steps\": 50\n",
    "})\n",
    "modelId = \"stability.stable-diffusion-xl-v1\"\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    print(response_body[\"result\"])\n",
    "    print(f'{response_body.get(\"artifacts\")[0].get(\"base64\")[0:80]}...')\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a271fa6-13fd-480a-87a5-3702d29a5c43",
   "metadata": {},
   "source": [
    "**Note:** The output is a [base64 encoded](https://docs.python.org/3/library/base64.html) string of the image data. You can use any image processing library (such as [Pillow](https://pillow.readthedocs.io/en/stable/)) to decode the image as in the example below:\n",
    "\n",
    "```python\n",
    "import base64\n",
    "import io\n",
    "from PIL import Image\n",
    "\n",
    "base_64_img_str = response_body.get(\"artifacts\")[0].get(\"base64\")\n",
    "image = Image.open(io.BytesIO(base64.decodebytes(bytes(base_64_img_str, \"utf-8\"))))\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45072848-000a-4c22-8f08-2647e5c2230e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import base64\n",
    "import io\n",
    "from PIL import Image\n",
    "\n",
    "base_64_img_str = response_body.get(\"artifacts\")[0].get(\"base64\")\n",
    "image = Image.open(io.BytesIO(base64.decodebytes(bytes(base_64_img_str, \"utf-8\"))))\n",
    "image"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4621a301-53e4-4182-9fce-8ee422813e9d",
   "metadata": {},
   "source": [
    "## Generate streaming output\n",
    "\n",
    "For large language models, it can take noticeable time to generate long output sequences. Rather than waiting for the entire response to be available, latency-sensitive applications may like to **stream** the response to users.\n",
    "\n",
    "Run the code below to see how you can achieve this with Bedrock's `invoke_model_with_response_stream()` method - returning the response body in separate chunks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c69627e3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from IPython.display import clear_output, display, display_markdown, Markdown\n",
    "prompt_data = \"\"\"Command: Write me a blog about making strong business decisions as a leader.\n",
    "\n",
    "Blog:\n",
    "\"\"\"\n",
    "\n",
    "body = json.dumps({\"inputText\": prompt_data})\n",
    "modelId = \"amazon.titan-tg1-large\"  # (Change this, and the request body, to try different models)\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model_with_response_stream(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    stream = response.get('body')\n",
    "    output = []\n",
    "\n",
    "    if stream:\n",
    "        for event in stream:\n",
    "            chunk = event.get('chunk')\n",
    "            if chunk:\n",
    "                chunk_obj = json.loads(chunk.get('bytes').decode())\n",
    "                text = chunk_obj['outputText']\n",
    "                clear_output(wait=True)\n",
    "                output.append(text)\n",
    "                display_markdown(Markdown(''.join(output)))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adf097eb-1bfa-41f1-8135-cdd4ad6e9983",
   "metadata": {},
   "source": [
    "### Anthropic Claude"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd800ef5-78f7-43d4-b73e-c37c7609cb40",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# If you'd like to try your own prompt, edit this parameter!\n",
    "prompt_data = \"\"\"Human: Write me a blog about making strong business decisions as a leader.\n",
    "\n",
    "Assistant:\n",
    "\"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "be7fbbe8-78b8-4071-b80c-3f954185fad8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from IPython.display import clear_output, display, display_markdown, Markdown\n",
    "\n",
    "body = json.dumps({\"prompt\": prompt_data, \"max_tokens_to_sample\": 500})\n",
    "modelId = \"anthropic.claude-instant-v1\"  # (Change this to try different model versions)\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model_with_response_stream(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    stream = response.get('body')\n",
    "    output = []\n",
    "\n",
    "    if stream:\n",
    "        for event in stream:\n",
    "            chunk = event.get('chunk')\n",
    "            if chunk:\n",
    "                chunk_obj = json.loads(chunk.get('bytes').decode())\n",
    "                text = chunk_obj['completion']\n",
    "                clear_output(wait=True)\n",
    "                output.append(text)\n",
    "                display_markdown(Markdown(''.join(output)))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ef3451d-b66a-4b11-a1ed-734bf9e7bbec",
   "metadata": {},
   "source": [
    "## Generate embeddings\n",
    "\n",
    "Use text embeddings to convert text into meaningful vector representations. You input a body of text \n",
    "and the output is a (1 x n) vector. You can use embedding vectors for a wide variety of applications. \n",
    "Bedrock currently offers Titan Embeddings for text embedding that supports text similarity (finding the \n",
    "semantic similarity between bodies of text) and text retrieval (such as search).\n",
    "\n",
    "At the time of writing you can use `amazon.titan-embed-text-v1` as embedding model via the API. The input text size is 8192 tokens and the output vector length is 1536.\n",
    "\n",
    "To use a text embeddings model, use the InvokeModel API operation or the Python SDK.\n",
    "Use InvokeModel to retrieve the vector representation of the input text from the specified model.\n",
    "\n",
    "\n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"inputText\": \"<text>\"\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"embedding\": []\n",
    "}\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9645dbd8",
   "metadata": {},
   "source": [
    "Let's see how to generate embeddings of some text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1085cc56",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "prompt_data = \"Amazon Bedrock supports foundation models from industry-leading providers such as \\\n",
    "AI21 Labs, Anthropic, Stability AI, and Amazon. Choose the model that is best suited to achieving \\\n",
    "your unique goals.\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c54b424",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "body = json.dumps({\"inputText\": prompt_data})\n",
    "modelId = \"amazon.titan-embed-text-v1\"  # (Change this to try different embedding models)\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    embedding = response_body.get(\"embedding\")\n",
    "    print(f\"The embedding vector has {len(embedding)} values\\n{embedding[0:3]+['...']+embedding[-3:]}\")\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a48a0e8-147d-4525-a6b2-68a09af1b2c4",
   "metadata": {},
   "source": [
    "## Next steps\n",
    "\n",
    "In this notebook we showed some basic examples of invoking Amazon Bedrock models using the AWS Python SDK. You're now ready to explore the other labs to dive deeper on different use-cases and patterns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f8bb76df-4e99-4ebe-a954-53992ad317dc",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:eu-central-1:936697816551:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}