fine-tune-flan-t5-with-peft-lora-deeplearning-ai.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b7241daa",
   "metadata": {
    "papermill": {
     "duration": 0.018932,
     "end_time": "2023-07-13T20:35:30.951021",
     "exception": false,
     "start_time": "2023-07-13T20:35:30.932089",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Fine-Tune a Generative AI Model for Dialogue Summarization"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "607f0eb9",
   "metadata": {
    "papermill": {
     "duration": 0.017999,
     "end_time": "2023-07-13T20:35:30.987414",
     "exception": false,
     "start_time": "2023-07-13T20:35:30.969415",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Here I am just sharing an interesting tutorial that was included as part of a course that I recently audited on Coursera.\n",
    "\n",
    "I thought that this tutorial did a great job of explaining parameter efficient fine-tuning methods (as well as providing examples for how to implement these methods).\n",
    "\n",
    "\n",
    "Source:\n",
    " - https://www.coursera.org/learn/generative-ai-with-llms\n",
    " - https://www.coursera.org/learn/generative-ai-with-llms/gradedLti/x0gc1/lab-2-fine-tune-a-generative-ai-model-for-dialogue-summarization\n",
    "\n",
    "https://creativecommons.org/licenses/by-sa/2.0/legalcode"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c086db8",
   "metadata": {
    "papermill": {
     "duration": 0.018701,
     "end_time": "2023-07-13T20:35:31.024314",
     "exception": false,
     "start_time": "2023-07-13T20:35:31.005613",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "In this notebook, you will fine-tune an existing LLM from Hugging Face for enhanced dialogue summarization. You will use the [FLAN-T5](https://huggingface.co/docs/transformers/model_doc/flan-t5) model, which provides a high quality instruction tuned model and can summarize text out of the box. To improve the inferences, you will explore a full fine-tuning approach and evaluate the results with ROUGE metrics. Then you will perform Parameter Efficient Fine-Tuning (PEFT), evaluate the resulting model and see that the benefits of PEFT outweigh the slightly-lower performance metrics."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f20e934",
   "metadata": {
    "papermill": {
     "duration": 0.017862,
     "end_time": "2023-07-13T20:35:31.060394",
     "exception": false,
     "start_time": "2023-07-13T20:35:31.042532",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Table of Contents"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d10995f0",
   "metadata": {
    "papermill": {
     "duration": 0.018145,
     "end_time": "2023-07-13T20:35:31.096527",
     "exception": false,
     "start_time": "2023-07-13T20:35:31.078382",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "- [ 1 - Set up Kernel, Load Required Dependencies, Dataset and LLM](#1)\n",
    "  - [ 1.1 - Set up Kernel and Required Dependencies](#1.1)\n",
    "  - [ 1.2 - Load Dataset and LLM](#1.2)\n",
    "  - [ 1.3 - Test the Model with Zero Shot Inferencing](#1.3)\n",
    "- [ 2 - Perform Full Fine-Tuning](#2)\n",
    "  - [ 2.1 - Preprocess the Dialog-Summary Dataset](#2.1)\n",
    "  - [ 2.2 - Fine-Tune the Model with the Preprocessed Dataset](#2.2)\n",
    "  - [ 2.3 - Evaluate the Model Qualitatively (Human Evaluation)](#2.3)\n",
    "  - [ 2.4 - Evaluate the Model Quantitatively (with ROUGE Metric)](#2.4)\n",
    "- [ 3 - Perform Parameter Efficient Fine-Tuning (PEFT)](#3)\n",
    "  - [ 3.1 - Setup the PEFT/LoRA model for Fine-Tuning](#3.1)\n",
    "  - [ 3.2 - Train PEFT Adapter](#3.2)\n",
    "  - [ 3.3 - Evaluate the Model Qualitatively (Human Evaluation)](#3.3)\n",
    "  - [ 3.4 - Evaluate the Model Quantitatively (with ROUGE Metric)](#3.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7672c22b",
   "metadata": {
    "papermill": {
     "duration": 0.017992,
     "end_time": "2023-07-13T20:35:31.132656",
     "exception": false,
     "start_time": "2023-07-13T20:35:31.114664",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='1'></a>\n",
    "## 1 - Set up Kernel, Load Required Dependencies, Dataset and LLM"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "563dd499",
   "metadata": {
    "papermill": {
     "duration": 0.017795,
     "end_time": "2023-07-13T20:35:31.168562",
     "exception": false,
     "start_time": "2023-07-13T20:35:31.150767",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='1.1'></a>\n",
    "### 1.1 - Set up Kernel and Required Dependencies"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b87e5e1",
   "metadata": {
    "papermill": {
     "duration": 0.017897,
     "end_time": "2023-07-13T20:35:31.204648",
     "exception": false,
     "start_time": "2023-07-13T20:35:31.186751",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now install the required packages for the LLM and datasets.\n",
    "\n",
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSIxMjUiIHZpZXdCb3g9IjAgMCA4MDAgMTI1IiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgogICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJmYWRlR3JhZGllbnQiIHgxPSIwIiB4Mj0iMSI+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNGMEYwRjAiLz4KICAgICAgICAgICAgPHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIiBzdG9wLW9wYWNpdHk9IjAiLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICAgIDxtYXNrIGlkPSJmYWRlTWFzayI+CiAgICAgICAgICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSI3NTAiIGhlaWdodD0iMTI1IiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSIxMjUiIGZpbGw9InVybCgjZmFkZUdyYWRpZW50KSIvPgogICAgICAgIDwvbWFzaz4KICAgIDwvZGVmcz4KICAgIDxwYXRoIGQ9Ik0zLDUwIEE1MCw1MCAwIDAgMSA1MywzIEw3OTcsMyBMNzk3LDk3IEw5Nyw5NyBMNTAsMTE1IEwzLDk3IFoiIGZpbGw9IiNGMEYwRjAiIHN0cm9rZT0iI0UwRTBFMCIgc3Ryb2tlLXdpZHRoPSIxIiBtYXNrPSJ1cmwoI2ZhZGVNYXNrKSIvPgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIGZpbGw9IiM1N2M0ZjgiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIxIi8+CiAgICA8Y2lyY2xlIGN4PSI1MCIgY3k9IjUwIiByPSIyNSIgZmlsbD0iI0YwRjBGMCIvPgogICAgPGxpbmUgeDE9IjUwIiB5MT0iNTAiIHgyPSI1MCIgeTI9IjMwIiBzdHJva2U9IiM1N2M0ZjgiIHN0cm9rZS13aWR0aD0iMyIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIi8+CiAgICA8bGluZSB4MT0iNTAiIHkxPSI1MCIgeDI9IjY1IiB5Mj0iNTAiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIzIiBzdHJva2UtbGluZWNhcD0icm91bmQiLz4KICAgIDx0ZXh0IHg9IjEwMCIgeT0iMzQiIGZvbnQtZmFtaWx5PSJBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxNCIgZmlsbD0iIzMzMzMzMyI+VGhlIG5leHQgY2VsbCBtYXkgdGFrZSBhIGZldyBtaW51dGVzIHRvIHJ1bi4gUGxlYXNlIGJlIHBhdGllbnQuPC90ZXh0PgogICAgPHRleHQgeD0iMTAwIiB5PSI1NiIgZm9udC1mYW1pbHk9IkFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjE0IiBmaWxsPSIjMzMzMzMzIj5JZ25vcmUgdGhlIHdhcm5pbmdzIGFuZCBlcnJvcnMsIGFsb25nIHdpdGggdGhlIG5vdGUgYWJvdXQgcmVzdGFydGluZyB0aGUga2VybmVsIGF0IHRoZSBlbmQuPC90ZXh0Pgo8L3N2Zz4K\" alt=\"Time alert open medium\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "15f91d8a",
   "metadata": {
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2023-07-13T20:35:31.242822Z",
     "iopub.status.busy": "2023-07-13T20:35:31.242353Z",
     "iopub.status.idle": "2023-07-13T20:37:57.408012Z",
     "shell.execute_reply": "2023-07-13T20:37:57.406739Z"
    },
    "papermill": {
     "duration": 146.187542,
     "end_time": "2023-07-13T20:37:57.410330",
     "exception": false,
     "start_time": "2023-07-13T20:35:31.222788",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: pip in /opt/conda/lib/python3.10/site-packages (23.1.2)\r\n",
      "Note: you may need to restart the kernel to use updated packages.\n",
      "Note: you may need to restart the kernel to use updated packages.\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install --upgrade pip\n",
    "%pip install --disable-pip-version-check \\\n",
    "    torch==1.13.1 \\\n",
    "    torchdata==0.5.1 --quiet\n",
    "\n",
    "%pip install \\\n",
    "    transformers==4.27.2 \\\n",
    "    datasets==2.11.0 \\\n",
    "    evaluate==0.4.0 \\\n",
    "    rouge_score==0.1.2 \\\n",
    "    loralib==0.1.1 \\\n",
    "    peft==0.3.0 --quiet"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2051b0d3",
   "metadata": {
    "papermill": {
     "duration": 0.018113,
     "end_time": "2023-07-13T20:37:57.447156",
     "exception": false,
     "start_time": "2023-07-13T20:37:57.429043",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSI1MCIgdmlld0JveD0iMCAwIDgwMCA1MCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KICAgIDxkZWZzPgogICAgICAgIDxsaW5lYXJHcmFkaWVudCBpZD0iZmFkZUdyYWRpZW50IiB4MT0iMCIgeDI9IjEiPgogICAgICAgICAgICA8c3RvcCBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIi8+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iI0YwRjBGMCIgc3RvcC1vcGFjaXR5PSIwIi8+CiAgICAgICAgPC9saW5lYXJHcmFkaWVudD4KICAgICAgICA8bWFzayBpZD0iZmFkZU1hc2siPgogICAgICAgICAgICA8cmVjdCB4PSIwIiB5PSIwIiB3aWR0aD0iNzUwIiBoZWlnaHQ9IjUwIiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSI1MCIgZmlsbD0idXJsKCNmYWRlR3JhZGllbnQpIi8+CiAgICAgICAgPC9tYXNrPgogICAgPC9kZWZzPgogICAgPHBhdGggZD0iTTI1LDUwIFEwLDUwIDAsMjUgTDUwLDMgTDk3LDI1IEw3OTcsMjUgTDc5Nyw1MCBMMjUsNTAgWiIgZmlsbD0iI0YwRjBGMCIgc3Ryb2tlPSIjRTBFMEUwIiBzdHJva2Utd2lkdGg9IjEiIG1hc2s9InVybCgjZmFkZU1hc2spIi8+Cjwvc3ZnPgo=\" alt=\"Time alert close\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "5fbb9438",
   "metadata": {
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2023-07-13T20:37:57.485315Z",
     "iopub.status.busy": "2023-07-13T20:37:57.484982Z",
     "iopub.status.idle": "2023-07-13T20:38:39.096066Z",
     "shell.execute_reply": "2023-07-13T20:38:39.094882Z"
    },
    "papermill": {
     "duration": 41.63342,
     "end_time": "2023-07-13T20:38:39.098843",
     "exception": false,
     "start_time": "2023-07-13T20:37:57.465423",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting awscli\r\n",
      "  Downloading awscli-1.29.3-py3-none-any.whl (4.1 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.1/4.1 MB\u001b[0m \u001b[31m28.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting botocore==1.31.3 (from awscli)\r\n",
      "  Downloading botocore-1.31.3-py3-none-any.whl (11.0 MB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m11.0/11.0 MB\u001b[0m \u001b[31m37.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hCollecting docutils<0.17,>=0.10 (from awscli)\r\n",
      "  Downloading docutils-0.16-py2.py3-none-any.whl (548 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m548.2/548.2 kB\u001b[0m \u001b[31m28.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25hRequirement already satisfied: s3transfer<0.7.0,>=0.6.0 in /opt/conda/lib/python3.10/site-packages (from awscli) (0.6.1)\r\n",
      "Collecting PyYAML<5.5,>=3.10 (from awscli)\r\n",
      "  Downloading PyYAML-5.4.1.tar.gz (175 kB)\r\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m175.1/175.1 kB\u001b[0m \u001b[31m706.5 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n",
      "\u001b[?25h  Installing build dependencies ... \u001b[?25l-\b \b\\\b \b|\b \b/\b \b-\b \bdone\r\n",
      "\u001b[?25h  Getting requirements to build wheel ... \u001b[?25l-\b \b\\\b \b|\b \bdone\r\n",
      "\u001b[?25h  Preparing metadata (pyproject.toml) ... \u001b[?25l-\b \bdone\r\n",
      "\u001b[?25hCollecting colorama<0.4.5,>=0.2.5 (from awscli)\r\n",
      "  Downloading colorama-0.4.4-py2.py3-none-any.whl (16 kB)\r\n",
      "Collecting rsa<4.8,>=3.1.2 (from awscli)\r\n",
      "  Downloading rsa-4.7.2-py3-none-any.whl (34 kB)\r\n",
      "Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /opt/conda/lib/python3.10/site-packages (from botocore==1.31.3->awscli) (1.0.1)\r\n",
      "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /opt/conda/lib/python3.10/site-packages (from botocore==1.31.3->awscli) (2.8.2)\r\n",
      "Requirement already satisfied: urllib3<1.27,>=1.25.4 in /opt/conda/lib/python3.10/site-packages (from botocore==1.31.3->awscli) (1.26.15)\r\n",
      "Requirement already satisfied: pyasn1>=0.1.3 in /opt/conda/lib/python3.10/site-packages (from rsa<4.8,>=3.1.2->awscli) (0.4.8)\r\n",
      "Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.10/site-packages (from python-dateutil<3.0.0,>=2.1->botocore==1.31.3->awscli) (1.16.0)\r\n",
      "Building wheels for collected packages: PyYAML\r\n",
      "  Building wheel for PyYAML (pyproject.toml) ... \u001b[?25l-\b \b\\\b \b|\b \b/\b \b-\b \b\\\b \bdone\r\n",
      "\u001b[?25h  Created wheel for PyYAML: filename=PyYAML-5.4.1-cp310-cp310-linux_x86_64.whl size=142131 sha256=ab040e5bbdbf000df3e6d57038c92aa4ece2f0cd052778aea3af8ed24cd400b3\r\n",
      "  Stored in directory: /root/.cache/pip/wheels/c7/0d/22/696ee92245ad710f506eee79bb05c740d8abccd3ecdb778683\r\n",
      "Successfully built PyYAML\r\n",
      "Installing collected packages: rsa, PyYAML, docutils, colorama, botocore, awscli\r\n",
      "  Attempting uninstall: rsa\r\n",
      "    Found existing installation: rsa 4.9\r\n",
      "    Uninstalling rsa-4.9:\r\n",
      "      Successfully uninstalled rsa-4.9\r\n",
      "  Attempting uninstall: PyYAML\r\n",
      "    Found existing installation: PyYAML 6.0\r\n",
      "    Uninstalling PyYAML-6.0:\r\n",
      "      Successfully uninstalled PyYAML-6.0\r\n",
      "  Attempting uninstall: docutils\r\n",
      "    Found existing installation: docutils 0.20.1\r\n",
      "    Uninstalling docutils-0.20.1:\r\n",
      "      Successfully uninstalled docutils-0.20.1\r\n",
      "  Attempting uninstall: colorama\r\n",
      "    Found existing installation: colorama 0.4.6\r\n",
      "    Uninstalling colorama-0.4.6:\r\n",
      "      Successfully uninstalled colorama-0.4.6\r\n",
      "  Attempting uninstall: botocore\r\n",
      "    Found existing installation: botocore 1.29.161\r\n",
      "    Uninstalling botocore-1.29.161:\r\n",
      "      Successfully uninstalled botocore-1.29.161\r\n",
      "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\r\n",
      "dask-cudf 23.6.1 requires cupy-cuda11x>=12.0.0, which is not installed.\r\n",
      "aiobotocore 2.5.2 requires botocore<1.29.162,>=1.29.161, but you have botocore 1.31.3 which is incompatible.\r\n",
      "bayesian-optimization 1.4.3 requires colorama>=0.4.6, but you have colorama 0.4.4 which is incompatible.\r\n",
      "boto3 1.26.100 requires botocore<1.30.0,>=1.29.100, but you have botocore 1.31.3 which is incompatible.\r\n",
      "dask-cuda 23.6.0 requires dask==2023.3.2, but you have dask 2023.7.0 which is incompatible.\r\n",
      "dask-cudf 23.6.1 requires dask==2023.3.2, but you have dask 2023.7.0 which is incompatible.\r\n",
      "distributed 2023.3.2.1 requires dask==2023.3.2, but you have dask 2023.7.0 which is incompatible.\r\n",
      "jupyterlab-lsp 4.2.0 requires jupyter-lsp>=2.0.0, but you have jupyter-lsp 1.5.1 which is incompatible.\r\n",
      "kfp 2.0.1 requires google-cloud-storage<3,>=2.2.1, but you have google-cloud-storage 1.44.0 which is incompatible.\r\n",
      "raft-dask 23.6.2 requires dask==2023.3.2, but you have dask 2023.7.0 which is incompatible.\r\n",
      "ydata-profiling 4.3.1 requires scipy<1.11,>=1.4.1, but you have scipy 1.11.1 which is incompatible.\u001b[0m\u001b[31m\r\n",
      "\u001b[0mSuccessfully installed PyYAML-5.4.1 awscli-1.29.3 botocore-1.31.3 colorama-0.4.4 docutils-0.16 rsa-4.7.2\r\n"
     ]
    }
   ],
   "source": [
    "!pip install awscli "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "107309c4",
   "metadata": {
    "papermill": {
     "duration": 0.022674,
     "end_time": "2023-07-13T20:38:39.144980",
     "exception": false,
     "start_time": "2023-07-13T20:38:39.122306",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Import the necessary components. Some of them are new for this week, they will be discussed later in the notebook. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "7c71e7d5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:38:39.191839Z",
     "iopub.status.busy": "2023-07-13T20:38:39.191466Z",
     "iopub.status.idle": "2023-07-13T20:38:52.317001Z",
     "shell.execute_reply": "2023-07-13T20:38:52.316028Z"
    },
    "papermill": {
     "duration": 13.15198,
     "end_time": "2023-07-13T20:38:52.319772",
     "exception": false,
     "start_time": "2023-07-13T20:38:39.167792",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.10/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.5\n",
      "  warnings.warn(f\"A NumPy version >={np_minversion} and <{np_maxversion}\"\n",
      "/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so']\n",
      "caused by: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so: undefined symbol: _ZN3tsl6StatusC1EN10tensorflow5error4CodeESt17basic_string_viewIcSt11char_traitsIcEENS_14SourceLocationE']\n",
      "  warnings.warn(f\"unable to load libtensorflow_io_plugins.so: {e}\")\n",
      "/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so']\n",
      "caused by: ['/opt/conda/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so: undefined symbol: _ZTVN10tensorflow13GcsFileSystemE']\n",
      "  warnings.warn(f\"file system plugins are not loaded: {e}\")\n"
     ]
    }
   ],
   "source": [
    "from datasets import load_dataset\n",
    "from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, GenerationConfig, TrainingArguments, Trainer\n",
    "import torch\n",
    "import time\n",
    "import evaluate\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62fcc3b3",
   "metadata": {
    "papermill": {
     "duration": 0.022342,
     "end_time": "2023-07-13T20:38:52.365552",
     "exception": false,
     "start_time": "2023-07-13T20:38:52.343210",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='1.2'></a>\n",
    "### 1.2 - Load Dataset and LLM\n",
    "\n",
    "You are going to continue experimenting with the [DialogSum](https://huggingface.co/datasets/knkarthick/dialogsum) Hugging Face dataset. It contains 10,000+ dialogues with the corresponding manually labeled summaries and topics. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "905e32fe",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:38:52.412179Z",
     "iopub.status.busy": "2023-07-13T20:38:52.411827Z",
     "iopub.status.idle": "2023-07-13T20:38:56.589881Z",
     "shell.execute_reply": "2023-07-13T20:38:56.589009Z"
    },
    "papermill": {
     "duration": 4.203821,
     "end_time": "2023-07-13T20:38:56.591923",
     "exception": false,
     "start_time": "2023-07-13T20:38:52.388102",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f6bf08587d954f0f915352e005b63857",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading readme:   0%|          | 0.00/4.56k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading and preparing dataset csv/knkarthick--dialogsum to /root/.cache/huggingface/datasets/knkarthick___csv/knkarthick--dialogsum-c8fac5d84cd35861/0.0.0/6954658bab30a358235fa864b05cf819af0e179325c740e4bc853bcc7ec513e1...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0c4bec61eca8487691ac5cc738a3c4a4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading data files:   0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "18d43f15899e4c8f9299dbc2f8cf5cdb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading data:   0%|          | 0.00/11.3M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9af58bf5ea264a96a05302cbc66aa4dd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading data:   0%|          | 0.00/1.35M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "712f0da6a3814a9e96467e2f88a21984",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading data:   0%|          | 0.00/442k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1639273d6f474b7cbe7323eb7c3bd153",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Extracting data files:   0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e95e6582212146d5bf2ba468dcdd0956",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1007a7ba4bef4be29216bfc3b66ecd9f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating test split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "474b38719bc34419a4d68246a7580294",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating validation split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset csv downloaded and prepared to /root/.cache/huggingface/datasets/knkarthick___csv/knkarthick--dialogsum-c8fac5d84cd35861/0.0.0/6954658bab30a358235fa864b05cf819af0e179325c740e4bc853bcc7ec513e1. Subsequent calls will reuse this data.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c9b2d12f25444bf1ba1176d3dff708f7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "DatasetDict({\n",
       "    train: Dataset({\n",
       "        features: ['id', 'dialogue', 'summary', 'topic'],\n",
       "        num_rows: 12460\n",
       "    })\n",
       "    test: Dataset({\n",
       "        features: ['id', 'dialogue', 'summary', 'topic'],\n",
       "        num_rows: 1500\n",
       "    })\n",
       "    validation: Dataset({\n",
       "        features: ['id', 'dialogue', 'summary', 'topic'],\n",
       "        num_rows: 500\n",
       "    })\n",
       "})"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "huggingface_dataset_name = \"knkarthick/dialogsum\"\n",
    "\n",
    "dataset = load_dataset(huggingface_dataset_name)\n",
    "\n",
    "dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "163c95ad",
   "metadata": {
    "papermill": {
     "duration": 0.024209,
     "end_time": "2023-07-13T20:38:56.640863",
     "exception": false,
     "start_time": "2023-07-13T20:38:56.616654",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Load the pre-trained [FLAN-T5 model](https://huggingface.co/docs/transformers/model_doc/flan-t5) and its tokenizer directly from HuggingFace. Notice that you will be using the [small version](https://huggingface.co/google/flan-t5-base) of FLAN-T5. Setting `torch_dtype=torch.bfloat16` specifies the memory type to be used by this model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f38f5f8c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:38:56.691566Z",
     "iopub.status.busy": "2023-07-13T20:38:56.690905Z",
     "iopub.status.idle": "2023-07-13T20:39:16.785144Z",
     "shell.execute_reply": "2023-07-13T20:39:16.784101Z"
    },
    "papermill": {
     "duration": 20.1224,
     "end_time": "2023-07-13T20:39:16.787594",
     "exception": false,
     "start_time": "2023-07-13T20:38:56.665194",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# model_name='google/flan-t5-base'\n",
    "model_name='/kaggle/input/flan-t5/pytorch/base/4'\n",
    "original_model = AutoModelForSeq2SeqLM.from_pretrained(model_name, torch_dtype=torch.bfloat16)\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e6b267a",
   "metadata": {
    "papermill": {
     "duration": 0.024306,
     "end_time": "2023-07-13T20:39:16.837131",
     "exception": false,
     "start_time": "2023-07-13T20:39:16.812825",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "It is possible to pull out the number of model parameters and find out how many of them are trainable. The following function can be used to do that, at this stage, you do not need to go into details of it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a0b92886",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:39:16.891580Z",
     "iopub.status.busy": "2023-07-13T20:39:16.891234Z",
     "iopub.status.idle": "2023-07-13T20:39:16.899774Z",
     "shell.execute_reply": "2023-07-13T20:39:16.898830Z"
    },
    "papermill": {
     "duration": 0.040695,
     "end_time": "2023-07-13T20:39:16.902087",
     "exception": false,
     "start_time": "2023-07-13T20:39:16.861392",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable model parameters: 247577856\n",
      "all model parameters: 247577856\n",
      "percentage of trainable model parameters: 100.00%\n"
     ]
    }
   ],
   "source": [
    "def print_number_of_trainable_model_parameters(model):\n",
    "    trainable_model_params = 0\n",
    "    all_model_params = 0\n",
    "    for _, param in model.named_parameters():\n",
    "        all_model_params += param.numel()\n",
    "        if param.requires_grad:\n",
    "            trainable_model_params += param.numel()\n",
    "    return f\"trainable model parameters: {trainable_model_params}\\nall model parameters: {all_model_params}\\npercentage of trainable model parameters: {100 * trainable_model_params / all_model_params:.2f}%\"\n",
    "\n",
    "print(print_number_of_trainable_model_parameters(original_model))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e09d425",
   "metadata": {
    "papermill": {
     "duration": 0.024584,
     "end_time": "2023-07-13T20:39:16.950971",
     "exception": false,
     "start_time": "2023-07-13T20:39:16.926387",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='1.3'></a>\n",
    "### 1.3 - Test the Model with Zero Shot Inferencing\n",
    "\n",
    "Test the model with the zero shot inferencing. You can see that the model struggles to summarize the dialogue compared to the baseline summary, but it does pull out some important information from the text which indicates the model can be fine-tuned to the task at hand."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "be93a3fb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:39:17.001362Z",
     "iopub.status.busy": "2023-07-13T20:39:17.001056Z",
     "iopub.status.idle": "2023-07-13T20:39:19.640040Z",
     "shell.execute_reply": "2023-07-13T20:39:19.638877Z"
    },
    "papermill": {
     "duration": 2.666757,
     "end_time": "2023-07-13T20:39:19.642283",
     "exception": false,
     "start_time": "2023-07-13T20:39:16.975526",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------------------------------------------------------------------------------------------\n",
      "INPUT PROMPT:\n",
      "\n",
      "Summarize the following conversation.\n",
      "\n",
      "#Person1#: Have you considered upgrading your system?\n",
      "#Person2#: Yes, but I'm not sure what exactly I would need.\n",
      "#Person1#: You could consider adding a painting program to your software. It would allow you to make up your own flyers and banners for advertising.\n",
      "#Person2#: That would be a definite bonus.\n",
      "#Person1#: You might also want to upgrade your hardware because it is pretty outdated now.\n",
      "#Person2#: How can we do that?\n",
      "#Person1#: You'd probably need a faster processor, to begin with. And you also need a more powerful hard disc, more memory and a faster modem. Do you have a CD-ROM drive?\n",
      "#Person2#: No.\n",
      "#Person1#: Then you might want to add a CD-ROM drive too, because most new software programs are coming out on Cds.\n",
      "#Person2#: That sounds great. Thanks.\n",
      "\n",
      "Summary:\n",
      "\n",
      "---------------------------------------------------------------------------------------------------\n",
      "BASELINE HUMAN SUMMARY:\n",
      "#Person1# teaches #Person2# how to upgrade software and hardware in #Person2#'s system.\n",
      "\n",
      "---------------------------------------------------------------------------------------------------\n",
      "MODEL GENERATION - ZERO SHOT:\n",
      "#Person1#: I'm thinking of upgrading my computer.\n"
     ]
    }
   ],
   "source": [
    "index = 200\n",
    "\n",
    "dialogue = dataset['test'][index]['dialogue']\n",
    "summary = dataset['test'][index]['summary']\n",
    "\n",
    "prompt = f\"\"\"\n",
    "Summarize the following conversation.\n",
    "\n",
    "{dialogue}\n",
    "\n",
    "Summary:\n",
    "\"\"\"\n",
    "\n",
    "inputs = tokenizer(prompt, return_tensors='pt')\n",
    "output = tokenizer.decode(\n",
    "    original_model.generate(\n",
    "        inputs[\"input_ids\"], \n",
    "        max_new_tokens=200,\n",
    "    )[0], \n",
    "    skip_special_tokens=True\n",
    ")\n",
    "\n",
    "dash_line = '-'.join('' for x in range(100))\n",
    "print(dash_line)\n",
    "print(f'INPUT PROMPT:\\n{prompt}')\n",
    "print(dash_line)\n",
    "print(f'BASELINE HUMAN SUMMARY:\\n{summary}\\n')\n",
    "print(dash_line)\n",
    "print(f'MODEL GENERATION - ZERO SHOT:\\n{output}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "744e9ee5",
   "metadata": {
    "papermill": {
     "duration": 0.02471,
     "end_time": "2023-07-13T20:39:19.692498",
     "exception": false,
     "start_time": "2023-07-13T20:39:19.667788",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='2'></a>\n",
    "## 2 - Perform Full Fine-Tuning"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc095135",
   "metadata": {
    "papermill": {
     "duration": 0.029615,
     "end_time": "2023-07-13T20:39:19.749846",
     "exception": false,
     "start_time": "2023-07-13T20:39:19.720231",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='2.1'></a>\n",
    "### 2.1 - Preprocess the Dialog-Summary Dataset\n",
    "\n",
    "You need to convert the dialog-summary (prompt-response) pairs into explicit instructions for the LLM. Prepend an instruction to the start of the dialog with `Summarize the following conversation` and to the start of the summary with `Summary` as follows:\n",
    "\n",
    "Training prompt (dialogue):\n",
    "```\n",
    "Summarize the following conversation.\n",
    "\n",
    "    Chris: This is his part of the conversation.\n",
    "    Antje: This is her part of the conversation.\n",
    "    \n",
    "Summary: \n",
    "```\n",
    "\n",
    "Training response (summary):\n",
    "```\n",
    "Both Chris and Antje participated in the conversation.\n",
    "```\n",
    "\n",
    "Then preprocess the prompt-response dataset into tokens and pull out their `input_ids` (1 per token)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f671f647",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:39:19.806130Z",
     "iopub.status.busy": "2023-07-13T20:39:19.805022Z",
     "iopub.status.idle": "2023-07-13T20:39:33.665173Z",
     "shell.execute_reply": "2023-07-13T20:39:33.664182Z"
    },
    "papermill": {
     "duration": 13.888998,
     "end_time": "2023-07-13T20:39:33.667462",
     "exception": false,
     "start_time": "2023-07-13T20:39:19.778464",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fd05f305a47b41e3aabae9808951ba10",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/12460 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d686e6a558de4e47b5d6233e931bea46",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/1500 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "da4334de88ce482eb652240c0d098575",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/500 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def tokenize_function(example):\n",
    "    start_prompt = 'Summarize the following conversation.\\n\\n'\n",
    "    end_prompt = '\\n\\nSummary: '\n",
    "    prompt = [start_prompt + dialogue + end_prompt for dialogue in example[\"dialogue\"]]\n",
    "    example['input_ids'] = tokenizer(prompt, padding=\"max_length\", truncation=True, return_tensors=\"pt\").input_ids\n",
    "    example['labels'] = tokenizer(example[\"summary\"], padding=\"max_length\", truncation=True, return_tensors=\"pt\").input_ids\n",
    "    \n",
    "    return example\n",
    "\n",
    "# The dataset actually contains 3 diff splits: train, validation, test.\n",
    "# The tokenize_function code is handling all data across all splits in batches.\n",
    "tokenized_datasets = dataset.map(tokenize_function, batched=True)\n",
    "tokenized_datasets = tokenized_datasets.remove_columns(['id', 'topic', 'dialogue', 'summary',])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdd01e0a",
   "metadata": {
    "papermill": {
     "duration": 0.025272,
     "end_time": "2023-07-13T20:39:33.719034",
     "exception": false,
     "start_time": "2023-07-13T20:39:33.693762",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "To save some time in the lab, you will subsample the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6d721a9d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:39:33.772138Z",
     "iopub.status.busy": "2023-07-13T20:39:33.771248Z",
     "iopub.status.idle": "2023-07-13T20:39:44.509189Z",
     "shell.execute_reply": "2023-07-13T20:39:44.507926Z"
    },
    "papermill": {
     "duration": 10.767076,
     "end_time": "2023-07-13T20:39:44.511647",
     "exception": false,
     "start_time": "2023-07-13T20:39:33.744571",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3d71081073654e38b564b37161edb380",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Filter:   0%|          | 0/12460 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f53066503ce14d35895b40b3f4d044de",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Filter:   0%|          | 0/1500 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9012ffab869f4e1ba1fbd837f6375f22",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Filter:   0%|          | 0/500 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tokenized_datasets = tokenized_datasets.filter(lambda example, index: index % 100 == 0, with_indices=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "879432cf",
   "metadata": {
    "papermill": {
     "duration": 0.02561,
     "end_time": "2023-07-13T20:39:44.563892",
     "exception": false,
     "start_time": "2023-07-13T20:39:44.538282",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Check the shapes of all three parts of the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "5b8671a5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:39:44.617035Z",
     "iopub.status.busy": "2023-07-13T20:39:44.616158Z",
     "iopub.status.idle": "2023-07-13T20:39:44.623009Z",
     "shell.execute_reply": "2023-07-13T20:39:44.621590Z"
    },
    "papermill": {
     "duration": 0.035521,
     "end_time": "2023-07-13T20:39:44.625102",
     "exception": false,
     "start_time": "2023-07-13T20:39:44.589581",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shapes of the datasets:\n",
      "Training: (125, 2)\n",
      "Validation: (5, 2)\n",
      "Test: (15, 2)\n",
      "DatasetDict({\n",
      "    train: Dataset({\n",
      "        features: ['input_ids', 'labels'],\n",
      "        num_rows: 125\n",
      "    })\n",
      "    test: Dataset({\n",
      "        features: ['input_ids', 'labels'],\n",
      "        num_rows: 15\n",
      "    })\n",
      "    validation: Dataset({\n",
      "        features: ['input_ids', 'labels'],\n",
      "        num_rows: 5\n",
      "    })\n",
      "})\n"
     ]
    }
   ],
   "source": [
    "print(f\"Shapes of the datasets:\")\n",
    "print(f\"Training: {tokenized_datasets['train'].shape}\")\n",
    "print(f\"Validation: {tokenized_datasets['validation'].shape}\")\n",
    "print(f\"Test: {tokenized_datasets['test'].shape}\")\n",
    "\n",
    "print(tokenized_datasets)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ba5ae00",
   "metadata": {
    "papermill": {
     "duration": 0.02548,
     "end_time": "2023-07-13T20:39:44.676413",
     "exception": false,
     "start_time": "2023-07-13T20:39:44.650933",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The output dataset is ready for fine-tuning."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "567d8a6b",
   "metadata": {
    "papermill": {
     "duration": 0.025262,
     "end_time": "2023-07-13T20:39:44.727349",
     "exception": false,
     "start_time": "2023-07-13T20:39:44.702087",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='2.2'></a>\n",
    "### 2.2 - Fine-Tune the Model with the Preprocessed Dataset\n",
    "\n",
    "Now utilize the built-in Hugging Face `Trainer` class (see the documentation [here](https://huggingface.co/docs/transformers/main_classes/trainer)). Pass the preprocessed dataset with reference to the original model. Other training parameters are found experimentally and there is no need to go into details about those at the moment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "89aa39de",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:39:44.779779Z",
     "iopub.status.busy": "2023-07-13T20:39:44.779420Z",
     "iopub.status.idle": "2023-07-13T20:39:46.705805Z",
     "shell.execute_reply": "2023-07-13T20:39:46.704748Z"
    },
    "papermill": {
     "duration": 1.955115,
     "end_time": "2023-07-13T20:39:46.708055",
     "exception": false,
     "start_time": "2023-07-13T20:39:44.752940",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using the `WANDB_DISABLED` environment variable is deprecated and will be removed in v5. Use the --report_to flag to control the integrations used for logging result (for instance --report_to none).\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "os.environ[\"WANDB_DISABLED\"] = \"true\"\n",
    "\n",
    "output_dir = f'./dialogue-summary-training-{str(int(time.time()))}'\n",
    "\n",
    "training_args = TrainingArguments(\n",
    "    output_dir=output_dir,\n",
    "    learning_rate=1e-5,\n",
    "    num_train_epochs=1,\n",
    "    weight_decay=0.01,\n",
    "    logging_steps=1,\n",
    "    max_steps=1,\n",
    "    report_to=None,\n",
    ")\n",
    "\n",
    "trainer = Trainer(\n",
    "    model=original_model,\n",
    "    args=training_args,\n",
    "    train_dataset=tokenized_datasets['train'],\n",
    "    eval_dataset=tokenized_datasets['validation']\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3105e90c",
   "metadata": {
    "papermill": {
     "duration": 0.025862,
     "end_time": "2023-07-13T20:39:46.760396",
     "exception": false,
     "start_time": "2023-07-13T20:39:46.734534",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Start training process...\n",
    "\n",
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSIxMjUiIHZpZXdCb3g9IjAgMCA4MDAgMTI1IiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgogICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJmYWRlR3JhZGllbnQiIHgxPSIwIiB4Mj0iMSI+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNGMEYwRjAiLz4KICAgICAgICAgICAgPHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIiBzdG9wLW9wYWNpdHk9IjAiLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICAgIDxtYXNrIGlkPSJmYWRlTWFzayI+CiAgICAgICAgICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSI3NTAiIGhlaWdodD0iMTI1IiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSIxMjUiIGZpbGw9InVybCgjZmFkZUdyYWRpZW50KSIvPgogICAgICAgIDwvbWFzaz4KICAgIDwvZGVmcz4KICAgIDxwYXRoIGQ9Ik0zLDUwIEE1MCw1MCAwIDAgMSA1MywzIEw3OTcsMyBMNzk3LDk3IEw5Nyw5NyBMNTAsMTE1IEwzLDk3IFoiIGZpbGw9IiNGMEYwRjAiIHN0cm9rZT0iI0UwRTBFMCIgc3Ryb2tlLXdpZHRoPSIxIiBtYXNrPSJ1cmwoI2ZhZGVNYXNrKSIvPgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIGZpbGw9IiM1N2M0ZjgiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIxIi8+CiAgICA8Y2lyY2xlIGN4PSI1MCIgY3k9IjUwIiByPSIyNSIgZmlsbD0iI0YwRjBGMCIvPgogICAgPGxpbmUgeDE9IjUwIiB5MT0iNTAiIHgyPSI1MCIgeTI9IjMwIiBzdHJva2U9IiM1N2M0ZjgiIHN0cm9rZS13aWR0aD0iMyIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIi8+CiAgICA8bGluZSB4MT0iNTAiIHkxPSI1MCIgeDI9IjY1IiB5Mj0iNTAiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIzIiBzdHJva2UtbGluZWNhcD0icm91bmQiLz4KICAgIDx0ZXh0IHg9IjEwMCIgeT0iMzQiIGZvbnQtZmFtaWx5PSJBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxNCIgZmlsbD0iIzMzMzMzMyI+VGhlIG5leHQgY2VsbCBtYXkgdGFrZSBhIGZldyBtaW51dGVzIHRvIHJ1bi4gUGxlYXNlIGJlIHBhdGllbnQuPC90ZXh0PgogICAgPHRleHQgeD0iMTAwIiB5PSI1NiIgZm9udC1mYW1pbHk9IkFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjE0IiBmaWxsPSIjMzMzMzMzIj5Zb3UgY2FuIHNhZmVseSBpZ25vcmUgdGhlIHdhcm5pbmcgbWVzc2FnZXMuPC90ZXh0Pgo8L3N2Zz4K\" alt=\"Time alert open medium\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "62963ae6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:39:46.815278Z",
     "iopub.status.busy": "2023-07-13T20:39:46.814389Z",
     "iopub.status.idle": "2023-07-13T20:39:50.515531Z",
     "shell.execute_reply": "2023-07-13T20:39:50.514605Z"
    },
    "papermill": {
     "duration": 3.730678,
     "end_time": "2023-07-13T20:39:50.517772",
     "exception": false,
     "start_time": "2023-07-13T20:39:46.787094",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.10/site-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1' max='1' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1/1 00:00, Epoch 0/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>48.500000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "TrainOutput(global_step=1, training_loss=48.5, metrics={'train_runtime': 3.6732, 'train_samples_per_second': 2.178, 'train_steps_per_second': 0.272, 'total_flos': 5478058819584.0, 'train_loss': 48.5, 'epoch': 0.06})"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.train()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72f1c1b9",
   "metadata": {
    "papermill": {
     "duration": 0.025917,
     "end_time": "2023-07-13T20:39:50.570598",
     "exception": false,
     "start_time": "2023-07-13T20:39:50.544681",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSI1MCIgdmlld0JveD0iMCAwIDgwMCA1MCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KICAgIDxkZWZzPgogICAgICAgIDxsaW5lYXJHcmFkaWVudCBpZD0iZmFkZUdyYWRpZW50IiB4MT0iMCIgeDI9IjEiPgogICAgICAgICAgICA8c3RvcCBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIi8+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iI0YwRjBGMCIgc3RvcC1vcGFjaXR5PSIwIi8+CiAgICAgICAgPC9saW5lYXJHcmFkaWVudD4KICAgICAgICA8bWFzayBpZD0iZmFkZU1hc2siPgogICAgICAgICAgICA8cmVjdCB4PSIwIiB5PSIwIiB3aWR0aD0iNzUwIiBoZWlnaHQ9IjUwIiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSI1MCIgZmlsbD0idXJsKCNmYWRlR3JhZGllbnQpIi8+CiAgICAgICAgPC9tYXNrPgogICAgPC9kZWZzPgogICAgPHBhdGggZD0iTTI1LDUwIFEwLDUwIDAsMjUgTDUwLDMgTDk3LDI1IEw3OTcsMjUgTDc5Nyw1MCBMMjUsNTAgWiIgZmlsbD0iI0YwRjBGMCIgc3Ryb2tlPSIjRTBFMEUwIiBzdHJva2Utd2lkdGg9IjEiIG1hc2s9InVybCgjZmFkZU1hc2spIi8+Cjwvc3ZnPgo=\" alt=\"Time alert close\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0df40cf",
   "metadata": {
    "papermill": {
     "duration": 0.025669,
     "end_time": "2023-07-13T20:39:50.622456",
     "exception": false,
     "start_time": "2023-07-13T20:39:50.596787",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Training a fully fine-tuned version of the model would take a few hours on a GPU. To save time, download a checkpoint of the fully fine-tuned model to use in the rest of this notebook. This fully fine-tuned model will also be referred to as the **instruct model** in this lab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "6754caf5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:39:50.676644Z",
     "iopub.status.busy": "2023-07-13T20:39:50.675544Z",
     "iopub.status.idle": "2023-07-13T20:39:50.680195Z",
     "shell.execute_reply": "2023-07-13T20:39:50.679534Z"
    },
    "papermill": {
     "duration": 0.033907,
     "end_time": "2023-07-13T20:39:50.682204",
     "exception": false,
     "start_time": "2023-07-13T20:39:50.648297",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# !aws s3 cp --recursive s3://dlai-generative-ai/models/flan-dialogue-summary-checkpoint/ ./flan-dialogue-summary-checkpoint/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f204ba6b",
   "metadata": {
    "papermill": {
     "duration": 0.02557,
     "end_time": "2023-07-13T20:39:50.733503",
     "exception": false,
     "start_time": "2023-07-13T20:39:50.707933",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The size of the downloaded instruct model is approximately 1GB."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "0bf2e15b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:39:50.787460Z",
     "iopub.status.busy": "2023-07-13T20:39:50.786565Z",
     "iopub.status.idle": "2023-07-13T20:39:50.791347Z",
     "shell.execute_reply": "2023-07-13T20:39:50.790484Z"
    },
    "papermill": {
     "duration": 0.033823,
     "end_time": "2023-07-13T20:39:50.793281",
     "exception": false,
     "start_time": "2023-07-13T20:39:50.759458",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# !ls -alh /kaggle/working/flan-dialogue-summary-checkpoint/pytorch_model.bin"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4263e6b",
   "metadata": {
    "papermill": {
     "duration": 0.025732,
     "end_time": "2023-07-13T20:39:50.844831",
     "exception": false,
     "start_time": "2023-07-13T20:39:50.819099",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Create an instance of the `AutoModelForSeq2SeqLM` class for the instruct model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "69da30ab",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:39:50.898368Z",
     "iopub.status.busy": "2023-07-13T20:39:50.897568Z",
     "iopub.status.idle": "2023-07-13T20:39:51.750686Z",
     "shell.execute_reply": "2023-07-13T20:39:51.749645Z"
    },
    "papermill": {
     "duration": 0.882508,
     "end_time": "2023-07-13T20:39:51.753402",
     "exception": false,
     "start_time": "2023-07-13T20:39:50.870894",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "original_model = original_model.to('cpu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "a00116ab",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:39:51.807820Z",
     "iopub.status.busy": "2023-07-13T20:39:51.807463Z",
     "iopub.status.idle": "2023-07-13T20:40:03.778466Z",
     "shell.execute_reply": "2023-07-13T20:40:03.777415Z"
    },
    "papermill": {
     "duration": 12.000943,
     "end_time": "2023-07-13T20:40:03.781130",
     "exception": false,
     "start_time": "2023-07-13T20:39:51.780187",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "instruct_model = AutoModelForSeq2SeqLM.from_pretrained(\"/kaggle/input/generative-ai-with-llms-lab-2/lab_2/flan-dialogue-summary-checkpoint/\", torch_dtype=torch.bfloat16).to('cpu')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38e918e2",
   "metadata": {
    "papermill": {
     "duration": 0.026301,
     "end_time": "2023-07-13T20:40:03.834524",
     "exception": false,
     "start_time": "2023-07-13T20:40:03.808223",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='2.3'></a>\n",
    "### 2.3 - Evaluate the Model Qualitatively (Human Evaluation)\n",
    "\n",
    "As with many GenAI applications, a qualitative approach where you ask yourself the question \"Is my model behaving the way it is supposed to?\" is usually a good starting point. In the example below (the same one we started this notebook with), you can see how the fine-tuned model is able to create a reasonable summary of the dialogue compared to the original inability to understand what is being asked of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "141484e6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:40:03.889616Z",
     "iopub.status.busy": "2023-07-13T20:40:03.888530Z",
     "iopub.status.idle": "2023-07-13T20:40:15.503742Z",
     "shell.execute_reply": "2023-07-13T20:40:15.501513Z"
    },
    "papermill": {
     "duration": 11.644697,
     "end_time": "2023-07-13T20:40:15.505989",
     "exception": false,
     "start_time": "2023-07-13T20:40:03.861292",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------------------------------------------------------------------------------------------\n",
      "BASELINE HUMAN SUMMARY:\n",
      "#Person1# teaches #Person2# how to upgrade software and hardware in #Person2#'s system.\n",
      "---------------------------------------------------------------------------------------------------\n",
      "ORIGINAL MODEL:\n",
      "#Person1: I'm thinking of upgrading my system. #Person2: I'm not sure what exactly I would need. #Person1: You could consider adding a painting program to your software. #Person2: That would be a definite bonus. #Person2: You also need a faster processor, more memory and a faster modem. #Person1: You could also add a CD-ROM drive.\n",
      "---------------------------------------------------------------------------------------------------\n",
      "INSTRUCT MODEL:\n",
      "#Person1# suggests #Person2# upgrading #Person2#'s system, hardware, and CD-ROM drive. #Person2# thinks it's great.\n"
     ]
    }
   ],
   "source": [
    "index = 200\n",
    "dialogue = dataset['test'][index]['dialogue']\n",
    "human_baseline_summary = dataset['test'][index]['summary']\n",
    "\n",
    "prompt = f\"\"\"\n",
    "Summarize the following conversation.\n",
    "\n",
    "{dialogue}\n",
    "\n",
    "Summary:\n",
    "\"\"\"\n",
    "\n",
    "input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids\n",
    "\n",
    "original_model_outputs = original_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200, num_beams=1))\n",
    "original_model_text_output = tokenizer.decode(original_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "instruct_model_outputs = instruct_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200, num_beams=1))\n",
    "instruct_model_text_output = tokenizer.decode(instruct_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "print(dash_line)\n",
    "print(f'BASELINE HUMAN SUMMARY:\\n{human_baseline_summary}')\n",
    "print(dash_line)\n",
    "print(f'ORIGINAL MODEL:\\n{original_model_text_output}')\n",
    "print(dash_line)\n",
    "print(f'INSTRUCT MODEL:\\n{instruct_model_text_output}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1198e74",
   "metadata": {
    "papermill": {
     "duration": 0.026141,
     "end_time": "2023-07-13T20:40:15.558733",
     "exception": false,
     "start_time": "2023-07-13T20:40:15.532592",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='2.4'></a>\n",
    "### 2.4 - Evaluate the Model Quantitatively (with ROUGE Metric)\n",
    "\n",
    "The [ROUGE metric](https://en.wikipedia.org/wiki/ROUGE_(metric)) helps quantify the validity of summarizations produced by models. It compares summarizations to a \"baseline\" summary which is usually created by a human. While not perfect, it does indicate the overall increase in summarization effectiveness that we have accomplished by fine-tuning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "09af8c77",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:40:15.612355Z",
     "iopub.status.busy": "2023-07-13T20:40:15.612035Z",
     "iopub.status.idle": "2023-07-13T20:40:17.021490Z",
     "shell.execute_reply": "2023-07-13T20:40:17.020470Z"
    },
    "papermill": {
     "duration": 1.438922,
     "end_time": "2023-07-13T20:40:17.023929",
     "exception": false,
     "start_time": "2023-07-13T20:40:15.585007",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e157670e79774d35ad6d5b088b63083f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading builder script:   0%|          | 0.00/6.27k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "rouge = evaluate.load('rouge')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e740d985",
   "metadata": {
    "papermill": {
     "duration": 0.026712,
     "end_time": "2023-07-13T20:40:17.077783",
     "exception": false,
     "start_time": "2023-07-13T20:40:17.051071",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Generate the outputs for the sample of the test dataset (only 10 dialogues and summaries to save time), and save the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "bb95393f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:40:17.132709Z",
     "iopub.status.busy": "2023-07-13T20:40:17.131578Z",
     "iopub.status.idle": "2023-07-13T20:42:01.705716Z",
     "shell.execute_reply": "2023-07-13T20:42:01.704733Z"
    },
    "papermill": {
     "duration": 104.633157,
     "end_time": "2023-07-13T20:42:01.737144",
     "exception": false,
     "start_time": "2023-07-13T20:40:17.103987",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>human_baseline_summaries</th>\n",
       "      <th>original_model_summaries</th>\n",
       "      <th>instruct_model_summaries</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Ms. Dawson helps #Person1# to write a memo to ...</td>\n",
       "      <td>This memo is for the following:</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>In order to prevent employees from wasting tim...</td>\n",
       "      <td>Ms. Dawson, I need to take a moment to explain...</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ms. Dawson takes a dictation for #Person1# abo...</td>\n",
       "      <td>The following memo should be distributed to al...</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>#Person2# arrives late because of traffic jam....</td>\n",
       "      <td>The traffic in the city is congested.</td>\n",
       "      <td>#Person2# got stuck in traffic again. #Person1...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>#Person2# decides to follow #Person1#'s sugges...</td>\n",
       "      <td>The traffic is congested and the traffic is co...</td>\n",
       "      <td>#Person2# got stuck in traffic again. #Person1...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>#Person2# complains to #Person1# about the tra...</td>\n",
       "      <td>Taking public transport to work is a good idea...</td>\n",
       "      <td>#Person2# got stuck in traffic again. #Person1...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>#Person1# tells Kate that Masha and Hero get d...</td>\n",
       "      <td>#Person1#: Masha and Hero are getting divorced...</td>\n",
       "      <td>Masha and Hero are getting divorced. Kate can'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>#Person1# tells Kate that Masha and Hero are g...</td>\n",
       "      <td>Masha and Hero are getting divorced.</td>\n",
       "      <td>Masha and Hero are getting divorced. Kate can'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>#Person1# and Kate talk about the divorce betw...</td>\n",
       "      <td>#Person1#: Masha and Hero are getting divorced.</td>\n",
       "      <td>Masha and Hero are getting divorced. Kate can'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>#Person1# and Brian are at the birthday party ...</td>\n",
       "      <td>Brian's birthday is today.</td>\n",
       "      <td>Brian's birthday is coming. #Person1# invites ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                            human_baseline_summaries  \\\n",
       "0  Ms. Dawson helps #Person1# to write a memo to ...   \n",
       "1  In order to prevent employees from wasting tim...   \n",
       "2  Ms. Dawson takes a dictation for #Person1# abo...   \n",
       "3  #Person2# arrives late because of traffic jam....   \n",
       "4  #Person2# decides to follow #Person1#'s sugges...   \n",
       "5  #Person2# complains to #Person1# about the tra...   \n",
       "6  #Person1# tells Kate that Masha and Hero get d...   \n",
       "7  #Person1# tells Kate that Masha and Hero are g...   \n",
       "8  #Person1# and Kate talk about the divorce betw...   \n",
       "9  #Person1# and Brian are at the birthday party ...   \n",
       "\n",
       "                            original_model_summaries  \\\n",
       "0                    This memo is for the following:   \n",
       "1  Ms. Dawson, I need to take a moment to explain...   \n",
       "2  The following memo should be distributed to al...   \n",
       "3              The traffic in the city is congested.   \n",
       "4  The traffic is congested and the traffic is co...   \n",
       "5  Taking public transport to work is a good idea...   \n",
       "6  #Person1#: Masha and Hero are getting divorced...   \n",
       "7               Masha and Hero are getting divorced.   \n",
       "8    #Person1#: Masha and Hero are getting divorced.   \n",
       "9                         Brian's birthday is today.   \n",
       "\n",
       "                            instruct_model_summaries  \n",
       "0  #Person1# asks Ms. Dawson to take a dictation ...  \n",
       "1  #Person1# asks Ms. Dawson to take a dictation ...  \n",
       "2  #Person1# asks Ms. Dawson to take a dictation ...  \n",
       "3  #Person2# got stuck in traffic again. #Person1...  \n",
       "4  #Person2# got stuck in traffic again. #Person1...  \n",
       "5  #Person2# got stuck in traffic again. #Person1...  \n",
       "6  Masha and Hero are getting divorced. Kate can'...  \n",
       "7  Masha and Hero are getting divorced. Kate can'...  \n",
       "8  Masha and Hero are getting divorced. Kate can'...  \n",
       "9  Brian's birthday is coming. #Person1# invites ...  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dialogues = dataset['test'][0:10]['dialogue']\n",
    "human_baseline_summaries = dataset['test'][0:10]['summary']\n",
    "\n",
    "original_model_summaries = []\n",
    "instruct_model_summaries = []\n",
    "\n",
    "for _, dialogue in enumerate(dialogues):\n",
    "    prompt = f\"\"\"\n",
    "Summarize the following conversation.\n",
    "\n",
    "{dialogue}\n",
    "\n",
    "Summary: \"\"\"\n",
    "    input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids\n",
    "\n",
    "    original_model_outputs = original_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200))\n",
    "    original_model_text_output = tokenizer.decode(original_model_outputs[0], skip_special_tokens=True)\n",
    "    original_model_summaries.append(original_model_text_output)\n",
    "\n",
    "    instruct_model_outputs = instruct_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200))\n",
    "    instruct_model_text_output = tokenizer.decode(instruct_model_outputs[0], skip_special_tokens=True)\n",
    "    instruct_model_summaries.append(instruct_model_text_output)\n",
    "    \n",
    "zipped_summaries = list(zip(human_baseline_summaries, original_model_summaries, instruct_model_summaries))\n",
    " \n",
    "df = pd.DataFrame(zipped_summaries, columns = ['human_baseline_summaries', 'original_model_summaries', 'instruct_model_summaries'])\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccc56042",
   "metadata": {
    "papermill": {
     "duration": 0.028913,
     "end_time": "2023-07-13T20:42:01.793035",
     "exception": false,
     "start_time": "2023-07-13T20:42:01.764122",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Evaluate the models computing ROUGE metrics. Notice the improvement in the results!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "66277258",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:01.857205Z",
     "iopub.status.busy": "2023-07-13T20:42:01.856044Z",
     "iopub.status.idle": "2023-07-13T20:42:02.343794Z",
     "shell.execute_reply": "2023-07-13T20:42:02.342489Z"
    },
    "papermill": {
     "duration": 0.522966,
     "end_time": "2023-07-13T20:42:02.346624",
     "exception": false,
     "start_time": "2023-07-13T20:42:01.823658",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ORIGINAL MODEL:\n",
      "{'rouge1': 0.22787874538249353, 'rouge2': 0.06857281083696179, 'rougeL': 0.19409511929795903, 'rougeLsum': 0.19970336691167453}\n",
      "INSTRUCT MODEL:\n",
      "{'rouge1': 0.41026607717457186, 'rouge2': 0.17840645241958838, 'rougeL': 0.2977022096267017, 'rougeLsum': 0.2987374187518165}\n"
     ]
    }
   ],
   "source": [
    "original_model_results = rouge.compute(\n",
    "    predictions=original_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(original_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "instruct_model_results = rouge.compute(\n",
    "    predictions=instruct_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(instruct_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "print('ORIGINAL MODEL:')\n",
    "print(original_model_results)\n",
    "print('INSTRUCT MODEL:')\n",
    "print(instruct_model_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7f68d6c",
   "metadata": {
    "papermill": {
     "duration": 0.026527,
     "end_time": "2023-07-13T20:42:02.401501",
     "exception": false,
     "start_time": "2023-07-13T20:42:02.374974",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The file `data/dialogue-summary-training-results.csv` contains a pre-populated list of all model results which you can use to evaluate on a larger section of data. Let's do that for each of the models:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "75799a9c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:02.457603Z",
     "iopub.status.busy": "2023-07-13T20:42:02.456601Z",
     "iopub.status.idle": "2023-07-13T20:42:12.085372Z",
     "shell.execute_reply": "2023-07-13T20:42:12.083763Z"
    },
    "papermill": {
     "duration": 9.659188,
     "end_time": "2023-07-13T20:42:12.087550",
     "exception": false,
     "start_time": "2023-07-13T20:42:02.428362",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ORIGINAL MODEL:\n",
      "{'rouge1': 0.2334158581572823, 'rouge2': 0.07603964187010573, 'rougeL': 0.20145520923859048, 'rougeLsum': 0.20145899339006135}\n",
      "INSTRUCT MODEL:\n",
      "{'rouge1': 0.42161291557556113, 'rouge2': 0.18035380596301792, 'rougeL': 0.3384439349963909, 'rougeLsum': 0.33835653595561666}\n"
     ]
    }
   ],
   "source": [
    "# results = pd.read_csv(\"data/dialogue-summary-training-results.csv\")\n",
    "results = pd.read_csv(\"/kaggle/input/generative-ai-with-llms-lab-2/lab_2/data/dialogue-summary-training-results.csv\")\n",
    "\n",
    "human_baseline_summaries = results['human_baseline_summaries'].values\n",
    "original_model_summaries = results['original_model_summaries'].values\n",
    "instruct_model_summaries = results['instruct_model_summaries'].values\n",
    "\n",
    "original_model_results = rouge.compute(\n",
    "    predictions=original_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(original_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "instruct_model_results = rouge.compute(\n",
    "    predictions=instruct_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(instruct_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "print('ORIGINAL MODEL:')\n",
    "print(original_model_results)\n",
    "print('INSTRUCT MODEL:')\n",
    "print(instruct_model_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10a8b2ae",
   "metadata": {
    "papermill": {
     "duration": 0.02724,
     "end_time": "2023-07-13T20:42:12.142545",
     "exception": false,
     "start_time": "2023-07-13T20:42:12.115305",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The results show substantial improvement in all ROUGE metrics:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "aafbaeee",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:12.198731Z",
     "iopub.status.busy": "2023-07-13T20:42:12.198350Z",
     "iopub.status.idle": "2023-07-13T20:42:12.204806Z",
     "shell.execute_reply": "2023-07-13T20:42:12.203884Z"
    },
    "papermill": {
     "duration": 0.037007,
     "end_time": "2023-07-13T20:42:12.207083",
     "exception": false,
     "start_time": "2023-07-13T20:42:12.170076",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Absolute percentage improvement of INSTRUCT MODEL over HUMAN BASELINE\n",
      "rouge1: 18.82%\n",
      "rouge2: 10.43%\n",
      "rougeL: 13.70%\n",
      "rougeLsum: 13.69%\n"
     ]
    }
   ],
   "source": [
    "print(\"Absolute percentage improvement of INSTRUCT MODEL over HUMAN BASELINE\")\n",
    "\n",
    "improvement = (np.array(list(instruct_model_results.values())) - np.array(list(original_model_results.values())))\n",
    "for key, value in zip(instruct_model_results.keys(), improvement):\n",
    "    print(f'{key}: {value*100:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "193c896c",
   "metadata": {
    "papermill": {
     "duration": 0.027288,
     "end_time": "2023-07-13T20:42:12.261613",
     "exception": false,
     "start_time": "2023-07-13T20:42:12.234325",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='3'></a>\n",
    "## 3 - Perform Parameter Efficient Fine-Tuning (PEFT)\n",
    "\n",
    "Now, let's perform **Parameter Efficient Fine-Tuning (PEFT)** fine-tuning as opposed to \"full fine-tuning\" as you did above. PEFT is a form of instruction fine-tuning that is much more efficient than full fine-tuning - with comparable evaluation results as you will see soon. \n",
    "\n",
    "PEFT is a generic term that includes **Low-Rank Adaptation (LoRA)** and prompt tuning (which is NOT THE SAME as prompt engineering!). In most cases, when someone says PEFT, they typically mean LoRA. LoRA, at a very high level, allows the user to fine-tune their model using fewer compute resources (in some cases, a single GPU). After fine-tuning for a specific task, use case, or tenant with LoRA, the result is that the original LLM remains unchanged and a newly-trained “LoRA adapter” emerges. This LoRA adapter is much, much smaller than the original LLM - on the order of a single-digit % of the original LLM size (MBs vs GBs).  \n",
    "\n",
    "That said, at inference time, the LoRA adapter needs to be reunited and combined with its original LLM to serve the inference request.  The benefit, however, is that many LoRA adapters can re-use the original LLM which reduces overall memory requirements when serving multiple tasks and use cases."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e167c44",
   "metadata": {
    "papermill": {
     "duration": 0.0267,
     "end_time": "2023-07-13T20:42:12.315358",
     "exception": false,
     "start_time": "2023-07-13T20:42:12.288658",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='3.1'></a>\n",
    "### 3.1 - Setup the PEFT/LoRA model for Fine-Tuning\n",
    "\n",
    "You need to set up the PEFT/LoRA model for fine-tuning with a new layer/parameter adapter. Using PEFT/LoRA, you are freezing the underlying LLM and only training the adapter. Have a look at the LoRA configuration below. Note the rank (`r`) hyper-parameter, which defines the rank/dimension of the adapter to be trained."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "5f9d2db0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:12.371550Z",
     "iopub.status.busy": "2023-07-13T20:42:12.371196Z",
     "iopub.status.idle": "2023-07-13T20:42:12.391450Z",
     "shell.execute_reply": "2023-07-13T20:42:12.390576Z"
    },
    "papermill": {
     "duration": 0.050579,
     "end_time": "2023-07-13T20:42:12.393676",
     "exception": false,
     "start_time": "2023-07-13T20:42:12.343097",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from peft import LoraConfig, get_peft_model, TaskType\n",
    "\n",
    "lora_config = LoraConfig(\n",
    "    r=32, # Rank\n",
    "    lora_alpha=32,\n",
    "    target_modules=[\"q\", \"v\"],\n",
    "    lora_dropout=0.05,\n",
    "    bias=\"none\",\n",
    "    task_type=TaskType.SEQ_2_SEQ_LM # FLAN-T5\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01381b0e",
   "metadata": {
    "papermill": {
     "duration": 0.026697,
     "end_time": "2023-07-13T20:42:12.447830",
     "exception": false,
     "start_time": "2023-07-13T20:42:12.421133",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Add LoRA adapter layers/parameters to the original LLM to be trained."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "4937f1f9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:12.504281Z",
     "iopub.status.busy": "2023-07-13T20:42:12.503279Z",
     "iopub.status.idle": "2023-07-13T20:42:13.215615Z",
     "shell.execute_reply": "2023-07-13T20:42:13.214375Z"
    },
    "papermill": {
     "duration": 0.743203,
     "end_time": "2023-07-13T20:42:13.218042",
     "exception": false,
     "start_time": "2023-07-13T20:42:12.474839",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable model parameters: 3538944\n",
      "all model parameters: 251116800\n",
      "percentage of trainable model parameters: 1.41%\n"
     ]
    }
   ],
   "source": [
    "peft_model = get_peft_model(original_model, \n",
    "                            lora_config)\n",
    "print(print_number_of_trainable_model_parameters(peft_model))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0a39955",
   "metadata": {
    "papermill": {
     "duration": 0.026855,
     "end_time": "2023-07-13T20:42:13.272605",
     "exception": false,
     "start_time": "2023-07-13T20:42:13.245750",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='3.2'></a>\n",
    "### 3.2 - Train PEFT Adapter\n",
    "\n",
    "Define training arguments and create `Trainer` instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "e32c3c04",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:13.329420Z",
     "iopub.status.busy": "2023-07-13T20:42:13.328468Z",
     "iopub.status.idle": "2023-07-13T20:42:13.690955Z",
     "shell.execute_reply": "2023-07-13T20:42:13.689949Z"
    },
    "papermill": {
     "duration": 0.393413,
     "end_time": "2023-07-13T20:42:13.693445",
     "exception": false,
     "start_time": "2023-07-13T20:42:13.300032",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using the `WANDB_DISABLED` environment variable is deprecated and will be removed in v5. Use the --report_to flag to control the integrations used for logging result (for instance --report_to none).\n"
     ]
    }
   ],
   "source": [
    "output_dir = f'/kaggle/working/peft-dialogue-summary-training-{str(int(time.time()))}'\n",
    "\n",
    "peft_training_args = TrainingArguments(\n",
    "    output_dir=output_dir,\n",
    "    auto_find_batch_size=True,\n",
    "    learning_rate=1e-3, # Higher learning rate than full fine-tuning.\n",
    "    num_train_epochs=1,\n",
    "    logging_steps=1,\n",
    "    max_steps=1    \n",
    ")\n",
    "    \n",
    "peft_trainer = Trainer(\n",
    "    model=peft_model,\n",
    "    args=peft_training_args,\n",
    "    train_dataset=tokenized_datasets[\"train\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5103c7a",
   "metadata": {
    "papermill": {
     "duration": 0.026964,
     "end_time": "2023-07-13T20:42:13.748426",
     "exception": false,
     "start_time": "2023-07-13T20:42:13.721462",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now everything is ready to train the PEFT adapter and save the model.\n",
    "\n",
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSIxMjUiIHZpZXdCb3g9IjAgMCA4MDAgMTI1IiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgogICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJmYWRlR3JhZGllbnQiIHgxPSIwIiB4Mj0iMSI+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNGMEYwRjAiLz4KICAgICAgICAgICAgPHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIiBzdG9wLW9wYWNpdHk9IjAiLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICAgIDxtYXNrIGlkPSJmYWRlTWFzayI+CiAgICAgICAgICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSI3NTAiIGhlaWdodD0iMTI1IiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSIxMjUiIGZpbGw9InVybCgjZmFkZUdyYWRpZW50KSIvPgogICAgICAgIDwvbWFzaz4KICAgIDwvZGVmcz4KICAgIDxwYXRoIGQ9Ik0zLDUwIEE1MCw1MCAwIDAgMSA1MywzIEw3OTcsMyBMNzk3LDk3IEw5Nyw5NyBMNTAsMTE1IEwzLDk3IFoiIGZpbGw9IiNGMEYwRjAiIHN0cm9rZT0iI0UwRTBFMCIgc3Ryb2tlLXdpZHRoPSIxIiBtYXNrPSJ1cmwoI2ZhZGVNYXNrKSIvPgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIGZpbGw9IiM1N2M0ZjgiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIxIi8+CiAgICA8Y2lyY2xlIGN4PSI1MCIgY3k9IjUwIiByPSIyNSIgZmlsbD0iI0YwRjBGMCIvPgogICAgPGxpbmUgeDE9IjUwIiB5MT0iNTAiIHgyPSI1MCIgeTI9IjMwIiBzdHJva2U9IiM1N2M0ZjgiIHN0cm9rZS13aWR0aD0iMyIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIi8+CiAgICA8bGluZSB4MT0iNTAiIHkxPSI1MCIgeDI9IjY1IiB5Mj0iNTAiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIzIiBzdHJva2UtbGluZWNhcD0icm91bmQiLz4KICAgIDx0ZXh0IHg9IjEwMCIgeT0iMzQiIGZvbnQtZmFtaWx5PSJBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxNCIgZmlsbD0iIzMzMzMzMyI+VGhlIG5leHQgY2VsbCBtYXkgdGFrZSBhIGZldyBtaW51dGVzIHRvIHJ1bi48L3RleHQ+Cjwvc3ZnPgo=\" alt=\"Time alert open medium\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "431bbbd0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:13.807016Z",
     "iopub.status.busy": "2023-07-13T20:42:13.806673Z",
     "iopub.status.idle": "2023-07-13T20:42:16.012347Z",
     "shell.execute_reply": "2023-07-13T20:42:16.011427Z"
    },
    "papermill": {
     "duration": 2.239239,
     "end_time": "2023-07-13T20:42:16.014920",
     "exception": false,
     "start_time": "2023-07-13T20:42:13.775681",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.10/site-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='1' max='1' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [1/1 00:00, Epoch 0/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>48.750000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "('/kaggle/working/peft-dialogue-summary-checkpoint-local/tokenizer_config.json',\n",
       " '/kaggle/working/peft-dialogue-summary-checkpoint-local/special_tokens_map.json',\n",
       " '/kaggle/working/peft-dialogue-summary-checkpoint-local/spiece.model',\n",
       " '/kaggle/working/peft-dialogue-summary-checkpoint-local/added_tokens.json',\n",
       " '/kaggle/working/peft-dialogue-summary-checkpoint-local/tokenizer.json')"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "peft_trainer.train()\n",
    "\n",
    "peft_model_path=\"/kaggle/working/peft-dialogue-summary-checkpoint-local\"\n",
    "\n",
    "peft_trainer.model.save_pretrained(peft_model_path)\n",
    "tokenizer.save_pretrained(peft_model_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77a1f3a0",
   "metadata": {
    "papermill": {
     "duration": 0.027471,
     "end_time": "2023-07-13T20:42:16.070761",
     "exception": false,
     "start_time": "2023-07-13T20:42:16.043290",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSI1MCIgdmlld0JveD0iMCAwIDgwMCA1MCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KICAgIDxkZWZzPgogICAgICAgIDxsaW5lYXJHcmFkaWVudCBpZD0iZmFkZUdyYWRpZW50IiB4MT0iMCIgeDI9IjEiPgogICAgICAgICAgICA8c3RvcCBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIi8+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iI0YwRjBGMCIgc3RvcC1vcGFjaXR5PSIwIi8+CiAgICAgICAgPC9saW5lYXJHcmFkaWVudD4KICAgICAgICA8bWFzayBpZD0iZmFkZU1hc2siPgogICAgICAgICAgICA8cmVjdCB4PSIwIiB5PSIwIiB3aWR0aD0iNzUwIiBoZWlnaHQ9IjUwIiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSI1MCIgZmlsbD0idXJsKCNmYWRlR3JhZGllbnQpIi8+CiAgICAgICAgPC9tYXNrPgogICAgPC9kZWZzPgogICAgPHBhdGggZD0iTTI1LDUwIFEwLDUwIDAsMjUgTDUwLDMgTDk3LDI1IEw3OTcsMjUgTDc5Nyw1MCBMMjUsNTAgWiIgZmlsbD0iI0YwRjBGMCIgc3Ryb2tlPSIjRTBFMEUwIiBzdHJva2Utd2lkdGg9IjEiIG1hc2s9InVybCgjZmFkZU1hc2spIi8+Cjwvc3ZnPgo=\" alt=\"Time alert close\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16ac5971",
   "metadata": {
    "papermill": {
     "duration": 0.027389,
     "end_time": "2023-07-13T20:42:16.125603",
     "exception": false,
     "start_time": "2023-07-13T20:42:16.098214",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "That training was performed on a subset of data. To load a fully trained PEFT model, read a checkpoint of a PEFT model from S3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "d58810ae",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:16.183153Z",
     "iopub.status.busy": "2023-07-13T20:42:16.182588Z",
     "iopub.status.idle": "2023-07-13T20:42:16.187377Z",
     "shell.execute_reply": "2023-07-13T20:42:16.186323Z"
    },
    "papermill": {
     "duration": 0.03654,
     "end_time": "2023-07-13T20:42:16.189781",
     "exception": false,
     "start_time": "2023-07-13T20:42:16.153241",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# !aws s3 cp --recursive s3://dlai-generative-ai/models/peft-dialogue-summary-checkpoint/ /kaggle/working/peft-dialogue-summary-checkpoint-from-s3/ "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "272947b3",
   "metadata": {
    "papermill": {
     "duration": 0.029486,
     "end_time": "2023-07-13T20:42:16.248088",
     "exception": false,
     "start_time": "2023-07-13T20:42:16.218602",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Check that the size of this model is much less than the original LLM:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "6b6d8189",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:16.309881Z",
     "iopub.status.busy": "2023-07-13T20:42:16.309489Z",
     "iopub.status.idle": "2023-07-13T20:42:16.313652Z",
     "shell.execute_reply": "2023-07-13T20:42:16.312756Z"
    },
    "papermill": {
     "duration": 0.037268,
     "end_time": "2023-07-13T20:42:16.315750",
     "exception": false,
     "start_time": "2023-07-13T20:42:16.278482",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# !ls -al /kaggle/working/peft-dialogue-summary-checkpoint-from-s3/adapter_model.bin"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ef91a5d",
   "metadata": {
    "papermill": {
     "duration": 0.027234,
     "end_time": "2023-07-13T20:42:16.371189",
     "exception": false,
     "start_time": "2023-07-13T20:42:16.343955",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Prepare this model by adding an adapter to the original FLAN-T5 model. You are setting `is_trainable=False` because the plan is only to perform inference with this PEFT model. If you were preparing the model for further training, you would set `is_trainable=True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "88be5774",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:16.428296Z",
     "iopub.status.busy": "2023-07-13T20:42:16.427439Z",
     "iopub.status.idle": "2023-07-13T20:42:22.901943Z",
     "shell.execute_reply": "2023-07-13T20:42:22.900911Z"
    },
    "papermill": {
     "duration": 6.505513,
     "end_time": "2023-07-13T20:42:22.904349",
     "exception": false,
     "start_time": "2023-07-13T20:42:16.398836",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from peft import PeftModel, PeftConfig\n",
    "\n",
    "peft_model_base = AutoModelForSeq2SeqLM.from_pretrained(\"/kaggle/input/flan-t5/pytorch/base/4\", torch_dtype=torch.bfloat16)\n",
    "tokenizer = AutoTokenizer.from_pretrained(\"/kaggle/input/flan-t5/pytorch/base/4\")\n",
    "\n",
    "peft_model = PeftModel.from_pretrained(peft_model_base, \n",
    "                                       '/kaggle/input/generative-ai-with-llms-lab-2/lab_2/peft-dialogue-summary-checkpoint-from-s3', \n",
    "                                       torch_dtype=torch.bfloat16,\n",
    "                                       is_trainable=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9f04d55",
   "metadata": {
    "papermill": {
     "duration": 0.027183,
     "end_time": "2023-07-13T20:42:22.959706",
     "exception": false,
     "start_time": "2023-07-13T20:42:22.932523",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The number of trainable parameters will be `0` due to `is_trainable=False` setting:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "0bae1802",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:23.016788Z",
     "iopub.status.busy": "2023-07-13T20:42:23.015211Z",
     "iopub.status.idle": "2023-07-13T20:42:23.026511Z",
     "shell.execute_reply": "2023-07-13T20:42:23.025147Z"
    },
    "papermill": {
     "duration": 0.041577,
     "end_time": "2023-07-13T20:42:23.028483",
     "exception": false,
     "start_time": "2023-07-13T20:42:22.986906",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable model parameters: 0\n",
      "all model parameters: 251116800\n",
      "percentage of trainable model parameters: 0.00%\n"
     ]
    }
   ],
   "source": [
    "print(print_number_of_trainable_model_parameters(peft_model))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cfc69845",
   "metadata": {
    "papermill": {
     "duration": 0.026939,
     "end_time": "2023-07-13T20:42:23.082929",
     "exception": false,
     "start_time": "2023-07-13T20:42:23.055990",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='3.3'></a>\n",
    "### 3.3 - Evaluate the Model Qualitatively (Human Evaluation)\n",
    "\n",
    "Make inferences for the same example as in sections [1.3](#1.3) and [2.3](#2.3), with the original model, fully fine-tuned and PEFT model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "44e417ac",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:23.139257Z",
     "iopub.status.busy": "2023-07-13T20:42:23.138377Z",
     "iopub.status.idle": "2023-07-13T20:42:23.662748Z",
     "shell.execute_reply": "2023-07-13T20:42:23.661748Z"
    },
    "papermill": {
     "duration": 0.554977,
     "end_time": "2023-07-13T20:42:23.665123",
     "exception": false,
     "start_time": "2023-07-13T20:42:23.110146",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "peft_model = peft_model.to('cpu')\n",
    "instruct_model = instruct_model.to('cpu')\n",
    "original_model = original_model.to('cpu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "d859a5ac",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:23.722844Z",
     "iopub.status.busy": "2023-07-13T20:42:23.721955Z",
     "iopub.status.idle": "2023-07-13T20:42:46.963105Z",
     "shell.execute_reply": "2023-07-13T20:42:46.959358Z"
    },
    "papermill": {
     "duration": 23.272325,
     "end_time": "2023-07-13T20:42:46.965415",
     "exception": false,
     "start_time": "2023-07-13T20:42:23.693090",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------------------------------------------------------------------------------------------\n",
      "BASELINE HUMAN SUMMARY:\n",
      "#Person1# teaches #Person2# how to upgrade software and hardware in #Person2#'s system.\n",
      "---------------------------------------------------------------------------------------------------\n",
      "ORIGINAL MODEL:\n",
      "#Person1#: Have you considered upgrading your computer? #Person2#: I'm not sure yet. #Person1#: You're right. #Person2#: I'd like to upgrade my computer. #Person2#: Is it possible to make up your own flyers and banners? #Person2#: I'm not sure. #Person1#: Is it possible to make up your own flyers and banners? #Person2#: I'm not sure. #Person2#: I'm not sure. #Person2#: I'm not sure. #Person2#: I'm not sure. #Person1#: I'm not sure. #Person2#: I'm not sure. #Person2#: I'm not sure. #Person2#: I'm not\n",
      "---------------------------------------------------------------------------------------------------\n",
      "INSTRUCT MODEL:\n",
      "#Person1# suggests #Person2# upgrading #Person2#'s system, hardware, and CD-ROM drive. #Person2# thinks it's great.\n",
      "---------------------------------------------------------------------------------------------------\n",
      "PEFT MODEL: #Person1# recommends adding a painting program to #Person2#'s software and upgrading hardware. #Person2# also wants to upgrade the hardware because it's outdated now.\n"
     ]
    }
   ],
   "source": [
    "index = 200\n",
    "dialogue = dataset['test'][index]['dialogue']\n",
    "baseline_human_summary = dataset['test'][index]['summary']\n",
    "\n",
    "prompt = f\"\"\"\n",
    "Summarize the following conversation.\n",
    "\n",
    "{dialogue}\n",
    "\n",
    "Summary: \"\"\"\n",
    "\n",
    "input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids\n",
    "\n",
    "original_model_outputs = original_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200, num_beams=1))\n",
    "original_model_text_output = tokenizer.decode(original_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "instruct_model_outputs = instruct_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200, num_beams=1))\n",
    "instruct_model_text_output = tokenizer.decode(instruct_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "peft_model_outputs = peft_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200, num_beams=1))\n",
    "peft_model_text_output = tokenizer.decode(peft_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "print(dash_line)\n",
    "print(f'BASELINE HUMAN SUMMARY:\\n{human_baseline_summary}')\n",
    "print(dash_line)\n",
    "print(f'ORIGINAL MODEL:\\n{original_model_text_output}')\n",
    "print(dash_line)\n",
    "print(f'INSTRUCT MODEL:\\n{instruct_model_text_output}')\n",
    "print(dash_line)\n",
    "print(f'PEFT MODEL: {peft_model_text_output}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2636aec",
   "metadata": {
    "papermill": {
     "duration": 0.027281,
     "end_time": "2023-07-13T20:42:47.020822",
     "exception": false,
     "start_time": "2023-07-13T20:42:46.993541",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<a name='3.4'></a>\n",
    "### 3.4 - Evaluate the Model Quantitatively (with ROUGE Metric)\n",
    "Perform inferences for the sample of the test dataset (only 10 dialogues and summaries to save time). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "4aa7cb7f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:42:47.077679Z",
     "iopub.status.busy": "2023-07-13T20:42:47.076763Z",
     "iopub.status.idle": "2023-07-13T20:45:43.444751Z",
     "shell.execute_reply": "2023-07-13T20:45:43.443681Z"
    },
    "papermill": {
     "duration": 176.42817,
     "end_time": "2023-07-13T20:45:43.476373",
     "exception": false,
     "start_time": "2023-07-13T20:42:47.048203",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>human_baseline_summaries</th>\n",
       "      <th>original_model_summaries</th>\n",
       "      <th>instruct_model_summaries</th>\n",
       "      <th>peft_model_summaries</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Ms. Dawson helps #Person1# to write a memo to ...</td>\n",
       "      <td>A memo has been sent to all employees.</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>In order to prevent employees from wasting tim...</td>\n",
       "      <td>#Person1#: Hello, everyone. #Person2#: Hello, ...</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ms. Dawson takes a dictation for #Person1# abo...</td>\n",
       "      <td>Employees are required to take notice of a mem...</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>#Person2# arrives late because of traffic jam....</td>\n",
       "      <td>People are talking about the car and the subway.</td>\n",
       "      <td>#Person2# got stuck in traffic again. #Person1...</td>\n",
       "      <td>#Person2# got stuck in traffic and #Person1# s...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>#Person2# decides to follow #Person1#'s sugges...</td>\n",
       "      <td>The traffic jams are bothering people.</td>\n",
       "      <td>#Person2# got stuck in traffic again. #Person1...</td>\n",
       "      <td>#Person2# got stuck in traffic and #Person1# s...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>#Person2# complains to #Person1# about the tra...</td>\n",
       "      <td>People are talking about the car and the envir...</td>\n",
       "      <td>#Person2# got stuck in traffic again. #Person1...</td>\n",
       "      <td>#Person2# got stuck in traffic and #Person1# s...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>#Person1# tells Kate that Masha and Hero get d...</td>\n",
       "      <td>Masha and Hero are getting divorces.</td>\n",
       "      <td>Masha and Hero are getting divorced. Kate can'...</td>\n",
       "      <td>Kate tells #Person2# Masha and Hero are gettin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>#Person1# tells Kate that Masha and Hero are g...</td>\n",
       "      <td>#Person1: Masha and Hero are getting divorced....</td>\n",
       "      <td>Masha and Hero are getting divorced. Kate can'...</td>\n",
       "      <td>Kate tells #Person2# Masha and Hero are gettin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>#Person1# and Kate talk about the divorce betw...</td>\n",
       "      <td>Masha and Hero are divorced.</td>\n",
       "      <td>Masha and Hero are getting divorced. Kate can'...</td>\n",
       "      <td>Kate tells #Person2# Masha and Hero are gettin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>#Person1# and Brian are at the birthday party ...</td>\n",
       "      <td>Brian is having a birthday party.</td>\n",
       "      <td>Brian's birthday is coming. #Person1# invites ...</td>\n",
       "      <td>Brian remembers his birthday and invites #Pers...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                            human_baseline_summaries  \\\n",
       "0  Ms. Dawson helps #Person1# to write a memo to ...   \n",
       "1  In order to prevent employees from wasting tim...   \n",
       "2  Ms. Dawson takes a dictation for #Person1# abo...   \n",
       "3  #Person2# arrives late because of traffic jam....   \n",
       "4  #Person2# decides to follow #Person1#'s sugges...   \n",
       "5  #Person2# complains to #Person1# about the tra...   \n",
       "6  #Person1# tells Kate that Masha and Hero get d...   \n",
       "7  #Person1# tells Kate that Masha and Hero are g...   \n",
       "8  #Person1# and Kate talk about the divorce betw...   \n",
       "9  #Person1# and Brian are at the birthday party ...   \n",
       "\n",
       "                            original_model_summaries  \\\n",
       "0             A memo has been sent to all employees.   \n",
       "1  #Person1#: Hello, everyone. #Person2#: Hello, ...   \n",
       "2  Employees are required to take notice of a mem...   \n",
       "3   People are talking about the car and the subway.   \n",
       "4             The traffic jams are bothering people.   \n",
       "5  People are talking about the car and the envir...   \n",
       "6               Masha and Hero are getting divorces.   \n",
       "7  #Person1: Masha and Hero are getting divorced....   \n",
       "8                       Masha and Hero are divorced.   \n",
       "9                  Brian is having a birthday party.   \n",
       "\n",
       "                            instruct_model_summaries  \\\n",
       "0  #Person1# asks Ms. Dawson to take a dictation ...   \n",
       "1  #Person1# asks Ms. Dawson to take a dictation ...   \n",
       "2  #Person1# asks Ms. Dawson to take a dictation ...   \n",
       "3  #Person2# got stuck in traffic again. #Person1...   \n",
       "4  #Person2# got stuck in traffic again. #Person1...   \n",
       "5  #Person2# got stuck in traffic again. #Person1...   \n",
       "6  Masha and Hero are getting divorced. Kate can'...   \n",
       "7  Masha and Hero are getting divorced. Kate can'...   \n",
       "8  Masha and Hero are getting divorced. Kate can'...   \n",
       "9  Brian's birthday is coming. #Person1# invites ...   \n",
       "\n",
       "                                peft_model_summaries  \n",
       "0  #Person1# asks Ms. Dawson to take a dictation ...  \n",
       "1  #Person1# asks Ms. Dawson to take a dictation ...  \n",
       "2  #Person1# asks Ms. Dawson to take a dictation ...  \n",
       "3  #Person2# got stuck in traffic and #Person1# s...  \n",
       "4  #Person2# got stuck in traffic and #Person1# s...  \n",
       "5  #Person2# got stuck in traffic and #Person1# s...  \n",
       "6  Kate tells #Person2# Masha and Hero are gettin...  \n",
       "7  Kate tells #Person2# Masha and Hero are gettin...  \n",
       "8  Kate tells #Person2# Masha and Hero are gettin...  \n",
       "9  Brian remembers his birthday and invites #Pers...  "
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dialogues = dataset['test'][0:10]['dialogue']\n",
    "human_baseline_summaries = dataset['test'][0:10]['summary']\n",
    "\n",
    "original_model_summaries = []\n",
    "instruct_model_summaries = []\n",
    "peft_model_summaries = []\n",
    "\n",
    "for idx, dialogue in enumerate(dialogues):\n",
    "    prompt = f\"\"\"\n",
    "Summarize the following conversation.\n",
    "\n",
    "{dialogue}\n",
    "\n",
    "Summary: \"\"\"\n",
    "    \n",
    "    input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids\n",
    "\n",
    "    human_baseline_text_output = human_baseline_summaries[idx]\n",
    "    \n",
    "    original_model_outputs = original_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200))\n",
    "    original_model_text_output = tokenizer.decode(original_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "    instruct_model_outputs = instruct_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200))\n",
    "    instruct_model_text_output = tokenizer.decode(instruct_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "    peft_model_outputs = peft_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200))\n",
    "    peft_model_text_output = tokenizer.decode(peft_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "    original_model_summaries.append(original_model_text_output)\n",
    "    instruct_model_summaries.append(instruct_model_text_output)\n",
    "    peft_model_summaries.append(peft_model_text_output)\n",
    "\n",
    "zipped_summaries = list(zip(human_baseline_summaries, original_model_summaries, instruct_model_summaries, peft_model_summaries))\n",
    " \n",
    "df = pd.DataFrame(zipped_summaries, columns = ['human_baseline_summaries', 'original_model_summaries', 'instruct_model_summaries', 'peft_model_summaries'])\n",
    "df"
   ]
  },
  {
   "cell_type": "raw",
   "id": "d3e4cd9b",
   "metadata": {
    "papermill": {
     "duration": 0.027463,
     "end_time": "2023-07-13T20:45:43.531665",
     "exception": false,
     "start_time": "2023-07-13T20:45:43.504202",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Compute ROUGE score for this subset of the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "b0ab696e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:45:43.589038Z",
     "iopub.status.busy": "2023-07-13T20:45:43.588649Z",
     "iopub.status.idle": "2023-07-13T20:45:44.830965Z",
     "shell.execute_reply": "2023-07-13T20:45:44.829961Z"
    },
    "papermill": {
     "duration": 1.274009,
     "end_time": "2023-07-13T20:45:44.833369",
     "exception": false,
     "start_time": "2023-07-13T20:45:43.559360",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ORIGINAL MODEL:\n",
      "{'rouge1': 0.22914208300897826, 'rouge2': 0.0768445322793149, 'rougeL': 0.20024811540404724, 'rougeLsum': 0.20452598039619796}\n",
      "INSTRUCT MODEL:\n",
      "{'rouge1': 0.41026607717457186, 'rouge2': 0.17840645241958838, 'rougeL': 0.2977022096267017, 'rougeLsum': 0.2987374187518165}\n",
      "PEFT MODEL:\n",
      "{'rouge1': 0.3725351062275605, 'rouge2': 0.12138811933618107, 'rougeL': 0.27620639623170606, 'rougeLsum': 0.2758134870822362}\n"
     ]
    }
   ],
   "source": [
    "rouge = evaluate.load('rouge')\n",
    "\n",
    "original_model_results = rouge.compute(\n",
    "    predictions=original_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(original_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "instruct_model_results = rouge.compute(\n",
    "    predictions=instruct_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(instruct_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "peft_model_results = rouge.compute(\n",
    "    predictions=peft_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(peft_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "print('ORIGINAL MODEL:')\n",
    "print(original_model_results)\n",
    "print('INSTRUCT MODEL:')\n",
    "print(instruct_model_results)\n",
    "print('PEFT MODEL:')\n",
    "print(peft_model_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb5df8bb",
   "metadata": {
    "papermill": {
     "duration": 0.028364,
     "end_time": "2023-07-13T20:45:44.890507",
     "exception": false,
     "start_time": "2023-07-13T20:45:44.862143",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Notice, that PEFT model results are not too bad, while the training process was much easier!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1986b2d5",
   "metadata": {
    "papermill": {
     "duration": 0.029943,
     "end_time": "2023-07-13T20:45:44.949084",
     "exception": false,
     "start_time": "2023-07-13T20:45:44.919141",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "You already computed ROUGE score on the full dataset, after loading the results from the `data/dialogue-summary-training-results.csv` file. Load the values for the PEFT model now and check its performance compared to other models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "9a557818",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:45:45.006953Z",
     "iopub.status.busy": "2023-07-13T20:45:45.005926Z",
     "iopub.status.idle": "2023-07-13T20:45:59.713651Z",
     "shell.execute_reply": "2023-07-13T20:45:59.712352Z"
    },
    "papermill": {
     "duration": 14.739038,
     "end_time": "2023-07-13T20:45:59.715848",
     "exception": false,
     "start_time": "2023-07-13T20:45:44.976810",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ORIGINAL MODEL:\n",
      "{'rouge1': 0.2334158581572823, 'rouge2': 0.07603964187010573, 'rougeL': 0.20145520923859048, 'rougeLsum': 0.20145899339006135}\n",
      "INSTRUCT MODEL:\n",
      "{'rouge1': 0.42161291557556113, 'rouge2': 0.18035380596301792, 'rougeL': 0.3384439349963909, 'rougeLsum': 0.33835653595561666}\n",
      "PEFT MODEL:\n",
      "{'rouge1': 0.40810631575616746, 'rouge2': 0.1633255794568712, 'rougeL': 0.32507074586565354, 'rougeLsum': 0.3248950182867091}\n"
     ]
    }
   ],
   "source": [
    "human_baseline_summaries = results['human_baseline_summaries'].values\n",
    "original_model_summaries = results['original_model_summaries'].values\n",
    "instruct_model_summaries = results['instruct_model_summaries'].values\n",
    "peft_model_summaries     = results['peft_model_summaries'].values\n",
    "\n",
    "original_model_results = rouge.compute(\n",
    "    predictions=original_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(original_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "instruct_model_results = rouge.compute(\n",
    "    predictions=instruct_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(instruct_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "peft_model_results = rouge.compute(\n",
    "    predictions=peft_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(peft_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "print('ORIGINAL MODEL:')\n",
    "print(original_model_results)\n",
    "print('INSTRUCT MODEL:')\n",
    "print(instruct_model_results)\n",
    "print('PEFT MODEL:')\n",
    "print(peft_model_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cae9ae28",
   "metadata": {
    "papermill": {
     "duration": 0.028221,
     "end_time": "2023-07-13T20:45:59.773514",
     "exception": false,
     "start_time": "2023-07-13T20:45:59.745293",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The results show less of an improvement over full fine-tuning, but the benefits of PEFT typically outweigh the slightly-lower performance metrics.\n",
    "\n",
    "Calculate the improvement of PEFT over the original model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "1c6193b3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:45:59.831750Z",
     "iopub.status.busy": "2023-07-13T20:45:59.830702Z",
     "iopub.status.idle": "2023-07-13T20:45:59.837921Z",
     "shell.execute_reply": "2023-07-13T20:45:59.836609Z"
    },
    "papermill": {
     "duration": 0.038401,
     "end_time": "2023-07-13T20:45:59.839974",
     "exception": false,
     "start_time": "2023-07-13T20:45:59.801573",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Absolute percentage improvement of PEFT MODEL over HUMAN BASELINE\n",
      "rouge1: 17.47%\n",
      "rouge2: 8.73%\n",
      "rougeL: 12.36%\n",
      "rougeLsum: 12.34%\n"
     ]
    }
   ],
   "source": [
    "print(\"Absolute percentage improvement of PEFT MODEL over HUMAN BASELINE\")\n",
    "\n",
    "improvement = (np.array(list(peft_model_results.values())) - np.array(list(original_model_results.values())))\n",
    "for key, value in zip(peft_model_results.keys(), improvement):\n",
    "    print(f'{key}: {value*100:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9a84d8a",
   "metadata": {
    "papermill": {
     "duration": 0.028374,
     "end_time": "2023-07-13T20:45:59.897099",
     "exception": false,
     "start_time": "2023-07-13T20:45:59.868725",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now calculate the improvement of PEFT over a full fine-tuned model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "25575dba",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-13T20:45:59.955172Z",
     "iopub.status.busy": "2023-07-13T20:45:59.954833Z",
     "iopub.status.idle": "2023-07-13T20:45:59.961400Z",
     "shell.execute_reply": "2023-07-13T20:45:59.960368Z"
    },
    "papermill": {
     "duration": 0.038977,
     "end_time": "2023-07-13T20:45:59.964420",
     "exception": false,
     "start_time": "2023-07-13T20:45:59.925443",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Absolute percentage improvement of PEFT MODEL over INSTRUCT MODEL\n",
      "rouge1: -1.35%\n",
      "rouge2: -1.70%\n",
      "rougeL: -1.34%\n",
      "rougeLsum: -1.35%\n"
     ]
    }
   ],
   "source": [
    "print(\"Absolute percentage improvement of PEFT MODEL over INSTRUCT MODEL\")\n",
    "\n",
    "improvement = (np.array(list(peft_model_results.values())) - np.array(list(instruct_model_results.values())))\n",
    "for key, value in zip(peft_model_results.keys(), improvement):\n",
    "    print(f'{key}: {value*100:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91a380f1",
   "metadata": {
    "papermill": {
     "duration": 0.033431,
     "end_time": "2023-07-13T20:46:00.028485",
     "exception": false,
     "start_time": "2023-07-13T20:45:59.995054",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Here you see a small percentage decrease in the ROUGE metrics vs. full fine-tuned. However, the training requires much less computing and memory resources (often just a single GPU)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33b6ce79",
   "metadata": {
    "papermill": {
     "duration": 0.028169,
     "end_time": "2023-07-13T20:46:00.084778",
     "exception": false,
     "start_time": "2023-07-13T20:46:00.056609",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "72ba5185",
   "metadata": {
    "papermill": {
     "duration": 0.027865,
     "end_time": "2023-07-13T20:46:00.140665",
     "exception": false,
     "start_time": "2023-07-13T20:46:00.112800",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Source:\n",
    " - https://www.coursera.org/learn/generative-ai-with-llms\n",
    " - https://www.coursera.org/learn/generative-ai-with-llms/gradedLti/x0gc1/lab-2-fine-tune-a-generative-ai-model-for-dialogue-summarization\n",
    " \n",
    "https://creativecommons.org/licenses/by-sa/2.0/legalcode"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 643.033537,
   "end_time": "2023-07-13T20:46:03.746634",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2023-07-13T20:35:20.713097",
   "version": "2.4.0"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "023d246081134f4bbd5bf14fa3d5be28": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "02ca147189024db183d2034153f53b4c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "0365556f57ed42dba068e4680bac69ce": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "03d65615ae324d3a9ee7a5e4da98efbb": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "041646f376014b95b933de37351c3eae": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "041a01f81b2442afb007a0d2e0f6f412": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_824b0375c3ba427eb3eb16a0d6e17553",
       "max": 1353147.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_29920213c2c845cb87feaffe8d239f59",
       "value": 1353147.0
      }
     },
     "0a80bd40f05e487d95426d80a8dff826": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "0a83f38cee874fc28981edee0b6c9ede": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "0c4bec61eca8487691ac5cc738a3c4a4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_efb0eeea2c5f4e48be04d56e47acff99",
        "IPY_MODEL_23a431d0b3974b5a8eeb1892804cb589",
        "IPY_MODEL_7f757a8a486948e484180d582adb4e5d"
       ],
       "layout": "IPY_MODEL_7ca70a98ac16420dbe7fe68aa01760d9"
      }
     },
     "1007a7ba4bef4be29216bfc3b66ecd9f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_d646e0aa4a674d3eaf7b878406d2b74c",
        "IPY_MODEL_9e8c967aabdc423b83e99cbe392d431d",
        "IPY_MODEL_b2a34b9e92ef444c98733c36565d8e45"
       ],
       "layout": "IPY_MODEL_66ed1e34d73e4eccb454f927c84d6eaf"
      }
     },
     "102a8f21c86e43d2ae5dc5438fa230f2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "155fec12824d4ea69db7dab4bb889dda": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "1639273d6f474b7cbe7323eb7c3bd153": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_d6c874861c864ed2b72a19cb1c8e0c6e",
        "IPY_MODEL_9151c4bfe2e24a33923503f6ec6c422f",
        "IPY_MODEL_33e9586aeb9b43b2a0ac26e5d8b35fd7"
       ],
       "layout": "IPY_MODEL_44f77a232979401ca63fd11c5a348991"
      }
     },
     "16f7c91e524c494db81fe42c9bd348d8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_89f3e555be3f422db006d98816e2f946",
       "max": 500.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_ae1b81044c0b4a0b9fc3aa22374dfc33",
       "value": 500.0
      }
     },
     "187fd2f162f040009d4e244f73a23490": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "18d43f15899e4c8f9299dbc2f8cf5cdb": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_62549814481f4f48b010369ccb565559",
        "IPY_MODEL_f1cee82c9ec3428ba014321b547b3d04",
        "IPY_MODEL_b9012b4771be496680557751ce03765e"
       ],
       "layout": "IPY_MODEL_4a306b69012d4370a382d31dc786885e"
      }
     },
     "18f645853bdf414e9910d4016d616ab0": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_cc797c9552704148aa9e637ad99a3dfd",
       "placeholder": "​",
       "style": "IPY_MODEL_aac2b5fb03e1493084650456167c355c",
       "value": " 12460/12460 [00:11&lt;00:00, 1085.36 examples/s]"
      }
     },
     "1a3039c93f6548f9861272bceff6a05a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "1b71f7053a0641d5b72459f628e08d98": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_2732bc9e2a2f4abb965f611ee8587e25",
       "placeholder": "​",
       "style": "IPY_MODEL_405dead4270644a2b8a7f6ab947ec4db",
       "value": " 1500/1500 [00:01&lt;00:00, 1440.76 examples/s]"
      }
     },
     "1bc7b69dbb7d431a855aedfbe1f20fef": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "1cc36b2d29ae4482b7857a1a4d12dc3f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "1cf7abeae1084829835b64867e5388c2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "20a190cae1ee460183389e49aa58f9c4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "20fc99b77994498091d34d4796a1f299": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "21217b236fcb4e5294c19a744f581306": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "23a431d0b3974b5a8eeb1892804cb589": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_ed9465224c2e435494de7f707f76ef40",
       "max": 3.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_c226764e26d94130907b57ca817b7a08",
       "value": 3.0
      }
     },
     "24b544ed008149aea02d9432b7b4d02f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "24c606093a9144d7a1d5ce4a21669351": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "260e0e4d59e04192bf0c3354351bb72f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_53694acb571640a99214d476fa3b6577",
       "placeholder": "​",
       "style": "IPY_MODEL_742abba6e44d4d4b99b9054300a98557",
       "value": " 1.35M/1.35M [00:00&lt;00:00, 5.25MB/s]"
      }
     },
     "272072ce6c94474cbddc07383253b1ef": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_598306b8aec64d15b03cbc3013f9cafd",
       "max": 500.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_5b0f08951407457094e285814be16cc1",
       "value": 500.0
      }
     },
     "2732bc9e2a2f4abb965f611ee8587e25": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "2949133081224031986fd2428a514aa7": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": "hidden",
       "width": null
      }
     },
     "29920213c2c845cb87feaffe8d239f59": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "2d10214832884bd796e97c9fe018c7ed": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_ce3899c43beb4578acbb3b8d3295c60a",
       "max": 1500.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_a4aa9e3326d140128bb2fda6226548a1",
       "value": 1500.0
      }
     },
     "2feb3a6727ab4626a0fb9c471090a838": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "30e784fa229040afb62088e645dccbd8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "330c30a3cf224f29b336c6cc0bab109a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3387941e8c774d3796cdb3c940ecc25d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "33de03ee277640dc93f90cc17f4f8a5e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "33e2e0a40d524c18a28729bf6e665612": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_f628296a644e439aac20217b0ad88463",
       "placeholder": "​",
       "style": "IPY_MODEL_20a190cae1ee460183389e49aa58f9c4",
       "value": " 1500/1500 [00:01&lt;00:00, 1077.37 examples/s]"
      }
     },
     "33e9586aeb9b43b2a0ac26e5d8b35fd7": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_5f0087a379bf440cb1533761bfe6f0ff",
       "placeholder": "​",
       "style": "IPY_MODEL_4445bc25a9884320a4e490ee34f8cea8",
       "value": " 3/3 [00:00&lt;00:00, 156.08it/s]"
      }
     },
     "385573b184e542a2bc14a1cc04e47143": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "39d7b5f8573143cf9c41f231d6b00baf": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_59df131bc0e04f3c9528da3e976861b8",
       "max": 4563.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_8b9f7e8f62624512b8a6d42992363596",
       "value": 4563.0
      }
     },
     "3b10b88ca5db4565841b9f51fa3f2be0": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_7d162940a13b4155a4339ae60a95c731",
       "placeholder": "​",
       "style": "IPY_MODEL_24b544ed008149aea02d9432b7b4d02f",
       "value": " 3/3 [00:00&lt;00:00, 180.37it/s]"
      }
     },
     "3b3878e39d1d4968bcc578575d69d3ea": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_d2da5574ed2c4118b9b91dc4d4016a49",
       "placeholder": "​",
       "style": "IPY_MODEL_db0a22503f0a47a393b69e1d9ef5c385",
       "value": "Downloading builder script: 100%"
      }
     },
     "3d71081073654e38b564b37161edb380": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_5a82d22781fa481785e792130e139db7",
        "IPY_MODEL_4d1f2c40e3d74f619449346b7438be3f",
        "IPY_MODEL_53a6a16da91c492c8c38dae9b3db11ee"
       ],
       "layout": "IPY_MODEL_c68737f67d1c49cbb05fc8bfa6735396"
      }
     },
     "3e79a0629cc74c56b829b50e09ce3bcf": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "4052179142364e8fa574b5e87c7ebb91": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "405dead4270644a2b8a7f6ab947ec4db": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "42c000ddcbb141bfb517cfc37c0ed8a2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "4445bc25a9884320a4e490ee34f8cea8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "445aab41973c47b9ad9a37caa221bd8e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_fe47fd5054cb48508c4bb133ebb0690e",
       "placeholder": "​",
       "style": "IPY_MODEL_c22441e7f0a14dedb5159af11e05bf6e",
       "value": " 500/500 [00:00&lt;00:00, 1388.88 examples/s]"
      }
     },
     "44f77a232979401ca63fd11c5a348991": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "4543f93d66f64541861a26ca57c9652c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "474b38719bc34419a4d68246a7580294": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_62da814a294e4500a0284fbfd0c8913f",
        "IPY_MODEL_82f42de0bda34e6b91dc8892caf3aa24",
        "IPY_MODEL_da3071cd7ed544a78c7e33f27adcbdde"
       ],
       "layout": "IPY_MODEL_d57f44cbcc0343ef97db4304e547fdc2"
      }
     },
     "4874a03fcbcc4b72a2b4eae8312b8007": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_6c216f5b2e1640e08e7d61d103b07bd8",
       "placeholder": "​",
       "style": "IPY_MODEL_023d246081134f4bbd5bf14fa3d5be28",
       "value": "100%"
      }
     },
     "490efd3b09aa4ff7b5883f0707f6d916": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "4a07db543523422e958b47c16f352e8e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "4a306b69012d4370a382d31dc786885e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "4be551620afd40fa8fa9c668581535b5": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "4c37abf97f124aa38b954c5580a14a1b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "4cd6ec42e67c43a0afbc0f6cdac5f54c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_4052179142364e8fa574b5e87c7ebb91",
       "placeholder": "​",
       "style": "IPY_MODEL_6c70d02bc9ef44668f80722dc9b09ddb",
       "value": "Map: 100%"
      }
     },
     "4d1f2c40e3d74f619449346b7438be3f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_8d7fa5673552450c8bd4f810f08e036d",
       "max": 12460.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_3387941e8c774d3796cdb3c940ecc25d",
       "value": 12460.0
      }
     },
     "4d821fbc48a44fc8b8f41a4cc92e9fe9": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "50ea8c0d12d24ee7aef3051bc1e6f3f5": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "52bec2bf52b14cb4b37313939b194d92": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_e6b468c53976426fbb494b04e3742bde",
       "max": 6270.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_4a07db543523422e958b47c16f352e8e",
       "value": 6270.0
      }
     },
     "53694acb571640a99214d476fa3b6577": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "53a6a16da91c492c8c38dae9b3db11ee": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_b5b73ac1673e44c2a76dd4e8e3a53cbd",
       "placeholder": "​",
       "style": "IPY_MODEL_187fd2f162f040009d4e244f73a23490",
       "value": " 12460/12460 [00:09&lt;00:00, 1387.70 examples/s]"
      }
     },
     "563acf499acf429a811a1f3916f814da": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_42c000ddcbb141bfb517cfc37c0ed8a2",
       "placeholder": "​",
       "style": "IPY_MODEL_20fc99b77994498091d34d4796a1f299",
       "value": "Downloading data: 100%"
      }
     },
     "572a3dbc8e9e4104815fe23e91d17f35": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "575eb6485c904b6d824e5b0c591ff876": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_7ad0f4c546494033b394730f11156401",
       "placeholder": "​",
       "style": "IPY_MODEL_30e784fa229040afb62088e645dccbd8",
       "value": "Map: 100%"
      }
     },
     "578637cfddcf464cbcc238e5e406a142": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_5fc9a414ebb2499f96a4570507509c48",
       "placeholder": "​",
       "style": "IPY_MODEL_0365556f57ed42dba068e4680bac69ce",
       "value": "Computing checksums: 100%"
      }
     },
     "58705663ac0645c1ae99e1b47d60f72f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_ef1974b5694f414aac8192f67bb8e32a",
       "placeholder": "​",
       "style": "IPY_MODEL_c49792ea655d42a8a32b9a649d696e2b",
       "value": " 442k/442k [00:00&lt;00:00, 613kB/s]"
      }
     },
     "598306b8aec64d15b03cbc3013f9cafd": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "59df131bc0e04f3c9528da3e976861b8": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "5a82d22781fa481785e792130e139db7": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_1a3039c93f6548f9861272bceff6a05a",
       "placeholder": "​",
       "style": "IPY_MODEL_0a83f38cee874fc28981edee0b6c9ede",
       "value": "Filter: 100%"
      }
     },
     "5b0f08951407457094e285814be16cc1": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "5b7e32e1684b4374927c3ac6cc2bc5b5": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "5f0087a379bf440cb1533761bfe6f0ff": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "5fc9a414ebb2499f96a4570507509c48": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "61679057815f4e4682889d4dabb153e5": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_578637cfddcf464cbcc238e5e406a142",
        "IPY_MODEL_8d4504acfa8d49cb986b96a8d41d202b",
        "IPY_MODEL_3b10b88ca5db4565841b9f51fa3f2be0"
       ],
       "layout": "IPY_MODEL_eb4d62e6dcba4e5e9f332768548fac60"
      }
     },
     "62549814481f4f48b010369ccb565559": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_041646f376014b95b933de37351c3eae",
       "placeholder": "​",
       "style": "IPY_MODEL_0a80bd40f05e487d95426d80a8dff826",
       "value": "Downloading data: 100%"
      }
     },
     "62da814a294e4500a0284fbfd0c8913f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_f4af6ee3a93c44c1bd32d8981fd2695a",
       "placeholder": "​",
       "style": "IPY_MODEL_e844175500654f1f9f0601ae5130b9c3",
       "value": "Generating validation split: "
      }
     },
     "6363f614182e4ec7989e9df51512b0cf": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "66ed1e34d73e4eccb454f927c84d6eaf": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": "hidden",
       "width": null
      }
     },
     "6846e95477be4baaa2c9c15a1eec7149": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "69ce5c54fdc84a55926c51e74ee9c542": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_b668516392334117a6bb2ea932f80546",
       "placeholder": "​",
       "style": "IPY_MODEL_1bc7b69dbb7d431a855aedfbe1f20fef",
       "value": " 10000/0 [00:00&lt;00:00, 56129.19 examples/s]"
      }
     },
     "6aeab14fe4884f0bb8228c4eec52badc": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "6c216f5b2e1640e08e7d61d103b07bd8": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "6c70d02bc9ef44668f80722dc9b09ddb": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "6d1aecf059104b368b336ad3d7dcb546": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "6dc4d33b38084a2f892d979b32afc16c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "712f0da6a3814a9e96467e2f88a21984": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_fd65373397fb4af38a3dea99e3fd2ee3",
        "IPY_MODEL_fd4b322748dc4dbbbd8f6117bba7337a",
        "IPY_MODEL_58705663ac0645c1ae99e1b47d60f72f"
       ],
       "layout": "IPY_MODEL_de4b36a01fa14eeca800331a638c01d9"
      }
     },
     "72343cf307734515b6c2f284c365466b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "742abba6e44d4d4b99b9054300a98557": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "74464e881a154a4889d3f4b9edcb28c4": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "774858b457b04dfca0658184538e17a5": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "78351c7967af4ffc94456928a0e21f49": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "78df5408f2104354a5412080786732d1": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "7946cfce28d448d893f72da5d8158bc1": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": "hidden",
       "width": null
      }
     },
     "7ad0f4c546494033b394730f11156401": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "7b7062fb06734aff9338861d610b500b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "7c077dc23a5a4282a6423831392f528f": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "7ca70a98ac16420dbe7fe68aa01760d9": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "7d162940a13b4155a4339ae60a95c731": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "7e56845c41554f60b84ac4518dfc8533": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "7f757a8a486948e484180d582adb4e5d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_d6641de537814039b5086dcac6ebe829",
       "placeholder": "​",
       "style": "IPY_MODEL_fb3725a40b2f45e8a8a4d625bad18253",
       "value": " 3/3 [00:01&lt;00:00,  1.69it/s]"
      }
     },
     "809b50d12de04b9da952e5089b75f71b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_c450d1a969a844bfaa497e5ed7495f56",
       "placeholder": "​",
       "style": "IPY_MODEL_490efd3b09aa4ff7b5883f0707f6d916",
       "value": " 500/500 [00:00&lt;00:00, 1030.80 examples/s]"
      }
     },
     "824b0375c3ba427eb3eb16a0d6e17553": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "82f42de0bda34e6b91dc8892caf3aa24": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "info",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_b5265411cb8149bdbb7c7b44da9bb101",
       "max": 1.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_7e56845c41554f60b84ac4518dfc8533",
       "value": 1.0
      }
     },
     "87b296d08616469ea21b469cbff01811": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "8912b90a17464cee8647f037d0da1b5d": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "898216844b954b70804b3204c66c7241": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "89f3e555be3f422db006d98816e2f946": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "8b9f7e8f62624512b8a6d42992363596": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "8d4504acfa8d49cb986b96a8d41d202b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_78df5408f2104354a5412080786732d1",
       "max": 3.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_6d1aecf059104b368b336ad3d7dcb546",
       "value": 3.0
      }
     },
     "8d7fa5673552450c8bd4f810f08e036d": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "8e1a230c85ef4ec896918983f498318e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "info",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_c15f421c8a994b7abdc3f1b8fc4162f2",
       "max": 1.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_78351c7967af4ffc94456928a0e21f49",
       "value": 1.0
      }
     },
     "8ee1189db24d42efb240d329756ecb37": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_dd34013a21f045b2b3a208d25472f5e7",
       "max": 12460.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_572a3dbc8e9e4104815fe23e91d17f35",
       "value": 12460.0
      }
     },
     "9012ffab869f4e1ba1fbd837f6375f22": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_d61ad07f27494e35b15a1474e395ff18",
        "IPY_MODEL_16f7c91e524c494db81fe42c9bd348d8",
        "IPY_MODEL_445aab41973c47b9ad9a37caa221bd8e"
       ],
       "layout": "IPY_MODEL_2949133081224031986fd2428a514aa7"
      }
     },
     "9151c4bfe2e24a33923503f6ec6c422f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_102a8f21c86e43d2ae5dc5438fa230f2",
       "max": 3.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_1cc36b2d29ae4482b7857a1a4d12dc3f",
       "value": 3.0
      }
     },
     "923e204bfb3a4bd3a66bb674a7b25f52": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": "hidden",
       "width": null
      }
     },
     "925f1f332e034cbf81fbaf6a7377aa1d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_ffd3fddffa7b47c7866a43378065d3c0",
       "placeholder": "​",
       "style": "IPY_MODEL_ee4c2a0f7abe4922b97b633aa7b0a994",
       "value": "Downloading readme: 100%"
      }
     },
     "972f966cd74445668e6fb456d684ff1c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": "hidden",
       "width": null
      }
     },
     "98c576c1150848baa014291b3552eff5": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": "hidden",
       "width": null
      }
     },
     "9af58bf5ea264a96a05302cbc66aa4dd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_563acf499acf429a811a1f3916f814da",
        "IPY_MODEL_041a01f81b2442afb007a0d2e0f6f412",
        "IPY_MODEL_260e0e4d59e04192bf0c3354351bb72f"
       ],
       "layout": "IPY_MODEL_74464e881a154a4889d3f4b9edcb28c4"
      }
     },
     "9e8c967aabdc423b83e99cbe392d431d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "info",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_a77b5ffb16f7454a973087c07ec1f682",
       "max": 1.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_b6c33ed86f1b4d8abfc0f9d64d14aa3d",
       "value": 1.0
      }
     },
     "a4aa9e3326d140128bb2fda6226548a1": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "a77b5ffb16f7454a973087c07ec1f682": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": "20px"
      }
     },
     "a85db000427d4925b0e85d14ab6154e3": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": "hidden",
       "width": null
      }
     },
     "aac2b5fb03e1493084650456167c355c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "ae1b81044c0b4a0b9fc3aa22374dfc33": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "b1bcb47b3dca4ff0bf515b9d94e47d53": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_8912b90a17464cee8647f037d0da1b5d",
       "placeholder": "​",
       "style": "IPY_MODEL_87b296d08616469ea21b469cbff01811",
       "value": " 3/3 [00:00&lt;00:00, 109.18it/s]"
      }
     },
     "b2a34b9e92ef444c98733c36565d8e45": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_d38c2f2ac3d24df69f10bf1e0de74dbc",
       "placeholder": "​",
       "style": "IPY_MODEL_72343cf307734515b6c2f284c365466b",
       "value": " 0/0 [00:00&lt;?, ? examples/s]"
      }
     },
     "b5186b9e2ca44198b34044da4313f706": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "b5265411cb8149bdbb7c7b44da9bb101": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": "20px"
      }
     },
     "b5b73ac1673e44c2a76dd4e8e3a53cbd": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "b668516392334117a6bb2ea932f80546": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "b6c33ed86f1b4d8abfc0f9d64d14aa3d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "b9012b4771be496680557751ce03765e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_1cf7abeae1084829835b64867e5388c2",
       "placeholder": "​",
       "style": "IPY_MODEL_21217b236fcb4e5294c19a744f581306",
       "value": " 11.3M/11.3M [00:00&lt;00:00, 54.0MB/s]"
      }
     },
     "beb99f65856f4e8db9cf5b833291ec65": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_f6547c707d64462c83bc41d030a7d164",
       "placeholder": "​",
       "style": "IPY_MODEL_4c37abf97f124aa38b954c5580a14a1b",
       "value": "Map: 100%"
      }
     },
     "c02d5132e8914669b97aec9c9a9d5598": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_d0002711187c40ba9d5a5125a990c26e",
       "placeholder": "​",
       "style": "IPY_MODEL_7b7062fb06734aff9338861d610b500b",
       "value": "Generating train split: "
      }
     },
     "c15f421c8a994b7abdc3f1b8fc4162f2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": "20px"
      }
     },
     "c22441e7f0a14dedb5159af11e05bf6e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "c226764e26d94130907b57ca817b7a08": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "c450d1a969a844bfaa497e5ed7495f56": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "c49792ea655d42a8a32b9a649d696e2b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "c68737f67d1c49cbb05fc8bfa6735396": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": "hidden",
       "width": null
      }
     },
     "c8487a39aa2e4dcaa20fac223c36b247": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_385573b184e542a2bc14a1cc04e47143",
       "placeholder": "​",
       "style": "IPY_MODEL_dd9aebcfa78944f79effcecaadb44379",
       "value": "Filter: 100%"
      }
     },
     "c9b2d12f25444bf1ba1176d3dff708f7": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_4874a03fcbcc4b72a2b4eae8312b8007",
        "IPY_MODEL_cdb8c3a9add04b9eb8252ce900b098f9",
        "IPY_MODEL_b1bcb47b3dca4ff0bf515b9d94e47d53"
       ],
       "layout": "IPY_MODEL_6dc4d33b38084a2f892d979b32afc16c"
      }
     },
     "cc797c9552704148aa9e637ad99a3dfd": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "cdb8c3a9add04b9eb8252ce900b098f9": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_02ca147189024db183d2034153f53b4c",
       "max": 3.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_6aeab14fe4884f0bb8228c4eec52badc",
       "value": 3.0
      }
     },
     "ce3899c43beb4578acbb3b8d3295c60a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "d0002711187c40ba9d5a5125a990c26e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "d2da5574ed2c4118b9b91dc4d4016a49": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "d38c2f2ac3d24df69f10bf1e0de74dbc": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "d57f44cbcc0343ef97db4304e547fdc2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": "hidden",
       "width": null
      }
     },
     "d61ad07f27494e35b15a1474e395ff18": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_6846e95477be4baaa2c9c15a1eec7149",
       "placeholder": "​",
       "style": "IPY_MODEL_f0721eae9db049b5bad22efaf60bfe95",
       "value": "Filter: 100%"
      }
     },
     "d641af6e1a874938943290ccb5ab1303": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "d646e0aa4a674d3eaf7b878406d2b74c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_03d65615ae324d3a9ee7a5e4da98efbb",
       "placeholder": "​",
       "style": "IPY_MODEL_6363f614182e4ec7989e9df51512b0cf",
       "value": "Generating test split: "
      }
     },
     "d6641de537814039b5086dcac6ebe829": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "d686e6a558de4e47b5d6233e931bea46": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_575eb6485c904b6d824e5b0c591ff876",
        "IPY_MODEL_2d10214832884bd796e97c9fe018c7ed",
        "IPY_MODEL_33e2e0a40d524c18a28729bf6e665612"
       ],
       "layout": "IPY_MODEL_98c576c1150848baa014291b3552eff5"
      }
     },
     "d6c874861c864ed2b72a19cb1c8e0c6e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_3e79a0629cc74c56b829b50e09ce3bcf",
       "placeholder": "​",
       "style": "IPY_MODEL_e15fd32d43f5433a8899721a66c12f47",
       "value": "Extracting data files: 100%"
      }
     },
     "da3071cd7ed544a78c7e33f27adcbdde": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_774858b457b04dfca0658184538e17a5",
       "placeholder": "​",
       "style": "IPY_MODEL_ea51070d9c3c4f27a7f20ceeb84b31f9",
       "value": " 0/0 [00:00&lt;?, ? examples/s]"
      }
     },
     "da4334de88ce482eb652240c0d098575": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_4cd6ec42e67c43a0afbc0f6cdac5f54c",
        "IPY_MODEL_272072ce6c94474cbddc07383253b1ef",
        "IPY_MODEL_809b50d12de04b9da952e5089b75f71b"
       ],
       "layout": "IPY_MODEL_972f966cd74445668e6fb456d684ff1c"
      }
     },
     "db0a22503f0a47a393b69e1d9ef5c385": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "dd34013a21f045b2b3a208d25472f5e7": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "dd9aebcfa78944f79effcecaadb44379": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "de4b36a01fa14eeca800331a638c01d9": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "e01b02b7e2f34a15be8ad609260bd0e2": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_7c077dc23a5a4282a6423831392f528f",
       "placeholder": "​",
       "style": "IPY_MODEL_ed8dec11f748411f950b2a225caccd36",
       "value": " 6.27k/6.27k [00:00&lt;00:00, 363kB/s]"
      }
     },
     "e157670e79774d35ad6d5b088b63083f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_3b3878e39d1d4968bcc578575d69d3ea",
        "IPY_MODEL_52bec2bf52b14cb4b37313939b194d92",
        "IPY_MODEL_e01b02b7e2f34a15be8ad609260bd0e2"
       ],
       "layout": "IPY_MODEL_24c606093a9144d7a1d5ce4a21669351"
      }
     },
     "e15fd32d43f5433a8899721a66c12f47": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "e6b468c53976426fbb494b04e3742bde": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "e844175500654f1f9f0601ae5130b9c3": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "e95e6582212146d5bf2ba468dcdd0956": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_c02d5132e8914669b97aec9c9a9d5598",
        "IPY_MODEL_8e1a230c85ef4ec896918983f498318e",
        "IPY_MODEL_69ce5c54fdc84a55926c51e74ee9c542"
       ],
       "layout": "IPY_MODEL_a85db000427d4925b0e85d14ab6154e3"
      }
     },
     "e980b6b4027e42548ff4322963e355b2": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_898216844b954b70804b3204c66c7241",
       "max": 1500.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_4d821fbc48a44fc8b8f41a4cc92e9fe9",
       "value": 1500.0
      }
     },
     "ea51070d9c3c4f27a7f20ceeb84b31f9": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "eb4d62e6dcba4e5e9f332768548fac60": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ed8dec11f748411f950b2a225caccd36": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "ed9465224c2e435494de7f707f76ef40": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ee4c2a0f7abe4922b97b633aa7b0a994": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "ef1974b5694f414aac8192f67bb8e32a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "efb0eeea2c5f4e48be04d56e47acff99": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_155fec12824d4ea69db7dab4bb889dda",
       "placeholder": "​",
       "style": "IPY_MODEL_33de03ee277640dc93f90cc17f4f8a5e",
       "value": "Downloading data files: 100%"
      }
     },
     "f0721eae9db049b5bad22efaf60bfe95": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "f1cee82c9ec3428ba014321b547b3d04": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_fb3181c06c1e423e98e69b5251a566b3",
       "max": 11321474.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_d641af6e1a874938943290ccb5ab1303",
       "value": 11321474.0
      }
     },
     "f4af6ee3a93c44c1bd32d8981fd2695a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "f53066503ce14d35895b40b3f4d044de": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_c8487a39aa2e4dcaa20fac223c36b247",
        "IPY_MODEL_e980b6b4027e42548ff4322963e355b2",
        "IPY_MODEL_1b71f7053a0641d5b72459f628e08d98"
       ],
       "layout": "IPY_MODEL_7946cfce28d448d893f72da5d8158bc1"
      }
     },
     "f628296a644e439aac20217b0ad88463": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "f6547c707d64462c83bc41d030a7d164": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "f6bf08587d954f0f915352e005b63857": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_925f1f332e034cbf81fbaf6a7377aa1d",
        "IPY_MODEL_39d7b5f8573143cf9c41f231d6b00baf",
        "IPY_MODEL_f724b09915644b1f9d2fd29b9b882c68"
       ],
       "layout": "IPY_MODEL_5b7e32e1684b4374927c3ac6cc2bc5b5"
      }
     },
     "f724b09915644b1f9d2fd29b9b882c68": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_50ea8c0d12d24ee7aef3051bc1e6f3f5",
       "placeholder": "​",
       "style": "IPY_MODEL_4be551620afd40fa8fa9c668581535b5",
       "value": " 4.56k/4.56k [00:00&lt;00:00, 280kB/s]"
      }
     },
     "fb3181c06c1e423e98e69b5251a566b3": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "fb3725a40b2f45e8a8a4d625bad18253": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "fd05f305a47b41e3aabae9808951ba10": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_beb99f65856f4e8db9cf5b833291ec65",
        "IPY_MODEL_8ee1189db24d42efb240d329756ecb37",
        "IPY_MODEL_18f645853bdf414e9910d4016d616ab0"
       ],
       "layout": "IPY_MODEL_923e204bfb3a4bd3a66bb674a7b25f52"
      }
     },
     "fd4b322748dc4dbbbd8f6117bba7337a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_2feb3a6727ab4626a0fb9c471090a838",
       "max": 441935.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_4543f93d66f64541861a26ca57c9652c",
       "value": 441935.0
      }
     },
     "fd65373397fb4af38a3dea99e3fd2ee3": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_330c30a3cf224f29b336c6cc0bab109a",
       "placeholder": "​",
       "style": "IPY_MODEL_b5186b9e2ca44198b34044da4313f706",
       "value": "Downloading data: 100%"
      }
     },
     "fe47fd5054cb48508c4bb133ebb0690e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ffd3fddffa7b47c7866a43378065d3c0": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}