poetry-generation.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Poetry Generation\n",
    "\n",
    "Zach O'Brien\n",
    "\n",
    "December 2022\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import itertools\n",
    "import pickle\n",
    "import random\n",
    "random.seed(1)  # To make results reproducible\n",
    "import os\n",
    "os.environ[\"TF_CPP_MIN_LOG_LEVEL\"] = \"3\"  # Suppress tensorflow debugging info\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from nltk.util import pad_sequence\n",
    "from nltk.lm.preprocessing import flatten\n",
    "\n",
    "from dataprep.parse_leaves_of_grass import leaves_of_grass_gutenberg_to_df\n",
    "from dataprep.ngram_lm_dataprep import preprocess_for_ngram_lm, postprocess_for_ngram_lm\n",
    "from models.ngram_language_model import LanguageModel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "## 1. Data Preparation\n",
    "\n",
    "The raw data I used for this project was the complete text of Walt Whitman's \"Leaves of Grass\". To use it with NLP models, I had to parse out individual poems from the collection and then do some preprocessing on them.\n",
    "\n",
    "### 1.1 Parse Raw Document\n",
    "\n",
    "(See my source code `dataprep/` directory for implementation details on data preparation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>book_title</th>\n",
       "      <th>poem_title</th>\n",
       "      <th>poem</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BOOK I.  INSCRIPTIONS</td>\n",
       "      <td>One’s-Self I Sing</td>\n",
       "      <td>One’s-self I sing, a simple separate person,\\n...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BOOK I.  INSCRIPTIONS</td>\n",
       "      <td>As I Ponder’d in Silence</td>\n",
       "      <td>As I ponder’d in silence,\\n  Returning upon my...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              book_title                poem_title  \\\n",
       "0  BOOK I.  INSCRIPTIONS         One’s-Self I Sing   \n",
       "1  BOOK I.  INSCRIPTIONS  As I Ponder’d in Silence   \n",
       "\n",
       "                                                poem  \n",
       "0  One’s-self I sing, a simple separate person,\\n...  \n",
       "1  As I ponder’d in silence,\\n  Returning upon my...  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with open(\"data/leaves-of-grass.txt\") as f:\n",
    "    lines = f.readlines()\n",
    "\n",
    "# I encapsulated the functionality of parsing the raw text into this function.\n",
    "# It essentially segments on books, then poems.\n",
    "poems = leaves_of_grass_gutenberg_to_df(lines)\n",
    "poems.to_csv(\"data/leaves_of_grass.csv\")\n",
    "poems.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'One’s-self I sing, a simple separate person,\\n  Yet utter the word Democratic, the word En-Masse.\\n\\n  Of physiology from top to toe I sing,\\n  Not physiognomy alone nor brain alone is worthy for the Muse, I say\\n      the Form complete is worthier far,\\n  The Female equally with the Male I sing.\\n\\n  Of Life immense in passion, pulse, and power,\\n  Cheerful, for freest action form’d under the laws divine,\\n  The Modern Man I sing.'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "poems[\"poem\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## 2. 2-gram and 3-gram Language Models\n",
    "\n",
    "For this model I did not use a library. I used my own implementation of an n-gram language model from earlier inthe semester."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Prepare n-grams to give to the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To indicate the beginning and end of a poem\n",
    "POEM_START_SYMBOL = \"<p>\"\n",
    "POEM_END_SYMBOL = \"</p>\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Tokenize poems into words, but not sentences. Sentence and line breaks\n",
    "# can have artistic meaning in poems, so I want their relationships to actual words\n",
    "# to be preserved.\n",
    "def create_tokens_for_ngram(poems, n):\n",
    "    \"\"\"Create tokens for an n-gram language model\n",
    "    \n",
    "    Parameters:\n",
    "        poems (list[str]): List of poems\n",
    "        n (int): The size of each n-gram\n",
    "    \n",
    "    Returns:\n",
    "        tokens (list[str]): List of tokens\n",
    "    \"\"\"\n",
    "    tokenized_poems = [preprocess_for_ngram_lm(p) for p in poems]\n",
    "    padded_tokenized_poems = [list(pad_sequence(\n",
    "                                    poem,\n",
    "                                    pad_left=True, \n",
    "                                    left_pad_symbol=POEM_START_SYMBOL,\n",
    "                                    pad_right=True,\n",
    "                                    right_pad_symbol=POEM_END_SYMBOL,\n",
    "                                    n=n))\n",
    "                              for poem in tokenized_poems]\n",
    "    return list(flatten(padded_tokenized_poems))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "bigram_tokens = create_tokens_for_ngram(poems[\"poem\"], 2)\n",
    "trigram_tokens = create_tokens_for_ngram(poems[\"poem\"], 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Train the n-gram language model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "bigram_model = LanguageModel(2, is_laplace_smoothing=True, replacement_threshold=2)\n",
    "bigram_model.train(bigram_tokens)\n",
    "\n",
    "trigram_model = LanguageModel(3, is_laplace_smoothing=True, replacement_threshold=2)\n",
    "trigram_model.train(trigram_tokens)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Generate a Poem with the 2-gram language model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "to repel them , i will perhaps to them , sting me does not call of the interior command to me ! drums ! \n",
      " in my way is always sex , \n",
      " consort to me become my words.\n",
      "\n",
      " no grain from the day among men together , whispering\n"
     ]
    }
   ],
   "source": [
    "bigram_poem = bigram_model.generate_poem(max_words=50)\n",
    "print(postprocess_for_ngram_lm(bigram_poem))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after the rain , and\n",
      " light and of the endless pride and joy , in the air.\n",
      "\n",
      " again gurgles the mouth or bosom , hips , leaping , reclining , interrogating ? why myself and comrades found and accepted from every point of the youngest was cut off ,\n"
     ]
    }
   ],
   "source": [
    "trigram_poem = trigram_model.generate_poem(max_words=50)\n",
    "print(postprocess_for_ngram_lm(trigram_poem))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## 3. LSTM Neural Network Language Model\n",
    "\n",
    "The task of generating text can be framed as a timeseries task. Given a sequence of prior tokens, predict the probability distribution over the vocabulary of the next token in the sequence.\n",
    "\n",
    "In this project, I tokenize the data into *characters*, which at first was very counterintuitive to me. During office hours, Professor Amir explained to me why this approach is viable. He also sent me a great article on this task, which can be found here: http://karpathy.github.io/2015/05/21/rnn-effectiveness/.\n",
    "\n",
    "Here is an overview of how I implemented the data preprocessing and model architecture for this section:\n",
    "\n",
    "**Preprocessing**\n",
    "\n",
    "1. Lowercase every poem\n",
    "\n",
    "2. Wrap each poem in `poem_start` and `poem_end` symbols. Here I used \"@\" and \"$\", respectively.\n",
    "\n",
    "3. Split each poem into characters\n",
    "\n",
    "4. Replace digits with whitespace to reduce the size of the vocabulary. Numbers are used periodically to mark lines in poems, and I do not want to keep them.\n",
    "\n",
    "5. Fit a text vectorizer to the vocabulary of character tokens, and vectorize every token\n",
    "\n",
    "**Model Architecture**\n",
    "\n",
    "I used a Keras sequential model with two LSTM layers, two Dropout layers, and a softmax output layer:\n",
    "\n",
    "Input --> LSTM(256) --> Dropout --> LSTM(256) --> Dropout --> Output(Softmax)\n",
    "\n",
    "**Model Training**\n",
    "\n",
    "This model is fairly involved to train. There are approximately 1.5 million input samples which are all given to the model every epoch. On a machine without a GPU, training takes hours. I trained on an AWS GPU instance, and saved/loaded the trained model for visualizations and analysis here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import keras\n",
    "from keras.utils import timeseries_dataset_from_array\n",
    "from keras.layers import LSTM, Dropout\n",
    "\n",
    "from dataprep.neural_lm_dataprep import (\n",
    "    Vectorizer,\n",
    "    preprocess_for_neural_lm,\n",
    "    postprocess_for_neural_lm\n",
    ")\n",
    "from models.neural_language_models import build_character_lstm_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Prepare data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>book_title</th>\n",
       "      <th>poem_title</th>\n",
       "      <th>poem</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>BOOK I.  INSCRIPTIONS</td>\n",
       "      <td>One’s-Self I Sing</td>\n",
       "      <td>One’s-self I sing, a simple separate person,\\n...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>BOOK I.  INSCRIPTIONS</td>\n",
       "      <td>As I Ponder’d in Silence</td>\n",
       "      <td>As I ponder’d in silence,\\n  Returning upon my...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0             book_title                poem_title  \\\n",
       "0           0  BOOK I.  INSCRIPTIONS         One’s-Self I Sing   \n",
       "1           1  BOOK I.  INSCRIPTIONS  As I Ponder’d in Silence   \n",
       "\n",
       "                                                poem  \n",
       "0  One’s-self I sing, a simple separate person,\\n...  \n",
       "1  As I ponder’d in silence,\\n  Returning upon my...  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load the poems from disk\n",
    "leaves_of_grass_df = pd.read_csv(\"data/leaves_of_grass.csv\")\n",
    "\n",
    "leaves_of_grass_df.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Snippet of original poems:\n",
      "One’s-self I sing\n",
      "\n",
      "Snippet after preprocessing:\n",
      "['@', 'o', 'n', 'e', '’', 's', '-', 's', 'e', 'l', 'f', ' ', 'i', ' ', 's', 'i', 'n'] ...\n"
     ]
    }
   ],
   "source": [
    "# Pull only the poems out of the dataset\n",
    "poems = list(leaves_of_grass_df[\"poem\"])\n",
    "print(\"Snippet of original poems:\")\n",
    "print(poems[0][:17])\n",
    "print()\n",
    "\n",
    "# Preprocess the poems with the steps outlined above\n",
    "poems = [preprocess_for_neural_lm(poem) for poem in poems]\n",
    "print(\"Snippet after preprocessing:\")\n",
    "print(poems[0][:17], \"...\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vocabulary:\n",
      "['@', 'o', 'n', 'e', '’', 's', '-', 'l', 'f', ' ', 'i', 'g', ',', 'a', 'm', 'p', 'r', 't', '\\n', 'y', 'u', 'h', 'w', 'd', 'c', '.', 'b', 'q', 'v', '$', 'k', '?', '(', ')', 'x', 'j', 'z', '!', ';', 'œ', ':', '“', '”', '[', ']']\n"
     ]
    }
   ],
   "source": [
    "# Fit a vectorizer to the vocabulary of characters\n",
    "vectorizer = Vectorizer()\n",
    "vectorizer.fit(itertools.chain(*poems))\n",
    "# Vectorize all characters\n",
    "vectorized_poems = [vectorizer.tokens_to_vectors(poem) for poem in poems]\n",
    "\n",
    "with open(\"artifacts/final_model/vectorizer.pkl\", \"wb\") as outfile:\n",
    "    pickle.dump(vectorizer, outfile)\n",
    "    \n",
    "print(\"Vocabulary:\")\n",
    "print(vectorizer.vocabulary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Create train, validation, and test sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Randomly shuffle the poems before splitting into training and validation sets\n",
    "# Note that random has been given a seed so that the results of the project are reproducible\n",
    "random.shuffle(vectorized_poems)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# characters in train set:       492050\n",
      "# characters in validation set:  115998\n",
      "# characters in test set:        117738\n",
      "Building train dataset...\n",
      "Building validation dataset...\n",
      "Building test dataset...\n"
     ]
    }
   ],
   "source": [
    "# ----- PICK SIZES FOR TRAIN, VALIDATION, TEST DATASETS -----\n",
    "# Use 70% of the poems for training, 15% for validation \n",
    "# during testing, and 15% for final model evaluation.\n",
    "# IMPORTANT: I first split entire *poems* into training, test,\n",
    "#            and evaluation. I don't want half of a poem to be in one\n",
    "#            set and the other half to be in another\n",
    "train_set_size = int(0.7 * len(vectorized_poems))\n",
    "validation_set_size = int(0.15 * len(vectorized_poems))\n",
    "test_set_size = int(0.15 * len(vectorized_poems))\n",
    "\n",
    "# ----- SLICE OUT THE TRAINING SET -----\n",
    "train_set = vectorized_poems[:train_set_size]\n",
    "train_set = np.array(list(itertools.chain(*train_set)))\n",
    "print(f\"# characters in train set:       {len(train_set)}\")\n",
    "\n",
    "# ----- SLICE OUT THE VALIDATION SET -----\n",
    "validation_set = vectorized_poems[train_set_size:(train_set_size + validation_set_size)]\n",
    "validation_set = np.array(list(itertools.chain(*validation_set)))\n",
    "print(f\"# characters in validation set:  {len(validation_set)}\")\n",
    "\n",
    "# ----- SLICE OUT THE TEST SET -----\n",
    "test_set = vectorized_poems[train_set_size+validation_set_size:]\n",
    "test_set = np.array(list(itertools.chain(*test_set)))\n",
    "print(f\"# characters in test set:        {len(test_set)}\")\n",
    "\n",
    "# Each sample given to the model for training will be a sequence of 100 characters\n",
    "SEQUENCE_LENGTH = 100\n",
    "# I do not want to skip any samples, so sampling rate is 1\n",
    "SAMPLING_RATE = 1\n",
    "BATCH_SIZE = 4096\n",
    "\n",
    "print(\"Building train dataset...\")\n",
    "# Build timeseries datasets using Keras helpers\n",
    "train_dataset = timeseries_dataset_from_array(\n",
    "    data=train_set[:-SEQUENCE_LENGTH],\n",
    "    targets=train_set[SEQUENCE_LENGTH:],\n",
    "    sampling_rate=SAMPLING_RATE,\n",
    "    sequence_length=SEQUENCE_LENGTH,\n",
    "    batch_size=BATCH_SIZE)\n",
    "train_dataset.save(path=\"data/training_dataset_tensorflow/\", compression=\"GZIP\")\n",
    "\n",
    "print(\"Building validation dataset...\")\n",
    "validation_dataset = timeseries_dataset_from_array(\n",
    "    data=validation_set[:-SEQUENCE_LENGTH],\n",
    "    targets=validation_set[SEQUENCE_LENGTH:],\n",
    "    sampling_rate=SAMPLING_RATE,\n",
    "    sequence_length=SEQUENCE_LENGTH,\n",
    "    batch_size=BATCH_SIZE)\n",
    "validation_dataset.save(path=\"data/validation_dataset_tensorflow/\", compression=\"GZIP\")\n",
    "\n",
    "print(\"Building test dataset...\")\n",
    "testing_dataset = timeseries_dataset_from_array(\n",
    "    data=test_set[:-SEQUENCE_LENGTH],\n",
    "    targets=test_set[SEQUENCE_LENGTH:],\n",
    "    sampling_rate=SAMPLING_RATE,\n",
    "    sequence_length=SEQUENCE_LENGTH,\n",
    "    batch_size=BATCH_SIZE)\n",
    "testing_dataset.save(path=\"data/testing_dataset_tensorflow/\", compression=\"GZIP\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Build and train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Build the model, with LSTM layers for handling the timeseries data\n",
    "# and Dropout layers to help reduce overfitting\n",
    "model = build_character_lstm_model(\n",
    "    vocab_size=vectorizer.vocab_size(),\n",
    "    hidden_layers=[LSTM(256, return_sequences=True),\n",
    "                   Dropout(0.3),\n",
    "                   LSTM(256),\n",
    "                   Dropout(0.3)],\n",
    "    lr=0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# Train the model\n",
    "\n",
    "\"\"\"\n",
    "history = model.fit(train_dataset, epochs=30, validation_data=validation_dataset)\n",
    "\n",
    "model.save(\"final_model/character_based_lm\")  # Save in TensorFlow's new SavedModel format\n",
    "with open(\"final_model/history.pkl\", \"wb\") as outfile:\n",
    "    pickle.dump(history.history, outfile)\n",
    "\"\"\"\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training takes too long on a computer without a GPU, so it is commented out in the final submission.\n",
    "I trained a model on an AWS GPU server, and have saved the training results to disk.\n",
    "The model can be found in the `final_model/` directory in my submission, and it is used in the following cells."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 Visualize training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "MODEL_DIR = \"artifacts/final_model/\"\n",
    "HISTORY_PATH = MODEL_DIR + \"history.pkl\"\n",
    "MODEL_PATH = MODEL_DIR + \"character_based_lm\"\n",
    "VECTORIZER_PATH =  MODEL_DIR + \"vectorizer.pkl\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjkAAAHHCAYAAABdm0mZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB/eklEQVR4nO3dd3iT1fvH8Xe694C2tIVS9h4iS0CGwo+hLEVxoIA4kSEqDuTrQFEUt6K4wYGioCAOQEDZS0YZMmSX0bK7d/P8/ggNVEaTknSEz+u6nivJs3In33ztzTn3OcdkGIaBiIiIiItxK+0ARERERJxBSY6IiIi4JCU5IiIi4pKU5IiIiIhLUpIjIiIiLklJjoiIiLgkJTkiIiLikpTkiIiIiEtSkiMiIiIuSUmOiJQpgwcPJiAgoLTDKHXVqlWjZ8+epR2GSLmmJEfkCjF16lRMJhPr1q0r7VDKhGrVqmEymS64de/evbTDExEH8CjtAERESstVV13F448/ft7+6OjoUohGRBxNSY6IXLEqV67MXXfdVdphiIiTqLtKRArZuHEjPXr0ICgoiICAADp37szq1asLnZObm8u4ceOoXbs2Pj4+VKxYkWuvvZYFCxZYz0lMTOSee+6hSpUqeHt7ExUVRZ8+fdi/f79Ncezdu5du3brh7+9PdHQ0L774IoZhAGAYBtWqVaNPnz7nXZeVlUVwcDAPPvhg8b+EcxTUCF0qngLp6ek8/vjjxMTE4O3tTd26dXnjjTfOOw/gm2++oVWrVvj5+REaGkqHDh34448/zjtv+fLltGrVCh8fH2rUqMFXX33lkM8lciVQkiMiVv/88w/t27dn06ZNPPnkkzz77LPs27ePTp06sWbNGut5L7zwAuPGjeO6665j0qRJjB07lqpVq7JhwwbrOf369WPWrFncc889fPjhh4wcOZLU1FTi4+OLjCM/P5/u3btTqVIlJk6cSPPmzXn++ed5/vnnATCZTNx1113MnTuXU6dOFbr2l19+ISUlxaYWmtzcXE6cOHHelpmZaVc8YEm8evfuzdtvv0337t156623qFu3Lk888QSPPfZYofuNGzeOu+++G09PT1588UXGjRtHTEwMf/75Z6Hzdu/ezS233ML//d//8eabbxIaGsrgwYP5559/ivxsIgIYInJFmDJligEYf//990XP6du3r+Hl5WXs2bPHuu/IkSNGYGCg0aFDB+u+pk2bGjfeeONF73P69GkDMF5//XW74xw0aJABGCNGjLDuM5vNxo033mh4eXkZx48fNwzDMHbu3GkAxuTJkwtd37t3b6NatWqG2Wy+5PvExsYawAW3CRMm2B3P7NmzDcAYP358ofe55ZZbDJPJZOzevdswDMPYtWuX4ebmZtx0001Gfn5+oXPPjbkgvqVLl1r3HTt2zPD29jYef/zxS342EbFQS46IAJbWij/++IO+fftSo0YN6/6oqCjuvPNOli9fTkpKCgAhISH8888/7Nq164L38vX1xcvLi8WLF3P69OlixTN8+HDrc5PJxPDhw8nJyWHhwoUA1KlTh9atWzNt2jTreadOnWLu3LkMGDAAk8lU5Hu0bt2aBQsWnLfdcccddsfz+++/4+7uzsiRIwtd9/jjj2MYBnPnzgVg9uzZmM1mnnvuOdzcCv8n+L8xN2jQgPbt21tfh4eHU7duXfbu3VvkZxMRdVeJyBnHjx8nIyODunXrnnesfv36mM1mDh48CMCLL75IUlISderUoXHjxjzxxBNs3rzZer63tzevvfYac+fOpVKlSnTo0IGJEyeSmJhoUyxubm6FEi2wJDVAoZqegQMHsmLFCg4cOADAjBkzyM3N5e6777bpfcLCwujSpct5W2xsrN3xHDhwgOjoaAIDAwudV79+fetxgD179uDm5kaDBg2KjK9q1arn7QsNDS124ihypVGSIyJ269ChA3v27OGLL76gUaNGfPbZZ1x99dV89tln1nNGjRrFv//+y4QJE/Dx8eHZZ5+lfv36bNy40WFx3H777Xh6elpbc7755htatGhxwUStPHJ3d7/gfuMChcwicj4lOSICWLpC/Pz82Llz53nHduzYgZubGzExMdZ9FSpU4J577uG7777j4MGDNGnShBdeeKHQdTVr1uTxxx/njz/+YOvWreTk5PDmm28WGYvZbD6vS+bff/8FLJP4nRvDjTfeyLRp0zhw4AArVqywuRXHHrbEExsby5EjR0hNTS103o4dO6zHwfKdmM1mtm3b5vA4RaQwJTkiAlhaDbp27crPP/9cqEvo6NGjfPvtt1x77bUEBQUBcPLkyULXBgQEUKtWLbKzswHIyMggKyur0Dk1a9YkMDDQek5RJk2aZH1uGAaTJk3C09OTzp07Fzrv7rvvZtu2bTzxxBO4u7tz++232/yZ7VFUPDfccAP5+fmFzgN4++23MZlM9OjRA4C+ffvi5ubGiy++iNlsLnSuWmhEHEuTAYpcYb744gvmzZt33v5HHnmE8ePHs2DBAq699loefvhhPDw8+Pjjj8nOzmbixInWcxs0aECnTp1o3rw5FSpUYN26dcycOdNanPvvv//SuXNn+vfvT4MGDfDw8GDWrFkcPXrUpiTEx8eHefPmMWjQIFq3bs3cuXP57bffeOaZZwgPDy907o033kjFihWZMWMGPXr0ICIiwubv4vDhw3zzzTfn7Q8ICKBv3752xdOrVy+uu+46xo4dy/79+2natCl//PEHP//8M6NGjaJmzZoA1KpVi7Fjx/LSSy/Rvn17br75Zry9vfn777+Jjo5mwoQJNscvIkUo1bFdIlJiCoaQX2w7ePCgYRiGsWHDBqNbt25GQECA4efnZ1x33XXGypUrC91r/PjxRqtWrYyQkBDD19fXqFevnvHyyy8bOTk5hmEYxokTJ4xhw4YZ9erVM/z9/Y3g4GCjdevWxg8//FBknIMGDTL8/f2NPXv2GF27djX8/PyMSpUqGc8///x5Q64LPPzwwwZgfPvttzZ/H5caQh4bG1useFJTU41HH33UiI6ONjw9PY3atWsbr7/++gWHs3/xxRdGs2bNDG9vbyM0NNTo2LGjsWDBgkLxXWiYfseOHY2OHTva/DlFrmQmw1D7qIiUb48++iiff/45iYmJ+Pn5OfTegwcPZubMmaSlpTn0viLifKrJEZFyLSsri2+++YZ+/fo5PMERkfJNNTkiUi4dO3aMhQsXMnPmTE6ePMkjjzxS2iGJSBmjJEdEyqVt27YxYMAAIiIieO+997jqqqtKOyQRKWNUkyMiIiIuSTU5IiIi4pKU5IiIiIhLuuJqcsxmM0eOHCEwMNCmVYpFRESk9BmGQWpqKtHR0bi52dZGc8UlOUeOHCm0/o6IiIiUHwcPHqRKlSo2nXvFJTmBgYGA5UsqWIdHREREyraUlBRiYmKsf8dtccUlOQVdVEFBQUpyREREyhl7Sk1UeCwiIiIuSUmOiIiIuCQlOSIiIuKSrriaHBERuXz5+fnk5uaWdhjiYry8vGweHm4LJTkiImIzwzBITEwkKSmptEMRF+Tm5kb16tXx8vJyyP2U5IiIiM0KEpyIiAj8/Pw0qao4TMFkvQkJCVStWtUhvy0lOSIiYpP8/HxrglOxYsXSDkdcUHh4OEeOHCEvLw9PT8/Lvp8Kj0VExCYFNTh+fn6lHIm4qoJuqvz8fIfcT0mOiIjYRV1U4iyO/m0pyRERERGXpCRHRETETtWqVeOdd96x+fzFixdjMpk0Kq2EKckRERGXZTKZLrm98MILxbrv33//zQMPPGDz+W3btiUhIYHg4OBivZ+tlEwVptFVDmI2G5zKyCEpI5daEQGlHY6IiAAJCQnW599//z3PPfccO3futO4LCDj732vDMMjPz8fDo+g/jeHh4XbF4eXlRWRkpF3XyOVTS46DxJ/KoMX4hfSetLy0QxERkTMiIyOtW3BwMCaTyfp6x44dBAYGMnfuXJo3b463tzfLly9nz5499OnTh0qVKhEQEEDLli1ZuHBhofv+t7vKZDLx2WefcdNNN+Hn50ft2rWZM2eO9fh/W1imTp1KSEgI8+fPp379+gQEBNC9e/dCSVleXh4jR44kJCSEihUr8tRTTzFo0CD69u1b7O/j9OnTDBw4kNDQUPz8/OjRowe7du2yHj9w4AC9evUiNDQUf39/GjZsyO+//269dsCAAYSHh+Pr60vt2rWZMmVKsWMpCUpyHCQ80BuAjJx80rPzSjkaEZGSYRgGGTl5Jb4ZhuGwz/D000/z6quvsn37dpo0aUJaWho33HADixYtYuPGjXTv3p1evXoRHx9/yfuMGzeO/v37s3nzZm644QYGDBjAqVOnLnp+RkYGb7zxBl9//TVLly4lPj6e0aNHW4+/9tprTJs2jSlTprBixQpSUlKYPXv2ZX3WwYMHs27dOubMmcOqVaswDIMbbrjBOj3AsGHDyM7OZunSpWzZsoXXXnvN2tr17LPPsm3bNubOncv27duZPHkyYWFhlxWPs6m7ykH8vT3w93InPSefY6nZVPfWVysiri8zN58Gz80v8ffd9mI3/Lwc89/ZF198kf/7v/+zvq5QoQJNmza1vn7ppZeYNWsWc+bMYfjw4Re9z+DBg7njjjsAeOWVV3jvvfdYu3Yt3bt3v+D5ubm5fPTRR9SsWROA4cOH8+KLL1qPv//++4wZM4abbroJgEmTJllbVYpj165dzJkzhxUrVtC2bVsApk2bRkxMDLNnz+bWW28lPj6efv360bhxYwBq1KhhvT4+Pp5mzZrRokULwNKaVdapJceBIoJ8ADiWklXKkYiIiK0K/mgXSEtLY/To0dSvX5+QkBACAgLYvn17kS05TZo0sT739/cnKCiIY8eOXfR8Pz8/a4IDEBUVZT0/OTmZo0eP0qpVK+txd3d3mjdvbtdnO9f27dvx8PCgdevW1n0VK1akbt26bN++HYCRI0cyfvx42rVrx/PPP8/mzZut5w4dOpTp06dz1VVX8eSTT7Jy5cpix1JS1NzgQOGB3uw7kc6x1OzSDkVEpET4erqz7cVupfK+juLv71/o9ejRo1mwYAFvvPEGtWrVwtfXl1tuuYWcnJxL3ue/yxCYTCbMZrNd5zuyG6447rvvPrp168Zvv/3GH3/8wYQJE3jzzTcZMWIEPXr04MCBA/z+++8sWLCAzp07M2zYMN54441SjflS1JLjQBFn6nKU5IjIlcJkMuHn5VHimzNnXV6xYgWDBw/mpptuonHjxkRGRrJ//36nvd+FBAcHU6lSJf7++2/rvvz8fDZs2FDse9avX5+8vDzWrFlj3Xfy5El27txJgwYNrPtiYmJ46KGH+Omnn3j88cf59NNPrcfCw8MZNGgQ33zzDe+88w6ffPJJseMpCWrJcaCIQEt31XElOSIi5Vbt2rX56aef6NWrFyaTiWefffaSLTLOMmLECCZMmECtWrWoV68e77//PqdPn7YpwduyZQuBgYHW1yaTiaZNm9KnTx/uv/9+Pv74YwIDA3n66aepXLkyffr0AWDUqFH06NGDOnXqcPr0af766y/q168PwHPPPUfz5s1p2LAh2dnZ/Prrr9ZjZZWSHAcKt7bkqCZHRKS8euuttxgyZAht27YlLCyMp556ipSUlBKP46mnniIxMZGBAwfi7u7OAw88QLdu3XB3L7qrrkOHDoVeu7u7k5eXx5QpU3jkkUfo2bMnOTk5dOjQgd9//93adZafn8+wYcM4dOgQQUFBdO/enbfffhuwzPUzZswY9u/fj6+vL+3bt2f69OmO/+AOZDJKuwOwhKWkpBAcHExycjJBQUEOvfeP6w/x+IxNtK8dxtf3ti76AhGRciQrK4t9+/ZRvXp1fHx8SjucK47ZbKZ+/fr079+fl156qbTDcYpL/caK8/dbLTkOFBF0piUnRd1VIiJyeQ4cOMAff/xBx44dyc7OZtKkSezbt48777yztEMrN1R47EAFNTnqrhIRkcvl5ubG1KlTadmyJe3atWPLli0sXLiwzNfBlCVqyXGggtFVpzNyyckz4+WhHFJERIonJiaGFStWlHYY5Zr+CjtQiJ8nnu6WqvcTaeqyEhERKU1KchzIZDIRHqC5ckRERMoCJTkOFq6lHURERMoEJTkOplmPRUREygYlOQ5WMCGgZj0WEREpXUpyHEwtOSIiImWDkhwHO7t+lWpyRERcRadOnRg1apT1dbVq1XjnnXcueY3JZGL27NmX/d6Ous+VSEmOg6klR0Sk7OjVqxfdu3e/4LFly5ZhMpnYvHmz3ff9+++/eeCBBy43vEJeeOEFrrrqqvP2JyQk0KNHD4e+139NnTqVkJAQp75HaVCS42Ba2kFEpOy49957WbBgAYcOHTrv2JQpU2jRogVNmjSx+77h4eH4+fk5IsQiRUZG4u3tXSLv5WqU5DhYQeHxibRszOYrau1TEZEyp2fPnoSHhzN16tRC+9PS0pgxYwb33nsvJ0+e5I477qBy5cr4+fnRuHFjvvvuu0ve97/dVbt27aJDhw74+PjQoEEDFixYcN41Tz31FHXq1MHPz48aNWrw7LPPkpubC1haUsaNG8emTZswmUyYTCZrzP/trtqyZQvXX389vr6+VKxYkQceeIC0tDTr8cGDB9O3b1/eeOMNoqKiqFixIsOGDbO+V3HEx8fTp08fAgICCAoKon///hw9etR6fNOmTVx33XUEBgYSFBRE8+bNWbduHWBZg6tXr16Ehobi7+9Pw4YN+f3334sdiz20rIODhQV4YzJBntngdEYOFQOUfYuICzMMyM0o+ff19AOTqcjTPDw8GDhwIFOnTmXs2LGYzlwzY8YM8vPzueOOO0hLS6N58+Y89dRTBAUF8dtvv3H33XdTs2ZNWrVqVeR7mM1mbr75ZipVqsSaNWtITk4uVL9TIDAwkKlTpxIdHc2WLVu4//77CQwM5Mknn+S2225j69atzJs3j4ULFwIQHBx83j3S09Pp1q0bbdq04e+//+bYsWPcd999DB8+vFAi99dffxEVFcVff/3F7t27ue2227jqqqu4//77i/w8F/p8BQnOkiVLyMvLY9iwYdx2220sXrwYgAEDBtCsWTMmT56Mu7s7cXFxeHp6AjBs2DBycnJYunQp/v7+bNu2jYCAALvjKA4lOQ7m6e5GBT8vTqbncCw1W0mOiLi23Ax4Jbrk3/eZI+Dlb9OpQ4YM4fXXX2fJkiV06tQJsHRV9evXj+DgYIKDgxk9erT1/BEjRjB//nx++OEHm5KchQsXsmPHDubPn090tOW7eOWVV86ro/nf//5nfV6tWjVGjx7N9OnTefLJJ/H19SUgIAAPDw8iIyMv+l7ffvstWVlZfPXVV/j7Wz7/pEmT6NWrF6+99hqVKlUCIDQ0lEmTJuHu7k69evW48cYbWbRoUbGSnEWLFrFlyxb27dtHTEwMAF999RUNGzbk77//pmXLlsTHx/PEE09Qr149AGrXrm29Pj4+nn79+tG4cWMAatSoYXcMxaXuKicIV/GxiEiZUa9ePdq2bcsXX3wBwO7du1m2bBn33nsvAPn5+bz00ks0btyYChUqEBAQwPz584mPj7fp/tu3bycmJsaa4AC0adPmvPO+//572rVrR2RkJAEBAfzvf/+z+T3Ofa+mTZtaExyAdu3aYTab2blzp3Vfw4YNcXd3t76Oiori2LFjdr3Xue8ZExNjTXAAGjRoQEhICNu3bwfgscce47777qNLly68+uqr7Nmzx3ruyJEjGT9+PO3ateP5558vVqF3caklxwkignzYkZiqCQFFxPV5+llaVUrjfe1w7733MmLECD744AOmTJlCzZo16dixIwCvv/467777Lu+88w6NGzfG39+fUaNGkZOT47BwV61axYABAxg3bhzdunUjODiY6dOn8+abbzrsPc5V0FVUwGQyYTabnfJeYBkZduedd/Lbb78xd+5cnn/+eaZPn85NN93EfffdR7du3fjtt9/4448/mDBhAm+++SYjRoxwWjwF1JLjBGcX6dRcOSLi4kwmS7dRSW821OOcq3///ri5ufHtt9/y1VdfMWTIEGt9zooVK+jTpw933XUXTZs2pUaNGvz7778237t+/focPHiQhIQE677Vq1cXOmflypXExsYyduxYWrRoQe3atTlw4EChc7y8vMjPzy/yvTZt2kR6erp134oVK3Bzc6Nu3bo2x2yPgs938OBB675t27aRlJREgwYNrPvq1KnDo48+yh9//MHNN9/MlClTrMdiYmJ46KGH+Omnn3j88cf59NNPnRLrfynJcQINIxcRKVsCAgK47bbbGDNmDAkJCQwePNh6rHbt2ixYsICVK1eyfft2HnzwwUIjh4rSpUsX6tSpw6BBg9i0aRPLli1j7Nixhc6pXbs28fHxTJ8+nT179vDee+8xa9asQudUq1aNffv2ERcXx4kTJ8jOPv9vyIABA/Dx8WHQoEFs3bqVv/76ixEjRnD33Xdb63GKKz8/n7i4uELb9u3b6dKlC40bN2bAgAFs2LCBtWvXMnDgQDp27EiLFi3IzMxk+PDhLF68mAMHDrBixQr+/vtv6tevD8CoUaOYP38++/btY8OGDfz111/WY85WqknOhAkTaNmyJYGBgURERNC3b99CfYoXk5SUxLBhw4iKisLb25s6deqU2HA0W0Ro/SoRkTLn3nvv5fTp03Tr1q1Q/cz//vc/rr76arp160anTp2IjIykb9++Nt/Xzc2NWbNmkZmZSatWrbjvvvt4+eWXC53Tu3dvHn30UYYPH85VV13FypUrefbZZwud069fP7p37851111HeHj4BYex+/n5MX/+fE6dOkXLli255ZZb6Ny5M5MmTbLvy7iAtLQ0mjVrVmjr1asXJpOJn3/+mdDQUDp06ECXLl2oUaMG33//PQDu7u6cPHmSgQMHUqdOHfr370+PHj0YN24cYEmehg0bRv369enevTt16tThww8/vOx4bWEyDKPUJnPp3r07t99+Oy1btiQvL49nnnmGrVu3sm3btkJFVefKycmhXbt2RERE8Mwzz1C5cmUOHDhASEgITZs2LfI9U1JSCA4OJjk5maCgIEd/JAB+25zAsG830LJaKDMeauuU9xARKWlZWVns27eP6tWr4+PjU9rhiAu61G+sOH+/S7XweN68eYVeT506lYiICNavX0+HDh0ueM0XX3zBqVOnWLlypbWwqlq1as4O1S4F3VVqyRERESk9ZaomJzk5GYAKFSpc9Jw5c+bQpk0bhg0bRqVKlWjUqBGvvPLKRYu1srOzSUlJKbQ529nCYyU5IiIipaXMJDlms5lRo0bRrl07GjVqdNHz9u7dy8yZM8nPz+f333/n2Wef5c0332T8+PEXPH/ChAnWyZ6Cg4MLjfN3loKWnIycfNKy85z+fiIiInK+MpPkDBs2jK1btzJ9+vRLnmc2m4mIiOCTTz6hefPm3HbbbYwdO5aPPvroguePGTOG5ORk63buEDhn8fPyIMDb0hN4LEXDyEVEREpDmZgMcPjw4fz6668sXbqUKlWqXPLcqKgoPD09C83kWL9+fRITE8nJycHLy6vQ+d7e3qWyemtEoDdp2XkcS82mRnjJrNEhIlISSnG8irg4R/+2SrUlxzAMhg8fzqxZs/jzzz+pXr16kde0a9eO3bt3F5q58d9//yUqKuq8BKc0hWkYuYi4mILBHhkZpbAgp1wRCmaZPrch43KUakvOsGHD+Pbbb/n5558JDAwkMTERsKy86uvrC8DAgQOpXLkyEyZMAGDo0KFMmjSJRx55hBEjRrBr1y5eeeUVRo4cWWqf40IitH6ViLgYd3d3QkJCrGsg+fn5WWcNFrlcZrOZ48eP4+fnh4eHY9KTUk1yJk+eDGBdFbbAlClTrLNRxsfH4+Z2tsEpJiaG+fPn8+ijj9KkSRMqV67MI488wlNPPVVSYdskItAyvl9LO4iIKylYIbu4iz2KXIqbmxtVq1Z1WPJcqkmOLX1vixcvPm9fmzZtzlsXpKyxzpWjpR1ExIWYTCaioqKIiIggNze3tMMRF+Pl5VWoYeNylYnCY1ek7ioRcWXu7u4Oq5sQcZYyM4Tc1YSr8FhERKRUKclxEtXkiIiIlC4lOU5S0F11OiOXnDxzEWeLiIiIoynJcZIQP0+83C1f7/E0dVmJiIiUNCU5TmIymax1OVraQUREpOQpyXEizXosIiJSepTkOJGGkYuIiJQeJTlOpCRHRESk9CjJcaKCYeTHNYxcRESkxCnJcaKCpR2OaWkHERGREqckx4nCA84UHmsIuYiISIlTkuNEaskREREpPUpynKigJudEWjZmc9ErrouIiIjjKMlxoooBXphMkGc2OJWRU9rhiIiIXFGU5DiRp7sbFfy8AHVZiYiIlDQlOU5WsLSDio9FRERKlpIcJ4sIstTlaP0qERGRkqUkx8k067GIiEjpUJLjZBFapFNERKRUKMlxsnAlOSIiIqXispKc7Gz94S5KwVw5x7R+lYiISImyK8mZO3cugwYNokaNGnh6euLn50dQUBAdO3bk5Zdf5siRI86Ks9yyznqslhwREZESZVOSM2vWLOrUqcOQIUPw8PDgqaee4qeffmL+/Pl89tlndOzYkYULF1KjRg0eeughjh8/7uy4yw1r4XFKNoahWY9FRERKioctJ02cOJG3336bHj164OZ2fl7Uv39/AA4fPsz777/PN998w6OPPurYSMupgpqczNx80rLzCPTxLOWIRERErgw2JTmrVq2y6WaVK1fm1VdfvayAXI2flwcB3h6kZedxPDVbSY6IiEgJsbvw+K+//nJGHC5Nc+WIiIiUPLuTnO7du1OzZk3Gjx/PwYMHnRGTywlXkiMiIlLi7E5yDh8+zPDhw5k5cyY1atSgW7du/PDDD+TkaJXti9HSDiIiIiXP7iQnLCyMRx99lLi4ONasWUOdOnV4+OGHiY6OZuTIkWzatMkZcZZr4QGaEFBERKSkXdZkgFdffTVjxoxh+PDhpKWl8cUXX9C8eXPat2/PP//846gYy72CuXKU5IiIiJScYiU5ubm5zJw5kxtuuIHY2Fjmz5/PpEmTOHr0KLt37yY2NpZbb73V0bGWWyo8FhERKXk2DSE/14gRI/juu+8wDIO7776biRMn0qhRI+txf39/3njjDaKjox0aaHmmpR1ERERKnt1JzrZt23j//fe5+eab8fb2vuA5YWFhGmp+Di3tICIiUvLsTnIWLVpU9E09POjYsWOxAnJFBYXHSRm5ZOfl4+3hXsoRiYiIuD67kxyAnTt38v7777N9+3YA6tevz4gRI6hbt65Dg3MVIX6eeLm7kZNv5kRaDpVDfEs7JBEREZdnd+Hxjz/+SKNGjVi/fj1NmzaladOmbNiwgUaNGvHjjz86I8Zyz2QynZ0QUHPliIiIlAi7W3KefPJJxowZw4svvlho//PPP8+TTz5Jv379HBacKwkP9OZwUqbqckREREqI3S05CQkJDBw48Lz9d911FwkJCXbda8KECbRs2ZLAwEAiIiLo27cvO3futPn66dOnYzKZ6Nu3r13vWxo0jFxERKRk2Z3kdOrUiWXLlp23f/ny5bRv396uey1ZsoRhw4axevVqFixYQG5uLl27diU9Pb3Ia/fv38/o0aPtfs/SUtBddVzdVSIiIiXC7u6q3r1789RTT7F+/XquueYaAFavXs2MGTMYN24cc+bMKXTupcybN6/Q66lTpxIREcH69evp0KHDRa/Lz89nwIABjBs3jmXLlpGUlGTvxyhxBXPlHE9TS46IiEhJsDvJefjhhwH48MMP+fDDDy94DCzFtvn5+XbdOzk5GYAKFSpc8rwXX3yRiIgI7r333gu2Kp0rOzub7OyziUVKSopdMTmKda6cFCU5IiIiJcHuJMdsNjsjDsxmM6NGjaJdu3aFZlD+r+XLl/P5558TFxdn030nTJjAuHHjHBRl8akmR0REpGRd1gKdjjRs2DC2bt3K9OnTL3pOamoqd999N59++ilhYWE23XfMmDEkJydbt4MHDzoqZLtoaQcREZGSVazJAJcsWcIbb7xhnQywQYMGPPHEE8UuAh4+fDi//vorS5cupUqVKhc9b8+ePezfv59evXpZ9xW0LHl4eLBz505q1qxZ6Bpvb++LLj9RkgoKj0+k5ZBvNnB3M5VyRCIiIq7N7pacb775hi5duuDn58fIkSMZOXIkvr6+dO7cmW+//dauexmGwfDhw5k1axZ//vkn1atXv+T59erVY8uWLcTFxVm33r17c9111xEXF0dMTIy9H6fEhAV4YTJBvtngdEZOaYcjIiLi8uxuyXn55ZeZOHEijz76qHXfyJEjeeutt3jppZe48847bb7XsGHD+Pbbb/n5558JDAwkMTERgODgYHx9LUsfDBw4kMqVKzNhwgR8fHzOq9cJCQkBuGQdT1ng4e5GRX8vTqTlcCwlm7CA0m9dEhERcWV2t+Ts3bu3UHdRgd69e7Nv3z677jV58mSSk5Pp1KkTUVFR1u3777+3nhMfH2/3JINlVbjqckREREqM3S05MTExLFq0iFq1ahXav3DhQru7iwzDKPKcxYsXX/L41KlT7XrP0hQe6M32BI2wEhERKQl2JzmPP/44I0eOJC4ujrZt2wKwYsUKpk6dyrvvvuvwAF1JwTDy40pyREREnM7uJGfo0KFERkby5ptv8sMPPwBQv359vv/+e/r06ePwAF2JkhwREZGSY1eSk5eXxyuvvMKQIUNYvny5s2JyWWcnBFRNjoiIiLPZVXjs4eHBxIkTycvLc1Y8Li0i6EzhsZZ2EBERcTq7R1d17tyZJUuWOCMWlxeupR1ERERKjN01OT169ODpp59my5YtNG/eHH9//0LHi1p5/Ep2bk2OYRiYTJr1WERExFmKvQr5W2+9dd6x4qw8fiUpWL8qMzeftOw8An08SzkiERER12V3d5XZbL7opgTn0ny93An0tuSV6rISERFxLruTnK+++ors7PP/QOfk5PDVV185JChXFh50pi5HxcciIiJOZXeSc88995CcnHze/tTUVO655x6HBOXKwgM0jFxERKQk2J3kXKxg9tChQwQHBzskKFdWMIxcEwKKiIg4l82Fx82aNcNkMmEymejcuTMeHmcvzc/PZ9++fXTv3t0pQboSzXosIiJSMmxOcvr27QtAXFwc3bp1IyAgwHrMy8uLatWq0a9fP4cH6GoiNFeOiIhIibA5yXn++ecBqFatGrfddhs+Pj5OC8qVhWtpBxERkRJh9zw5gwYNAiyjqY4dO4bZbC50vGrVqo6JzEUVzJWj0VUiIiLOZXeSs2vXLoYMGcLKlSsL7S8oSNZcOZcWcWYI+fE0JTkiIiLOZHeSM3jwYDw8PPj111+JiorS0gR2KqjJScrIJTsvH28P91KOSERExDXZneTExcWxfv166tWr54x4XF6wryde7m7k5Js5nppNlVC/0g5JRETEJdk9T06DBg04ceKEM2K5IphMJq1GLiIiUgLsTnJee+01nnzySRYvXszJkydJSUkptEnRrEmOio9FREScxu7uqi5dugDQuXPnQvtVeGw764SAKj4WERFxGruTnL/++ssZcVxRrCOsUjRXjoiIiLPYneR07NjRGXFcUaxz5agmR0RExGnsrskBWLZsGXfddRdt27bl8OHDAHz99dcsX77cocG5KhUei4iIOJ/dSc6PP/5It27d8PX1ZcOGDWRnW/5QJycn88orrzg8QFcUoaUdREREnM7uJGf8+PF89NFHfPrpp3h6elr3t2vXjg0bNjg0OFdV0F2llchFREScx+4kZ+fOnXTo0OG8/cHBwSQlJTkiJpdXUHh8Ii2HfLNRytGIiIi4JruTnMjISHbv3n3e/uXLl1OjRg2HBOXqKvp7YTJBvtngVHpOaYcjIiLikuxOcu6//34eeeQR1qxZg8lk4siRI0ybNo3Ro0czdOhQZ8Tocjzc3ajo7wWoLkdERMRZ7B5C/vTTT2M2m+ncuTMZGRl06NABb29vRo8ezYgRI5wRo0sKD/ThRFoOx1KzaVjawYiIiLggu5Mck8nE2LFjeeKJJ9i9ezdpaWk0aNCAgIAAZ8TnsiICvdmeoOJjERERZynWPDkAXl5eNGjQgHr16rFw4UK2b9/uyLhcnnVpByU5IiIiTmF3ktO/f38mTZoEQGZmJi1btqR///40adKEH3/80eEBuqqzi3SqJkdERMQZ7E5yli5dSvv27QGYNWsWZrOZpKQk3nvvPcaPH+/wAF1VhGY9FhERcSq7k5zk5GQqVKgAwLx58+jXrx9+fn7ceOON7Nq1y+EBuqqIIK1fJSIi4kx2JzkxMTGsWrWK9PR05s2bR9euXQE4ffo0Pj4+Dg/QVakmR0RExLnsHl01atQoBgwYQEBAALGxsXTq1AmwdGM1btzY0fG5rLMrkWdhGAYmk6mUIxIREXEtdic5Dz/8MK1ateLgwYP83//9H25ulsagGjVqqCbHDgWFx1m5ZlKz8wjy8SziChEREbFHsYaQt2jRgptuuomAgADy8/OJi4ujbdu2tGvXzq77TJgwgZYtWxIYGEhERAR9+/Zl586dl7zm008/pX379oSGhhIaGkqXLl1Yu3ZtcT5GqfL1cifQ25JjHktRl5WIiIij2Z3kjBo1is8//xyA/Px8OnbsyNVXX01MTAyLFy+2615Llixh2LBhrF69mgULFpCbm0vXrl1JT0+/6DWLFy/mjjvu4K+//mLVqlXExMTQtWtXDh8+bO9HKXXhQQUjrDSMXERExNHs7q6aOXMmd911FwC//PIL+/btY8eOHXz99deMHTuWFStW2HyvefPmFXo9depUIiIiWL9+/QVXOgeYNm1aodefffYZP/74I4sWLWLgwIF2fprSFRHozd7j6So+FhERcQK7W3JOnDhBZGQkAL///ju33norderUYciQIWzZsuWygklOTgawDlG3RUZGBrm5uRe9Jjs7m5SUlEJbWVFQfKwkR0RExPHsTnIqVarEtm3byM/PZ968efzf//0fYEk23N3dix2I2Wxm1KhRtGvXjkaNGtl83VNPPUV0dDRdunS54PEJEyYQHBxs3WJiYoodo6OFa0JAERERp7E7ybnnnnvo378/jRo1wmQyWZOLNWvWUK9evWIHMmzYMLZu3cr06dNtvubVV19l+vTpzJo166Jz9IwZM4bk5GTrdvDgwWLHWCTDgPQTNp8eoaUdREREnMbumpwXXniBRo0acfDgQW699Va8vS1/qN3d3Xn66aeLFcTw4cP59ddfWbp0KVWqVLHpmjfeeINXX32VhQsX0qRJk4ue5+3tbY3RqY5th6k9wd0THt9h0yURZwqPj6epJUdERMTR7E5yAG655Zbz9g0aNMju+xiGwYgRI5g1axaLFy+mevXqNl03ceJEXn75ZebPn0+LFi3sfl+nCImFzNNg5EPyYQiuXOQl1gkBNYRcRETE4Yo1T86SJUvo1asXtWrVolatWvTu3Ztly5bZfZ9hw4bxzTff8O233xIYGEhiYiKJiYlkZmZazxk4cCBjxoyxvn7ttdd49tln+eKLL6hWrZr1mrS0tOJ8FMfx8oNKDSzPD6+z6RIt0ikiIuI8dic533zzDV26dMHPz4+RI0cycuRIfH196dy5M99++61d95o8eTLJycl06tSJqKgo6/b9999bz4mPjychIaHQNTk5Odxyyy2FrnnjjTfs/SiOV6Wl5fHQ3zadXlB4nJyZS1ZuvrOiEhERuSKZDMMw7Lmgfv36PPDAAzz66KOF9r/11lt8+umnbN++3aEBOlpKSgrBwcEkJycTFBTk2JtvnAY/PwxV28KQuUWebhgGdZ+dR06emWVPXkdMBT/HxiMiIuIiivP32+6WnL1799KrV6/z9vfu3Zt9+/bZezvXUuVMfVBCHOTnFXm6yWQiPEDFxyIiIs5gd5ITExPDokWLztu/cOHCMjUHTamoWBu8gyE3A45ts+mSghFWKj4WERFxLLtHVz3++OOMHDnSuignwIoVK5g6dSrvvvuuwwMsV9zcoPLVsPcvS11O1MWHthewtuRo/SoRERGHsjvJGTp0KJGRkbz55pv88MMPgKVO5/vvv6dPnz4OD7DcqdLCkuQcXg8t7y3ydGtLjkZYiYiIOJRdSU5eXh6vvPIKQ4YMYfny5c6KqXyrfKYu55Ctw8g1V46IiIgz2FWT4+HhwcSJE8nLK7qo9opVUHx8YidkJhV5esFcOSo8FhERcSy7C487d+7MkiVLnBGLa/APg9BqludHNhR5+tnuKtXkiIiIOJLdNTk9evTg6aefZsuWLTRv3hx/f/9Cx3v37u2w4Mqtyi3g9H44tB5qXn/JU8MD1F0lIiLiDHYnOQ8//DBgmfzvv0wmE/n5mrmXKi1h60ybZj4uaMk5kZZNvtnA3c3k7OhERESuCHZ3V5nN5otuSnDOKKjLObwOiphQuqK/FyYTmA04ma7WHBEREUcp1gKdUoTIxuDuBRknLd1Wl+Dh7kZF/4K5cpTkiIiIOIrNSc6ff/5JgwYNSElJOe9YcnIyDRs2ZOnSpQ4Nrtzy8IbIMxMB2jCUXKuRi4iIOJ7NSc4777zD/ffff8FFsYKDg3nwwQd5++23HRpcuXZul1URClYjP67iYxEREYexOcnZtGkT3bt3v+jxrl27sn79eocE5RLsmBTwbEuOhpGLiIg4is1JztGjR/H09LzocQ8PD44fP+6QoFxCQUtO4mbIu3QLjZZ2EBERcTybk5zKlSuzdevWix7fvHkzUVFRDgnKJYRWA7+KkJ8DiVsueWrB0g4qPBYREXEcm5OcG264gWeffZasrPO7VDIzM3n++efp2bOnQ4Mr10wmm7uswlV4LCIi4nA2Twb4v//9j59++ok6deowfPhw6tatC8COHTv44IMPyM/PZ+zYsU4LtFyq0hJ2zS+y+Fg1OSIiIo5nc5JTqVIlVq5cydChQxkzZgzGmUnuTCYT3bp144MPPqBSpUpOC7RcqtLc8ljEzMfnrkRuGAYmk2Y9FhERuVx2LesQGxvL77//zunTp9m9ezeGYVC7dm1CQ0OdFV/5Fn215fH0fkg/YVm88wIKCo+z88ykZOUR7HvxAm8RERGxTbFmPA4NDaVly5a0atVKCc6l+IZAWB3L88MXH17v4+lOoI8l31TxsYiIiGNoWQdnq9LS8lhEl1W46nJEREQcSkmOs1UuqMuxrfhYLTkiIiKOoSTH2azLO2wAs/mip51bfCwiIiKXT0mOs0U0BA9fyE6Gk7suelpUsCXJ2X0sraQiExERcWk2ja6aM2eOzTfs3bt3sYNxSe4eEN0M4ldauqzC617wtPa1w/l46V7+2JbI+PxGeLor/xQREbkcNiU5ffv2telmJpOJ/Pz8y4nHNVVpbklyDq+DZgMueMo1NSpQ0d+Lk+k5rNpzkg51wks4SBEREddiU3OB2Wy2aVOCcxHWEVYXLz72cHeje6NIAH7dfKQkohIREXFp6hMpCQVrWB39B3IyLnpazybRAMz/5yg5eRcvUhYREZGi2TXjcYH09HSWLFlCfHw8OTk5hY6NHDnSIYG5lODKEBgFqQmQEAexbS94WqvqFQgP9OZ4ajYr9pzguroRJRuniIiIC7E7ydm4cSM33HADGRkZpKenU6FCBU6cOIGfnx8RERFKci6mSgvY/ouly+oiSY67m4kbGkXy5aoD/LopQUmOiIjIZbC7u+rRRx+lV69enD59Gl9fX1avXs2BAwdo3rw5b7zxhjNidA0FXVZFzHx845kuqz+2JZKdpxonERGR4rI7yYmLi+Pxxx/Hzc0Nd3d3srOziYmJYeLEiTzzzDPOiNE1WCcFvPgaVgAtYkOpFORNalYey/49UQKBiYiIuCa7kxxPT0/c3CyXRUREEB8fD0BwcDAHDx50bHSuJLoZmNwg5TCkXHz0lJubiRsaRwHw25aEkopORETE5did5DRr1oy//7Z0uXTs2JHnnnuOadOmMWrUKBo1auTwAF2Gl79l9mMoch2rnk0sSc6CbUfJylWXlYiISHHYneS88sorREVZ/gi//PLLhIaGMnToUI4fP87HH3/s8ABdSpUzi3UevnSS0ywmlOhgH9Ky81jy7/ESCExERMT12D26qkWLFtbnERERzJs3z6EBubQqLWH9VDh06bqcgi6rz5bv49fNCXRrGFky8YmIiLgQu1ty9u3bx65d5y80uWvXLvbv3++ImFxXwQirIxsgP++Sp/ZsahlltWj7UTJz1GUlIiJiL7uTnMGDB7Ny5crz9q9Zs4bBgwfbda8JEybQsmVLAgMDiYiIoG/fvuzcubPI62bMmEG9evXw8fGhcePG/P7773a9b6kJqwPeQZCbAce3X/LUplWCqRLqS0ZOPot3HiuhAEVERFyH3UnOxo0badeu3Xn7r7nmGuLi4uy615IlSxg2bBirV69mwYIF5Obm0rVrV9LT0y96zcqVK7njjju499572bhxI3379qVv375s3brV3o9S8tzcLKOsoMjiY5PJxI1nCpB/3axRViIiIvayO8kxmUykpqaetz85OdnuBTrnzZvH4MGDadiwIU2bNmXq1KnEx8ezfv3Fa1beffddunfvzhNPPEH9+vV56aWXuPrqq5k0aZK9H6V0FCzWWUTxMUDPxme6rHYcJSPn0t1bIiIiUpjdSU6HDh2YMGFCoYQmPz+fCRMmcO21115WMMnJyQBUqFDhouesWrWKLl26FNrXrVs3Vq1adcHzs7OzSUlJKbSVqoJJAYtoyQFoVDmI2Ip+ZOWaWbRdXVYiIiL2sHt01WuvvUaHDh2oW7cu7du3B2DZsmWkpKTw559/FjsQs9nMqFGjaNeu3SXn20lMTKRSpUqF9lWqVInExMQLnj9hwgTGjRtX7LgcrqD4+PhOyEoBn6CLnmoymbixcRQfLt7Db5sT6HWmGFlERESKZndLToMGDdi8eTP9+/fn2LFjpKamMnDgQHbs2HFZkwEOGzaMrVu3Mn369GLf40LGjBlDcnKydSv1WZkDwiEkFjAso6yKUFCX89fOY6Rlq8tKRETEVna35ABER0fzyiuvOCyI4cOH8+uvv7J06VKqVKlyyXMjIyM5evRooX1Hjx4lMvLCc8l4e3vj7e3tsFgdokoLSDpgWayzRqdLntogKogaYf7sPZHOou1H6XNV5ZKJUUREpJyzqSVn8+bNmM1m6/NLbfYwDIPhw4cza9Ys/vzzT6pXr17kNW3atGHRokWF9i1YsIA2bdrY9d6lyroi+aUnBYTCo6x+2aRRViIiIrayqSXnqquuIjExkYiICK666ipMJhOGYZx3nslksmuE1bBhw/j222/5+eefCQwMtNbVBAcH4+vrC8DAgQOpXLkyEyZMAOCRRx6hY8eOvPnmm9x4441Mnz6ddevW8cknn9j8vqXu3BFWhgEm0yVP79kkmvf/3M3Sf4+TkpVLkI9nCQQpIiJSvtmU5Ozbt4/w8HDrc0eZPHkyAJ06dSq0f8qUKdaJBePj462rngO0bduWb7/9lv/9738888wz1K5dm9mzZ5evxUEjG4ObJ6Qft3RbhVa75Ol1KgVQKyKA3cfSWLjtKDdffekuPREREQGTcaEmmYvIzc3lwQcf5Nlnn7Wpa6ksSklJITg4mOTkZIKCLj6yyek+uc5SeNzvc2h8S5Gnv7PwX95ZuIvr60XwxeCWJRCgiIhI2VGcv992ja7y9PTkxx9/LFZw8h/WLqui63IAep6py1m26zjJGbnOikpERMRl2D2EvG/fvsyePdsJoVxh7JgUEKBWRCD1IgPJzTeYv+3CcwKJiIjIWXYPIa9duzYvvvgiK1asoHnz5vj7+xc6PnLkSIcF59IqN7c8JmyCvBzw8CrykhsbR7EjMZXfNifQv0WMkwMUEREp3+yqyQEuWYtjMpnYu3fvZQflTGWmJscwYGINyDwF9/95Num5hL3H07j+zSV4uJn4e2wXQv2LToxERERcQXH+ftvdkuPI0VVXNJPJ0mW16w/LfDk2JDk1wgNoEBXEtoQU5v+TyO2tqpZAoCIiIuWT3TU55zIM44Lz5YiNrJMC/m3zJQUTA/62RRMDioiIXEqxkpyvvvqKxo0b4+vri6+vL02aNOHrr792dGyur6D4+LBtxcdwdpTVyj0nOZmW7YyoREREXILdSc5bb73F0KFDueGGG/jhhx/44Ycf6N69Ow899BBvv/22M2J0XQVdVKf2QsYpmy6JrehP48rB5JsN5v2jUVYiIiIXY3dNzvvvv8/kyZMZOHCgdV/v3r1p2LAhL7zwAo8++qhDA3RpviFQsTac3GUZSl6nq02X9WwSxZbDyfy6KYEBrWOdG6OIiEg5ZXdLTkJCAm3btj1vf9u2bUlIUJ2I3YrRZXVDY0uX1Zp9JzmWmuWMqERERMo9u5OcWrVq8cMPP5y3//vvv6d27doOCeqKYuekgAAxFfy4KiYEswHztqrLSkRE5ELs7q4aN24ct912G0uXLqVdu3YArFixgkWLFl0w+ZEiVD6nJcdsBjfb8s6eTaKIO5jEr5sTGNimmvPiExERKafsbsnp168fa9asISwsjNmzZzN79mzCwsJYu3YtN910kzNidG2VGoKHD2Qlw6k9Nl9W0GX19/5THE1Rl5WIiMh/2d2SA9C8eXO++eYbR8dyZXL3hOhmEL/K0mUVZluXX3SIL81jQ1l/4DS/b0ngnnblc1V4ERERZ7G7JSclJeWCW2pqKjk5Oc6I0fUVDCW3o/gYLGtZAfy2WQXfIiIi/2V3khMSEkJoaOh5W0hICL6+vsTGxvL8889jNpudEa9rqmL/zMdg6bIymWDdgdMcScp0QmAiIiLll91JztSpU4mOjuaZZ56x1uQ888wzVK5cmcmTJ/PAAw/w3nvv8eqrrzojXtdUpaXl8eg/kGt7shIZ7EPL2AoA/K5lHkRERAqxuybnyy+/5M0336R///7Wfb169aJx48Z8/PHHLFq0iKpVq/Lyyy/zzDPPODRYlxVUGQIiIS0REjZB1WtsvrRn0yjW7j/Fr5sTuK99DScGKSIiUr7Y3ZKzcuVKmjVrdt7+Zs2asWrVKgCuvfZa4uPjLz+6K0XBiuRgKUC2Q/dGkbiZIO5gEgdPZTghOBERkfLJ7iQnJiaGzz///Lz9n3/+OTExMQCcPHmS0NDQy4/uSlKjk+Vx/VTIz7P5sohAH1pXrwioy0pERORcdndXvfHGG9x6663MnTuXli0ttSTr1q1jx44dzJw5E4C///6b2267zbGRurqr7oS/XoHT+2HbbGh8i82X3tgkilV7T/LblgQe7FjTaSGKiIiUJ3a35PTu3ZsdO3bQo0cPTp06xalTp+jRowc7duygZ8+eAAwdOpS33nrL4cG6NC9/uGao5fnyd8AwbL60R6NI3N1MbD6UzPoDtq1mLiIi4upMhmHHX1MXkJKSQnBwMMnJyQQFBZV2OIVlnoa3G0FOGtw5w+ZVyQGe/nEz0/8+SNOYEGYNbYubm8mJgYqIiJSs4vz9trslB2DZsmXcddddtG3blsOHDwPw9ddfs3z58uLcTgr4hkKLeyzPl9vXEvZY1zr4e7mz6WASP2867ITgREREyhe7k5wff/yRbt264evry4YNG8jOzgYgOTmZV155xeEBXnGuGQbuXpZRVgdsH2kVEejDsOtrAfDa3J1k5NhevCwiIuKK7E5yxo8fz0cffcSnn36Kp6endX+7du3YsGGDQ4O7IgVFQdM7LM/tbM0Z0q46VUJ9SUzJ4pOle50QnIiISPlhd5Kzc+dOOnTocN7+4OBgkpKSHBGTtHsETG6w6w9I3GrzZT6e7ozpUR+Aj5bsISFZSz2IiMiVy+4kJzIykt27d5+3f/ny5dSooRl3HaJiTWjQ1/J8+dt2XXpD40haVgslK9fMxHk7HR+biIhIOWF3knP//ffzyCOPsGbNGkwmE0eOHGHatGmMHj2aoUOHOiPGK9O1j1oe//kJTtne9WQymXiuZ0NMJpi18TBxB5OcE5+IiEgZZ3eS8/TTT3PnnXfSuXNn0tLS6NChA/fddx8PPvggI0aMcEaMV6aoJlDr/8Aww4r37Lq0cZVg+l1dBYAXf/mHK2yWABEREeAy5snJyclh9+7dpKWl0aBBAwICAhwdm1OU6Xly/mv/Cph6g2W01agtEBhp86VHU7K47o3FZOTk894dzejdNNqJgYqIiDhXicyTM2TIEFJTU/Hy8qJBgwa0atWKgIAA0tPTGTJkiN1ByyXEtoWY1pCfA6s+sOvSSkE+DD2zxMOrv28nKzffGRGKiIiUWXYnOV9++SWZmeeP2snMzOSrr75ySFByhskE1z5meb7uC8uMyHa4v0MNooN9OJKcxWfLNKRcRESuLDYnOSkpKSQnJ2MYBqmpqaSkpFi306dP8/vvvxMREeHMWK9MdbpBREPLUg9/f2bXpT6e7jzVox4AHy7ew9GULGdEKCIiUibZnOSEhIRQoUIFTCYTderUITQ01LqFhYUxZMgQhg0b5sxYr0wm09mRVqsnQ06GXZf3bhpNs6ohZOTk8/p8DSkXEZErh82Fx0uWLMEwDK6//np+/PFHKlSoYD3m5eVFbGws0dFlv7i1XBUeF8jPg/evhqQD0GMitH7Qrss3xp/mpg9XYjLBnGHX0rhKsJMCFRERcY7i/P22e3TVgQMHiImJwc2tWGt7lrpymeSApavqt8chOAZGbgR3z6KvOceo6RuZHXeEVtUq8P2D12AyaZVyEREpP0pkdFVsbCxubm5kZGSwY8cONm/eXGgTJ7nqLvCPgOSDsGWG3Zc/2b0ePp5urN1/irlbE50QoIiISNlid5Jz/PhxevbsSWBgIA0bNqRZs2aFNnssXbqUXr16ER0djclkYvbs2UVeM23aNJo2bYqfnx9RUVEMGTKEkydP2vsxyh9PH2jzsOX58nfAbLbr8ugQXx7sYBlS/oqGlIuIyBXA7iRn1KhRJCUlsWbNGnx9fZk3bx5ffvkltWvXZs6cOXbdKz09naZNm/LBB7bNAbNixQoGDhzIvffeyz///MOMGTNYu3Yt999/v70fo3xqcS94B8OJnbDzd7svf7BjDSoFeXPodCZTVux3fHwiIiJliN1Jzp9//slbb71FixYtcHNzIzY2lrvuuouJEycyYcIEu+7Vo0cPxo8fz0033WTT+atWraJatWqMHDmS6tWrc+211/Lggw+ydu1aez9G+eQTBK3uszxf/hbYOVm1n5cHT3W3DCn/4K/dHEvVkHIREXFddic56enp1vlwQkNDOX78OACNGzdmw4YNjo3uP9q0acPBgwf5/fffMQyDo0ePMnPmTG644YaLXpOdnV1oTp+UlBSnxuh0rYeChw8cXg/7ltp9ed+rKtO0SjBp2Xm89ce/TghQRESkbLA7yalbty47d1rmW2natCkff/wxhw8f5qOPPiIqKsrhAZ6rXbt2TJs2jdtuuw0vLy8iIyMJDg6+ZHfXhAkTCA4Otm4xMTFOjdHpAsKh2d2W58vfsvtyNzcTz/ZsAMD36w7yz5FkR0YnIiJSZtid5DzyyCMkJCQA8PzzzzN37lyqVq3Ke++9xyuvvOLwAM+1bds2HnnkEZ577jnWr1/PvHnz2L9/Pw899NBFrxkzZgzJycnW7eDBg06NsUS0HQEmd9i7GA7b33rWoloFejaJwjDgpV+3aZVyERFxScVehbxAwVDyqlWrEhYWVvxATCZmzZpF3759L3rO3XffTVZWFjNmnB1CvXz5ctq3b8+RI0dsakkqt/Pk/NdPD8Lm6VC/F9z2jd2XHzqdwfVvLiEnz8zHdzenW0PbVzgXEREpaU6fJyclJQXzf4Yu+/n5cdVVV+Hl5WXPrYolIyPjvEkI3d3dAa681ohrR1ket/8Kx+2vrakS6sf97asDliHl2XkaUi4iIq7F5iRn1qxZtGjRgqys80fkZGZm0rJlS3755Re73jwtLY24uDji4uIA2LdvH3FxccTHxwOWrqaBAwdaz+/Vqxc//fQTkydPZu/evaxYsYKRI0fSqlWrcrGkhENF1Ie6NwAGrHi3WLcY2qkW4YHeHDiZwVcrDzg2PhERkVJmc5IzefJknnzySfz8/M475u/vz1NPPcWkSZPsevN169YVmkTwscceo1mzZjz33HMAJCQkWBMegMGDB/PWW28xadIkGjVqxK233krdunX56aef7Hpfl3HtY5bHzdMh+ZDdlwd4e/BEt7oAvLdoFyfTsh0ZnYiISKmyuSYnOjqapUuXUqtWrQse3717Nx06dODIkSMODdDRXKYmp8DUnrB/mWVoeY9X7b7cbDboNWk5/xxJoXfTaN69/SqtayUiImWOU2tyTp8+TV5e3kWP5+bmcvr0aVtvJ45y7aOWxw1fQrr9y1u4uZl4sU8j3N1MzNl0hC9X7ndsfCIiIqXE5iSnWrVqrFu37qLH161bR2xsrEOCEjvUvB6imkJuBqz5qFi3aB4byjM31Adg/G/bWbvvlCMjFBERKRU2Jzk333wzY8eO5ejRo+cdS0xM5H//+x/9+vVzaHBiA5PpbG3O2o8h/USxbjOkXTV6N40mz2zw8LQNHE3Rkg8iIlK+2VyTk5qaSps2bYiPj+euu+6ibl1LweqOHTuYNm0aMTExrF69msDAQKcGfLlcriYHwJwPH7WHY/9A/d7Q/ytL8mOnjJw8bv5wJTsSU7m6agjTH2iDl4fd80WKiIg4XHH+fts1GWBycjJjxozh+++/t9bfhISEcPvtt/Pyyy8TGhpavMhLkEsmOQBH4uCzzmDOg5s/gya3Fus2B06m0+v95aRk5XH3NbG81LeRY+MUEREpBqcnOQUMw+DEiRMYhkF4eHi5Go3jskkOwOLXYPEr4BMMD6+BoOKtJfbnjqMMmWqpv3rj1qbc0ryKI6MUERGxm9NnPC5gMpkIDw8nIiKiXCU4Lq/9YxDdDLKSYc5wKOYs0NfXq8SoLrUBGDtrC1sPaxFPEREpf2xKcrp3787q1auLPC81NZXXXnvtkquCixO5e0Lfj8DdG3YvtAwrL6aR19emc70IsvPMPPj1ek6n5zgwUBEREeezKcm59dZb6devHw0aNOCpp55ixowZrFixgvXr17Nw4ULee+89+vfvT1RUFBs2bKBXr17OjlsuJqIedH7W8nz+WDi9v1i3cXMz8dZtVxFb0Y/DSZmMnL6RfPMVtj6YiIiUazbX5GRnZzNjxgy+//57li9fTnKypQvDZDLRoEEDunXrxr333kv9+vWdGvDlcumanALmfMtMyPErIfZaGPQLuBVvlNSOxBRu+mAlmbn5PNypJk92r+fgYEVERIpWYoXHYBlplZmZScWKFfH09CzOLUrFFZHkAJzaB5PbQW46dJsAbR4u9q3mbDrCyO82AvDRXVfTvVHxCppFRESKq8QKjwGCg4OJjIwsVwnOFaVCdeg23vJ80Tg4/m+xb9W7aTT3XlsdgMd/2MTuY2mOiFBERMSpNNObK2t+j2XZh7wsmP0Q5F987bGiPN2jHq2rVyA9J58Hv15HWnbx7yUiIlISlOS4MpMJek8C72A4vB5WvF3sW3m6uzHpzquJDPJhz/F0Rv+wiWL2dIqIiJQIJTmuLrgy3PC65fni1yBhc7FvFR7ozeS7rsbL3Y15/yTy0ZK9DgpSRETE8ZTkXAma9Id6PcGcC7MegrzsYt+qWdVQnu/dAIDX5+9g2a7jjopSRETEoexOcg4ePMihQ4esr9euXcuoUaP45JNPHBqYOJDJBD3fAb8wyyKei1+9rNvd2aoq/VtUwWzAyO82cvBUhmPiFBERcSC7k5w777yTv/76C4DExET+7//+j7Vr1zJ27FhefPFFhwcoDhIQDj3P1OSseAcO/l3sW5lMJl7s04gmVYI5nZHL0GnrycrNd0ycIiIiDmJ3krN161ZatWoFwA8//ECjRo1YuXIl06ZNY+rUqY6OTxypQW9ochsYZstoq5zit8D4eLoz+a7mVPD3YuvhFMbO2qpCZBERKVPsTnJyc3Px9vYGYOHChfTu3RuAevXqkZCQ4NjoxPF6vAaB0XByt2X+nMtQOcSX9+9ohpsJftxwiBfm/INZSz+IiEgZYXeS07BhQz766COWLVvGggUL6N69OwBHjhyhYsWKDg9QHMw3FPq8b3m+5iPYt/SybteuVhjj+zbGZIIvVx1g9MxN5OWbHRCoiIjI5bE7yXnttdf4+OOP6dSpE3fccQdNmzYFYM6cOdZuLCnjanWxTBQIMHsYZKVc1u3ubF2Vt/o3xd3NxE8bDjP8241k56lGR0RESlex1q7Kz88nJSWF0NBQ6779+/fj5+dHRESEQwN0tCtm7aqiZKfB5LaQdACa3Q19Jl32Lef/k8iIbzeSk2+mQ51wPr6rOb5e7g4IVkRErnQlsnZVZmYm2dnZ1gTnwIEDvPPOO+zcubPMJzhyDu8AuOkjwAQbv4Z/51/2Lbs1jOTzwS3w9XRn6b/HGfjFGlKyci8/VhERkWKwO8np06cPX331FQBJSUm0bt2aN998k759+zJ58mSHByhOFNsW2gyzPJ8zAjJOXfYt29cO5+t7WxHo48Hf+09z56erOZWec9n3FRERsZfdSc6GDRto3749ADNnzqRSpUocOHCAr776ivfee8/hAYqTXf8shNWFtKPw432Qf/ktLy2qVeC7+6+xDi+/7eNVJCZnOSBYERER29md5GRkZBAYGAjAH3/8wc0334ybmxvXXHMNBw4ccHiA4mSePnDzJ+DhC3sWWVp0HDDfTaPKwfzwYBsig3zYdSyNWz9eSfxJzYwsIiIlx+4kp1atWsyePZuDBw8yf/58unbtCsCxY8eu7ELe8iz6Kuj/JZjcYdN3sPB5h9y2VkQAMx5qQ9UKfhw8lcmtH69k19FUh9xbRESkKHYnOc899xyjR4+mWrVqtGrVijZt2gCWVp1mzZo5PEApIXW6Qe8z3Y0r3oVVHzrktjEV/JjxUBvqVArgaEo2t32ymq2Hkx1ybxERkUsp1hDyxMREEhISaNq0KW5uljxp7dq1BAUFUa9ePYcH6UgaQl6EZW/CojNrkPX7HBrf4pDbnk7PYdCUtWw+lEygtwdf3NOSltUqOOTeIiLi+orz97tYSU6BgtXIq1SpUtxblDglOUUwDJj7FKz9GNw8YcAMqHmdQ26dmpXLvV+uY+2+U/h4uvHJ3S3oUCfcIfcWERHXViLz5JjNZl588UWCg4OJjY0lNjaWkJAQXnrpJcxmTedf7plM0P1VaHgTmHPh+7vgSJxDbh3o48mX97SiU91wsnLN3PflOuZt1XpnIiLiHHYnOWPHjmXSpEm8+uqrbNy4kY0bN/LKK6/w/vvv8+yzzzojRilpbm5w08dQrT3kpMG0W+DUXofc2tfLnU/ubsGNjaPIyTfz8LQN/Lj+kEPuLSIici67u6uio6P56KOPrKuPF/j55595+OGHOXz4sEMDdDR1V9khKxmm3AhHt0Bodbj3DwhwzKzW+WaDp3/czIwzCc7TPerxQPsauLmZHHJ/ERFxLSXSXXXq1KkLFhfXq1ePU6cuf8ZcKUN8guGumRBSFU7vs7ToZDtmCLi7m4nX+jVhcNtqALw6dweDpqzlWKomDRQREcewO8lp2rQpkyadv5jjpEmTrCuSiwsJjIS7ZoFfRUjYBN/fDXmOWabBzc3E870a8MpNjfHxdGPZrhP0eGcZf+446pD7i4jIlc3u7qolS5Zw4403UrVqVescOatWreLgwYP8/vvv1iUfyip1VxXT4fUwtSfkZkDjW+GmTyy1Ow6y+1gqI76LY3tCCgCD21bj6R718PHUKuYiIlJC3VUdO3bk33//5aabbiIpKYmkpCRuvvlmdu7cWeYTHLkMlZtD/6/BzQO2zIAFji0yrxURyKyH2zKkXXUApq7cT98PVmiGZBERKbZi/VM8Ojqal19+mR9//JEff/yR8ePHYzabeeCBB+y6z9KlS+nVqxfR0dGYTCZmz55d5DXZ2dmMHTuW2NhYvL29qVatGl988UVxPobYq3YX6POB5fmqSbDCsQuy+ni681yvBky5pyVhAV7sSEyl5/vL+Wb1AS5jOicREblCOay/4eTJk3z++ed2XZOenk7Tpk354IMPbL6mf//+LFq0iM8//5ydO3fy3XffUbduXXvDleJqejv835kZkRc8C5umO/wtrqsbwdxHOtChTjjZeWb+N3srD369ntPpjqkFEhGRK4NHab55jx496NGjh83nz5s3jyVLlrB3714qVLAsCVCtWjUnRScX1XYkpB6F1R/Az8PAPwxqdXHoW4QHejN1cEumrNzPa3N38Me2o2w6tJS3+19F21phDn0vERFxTY6rHC0Bc+bMoUWLFkycOJHKlStTp04dRo8eTWZmZmmHdmUxmaDreGh0C5jz4PuBlsJkB3NzM3HvtdX56eG21Aj352hKNgM+X8Nr83aQm6/ZtUVE5NLKVZKzd+9eli9fztatW5k1axbvvPMOM2fO5OGHH77oNdnZ2aSkpBTaxAHc3KDvZKjRCXLT4ZtbYN9Sp7xVo8rB/DriWu5oVRXDgMmL93DL5JXsP5HulPcTcTlZKbB3sWVtOpEriM1DyG+++eZLHk9KSmLJkiXk5+cXLxCTiVmzZtG3b9+LntO1a1eWLVtGYmIiwcHBAPz000/ccsstpKen4+vre941L7zwAuPGjTtvv4aQO0h2KnzZC45sBJMbXPcMXPu4Q4eXn2vulgSe/mkLyZm5+Hu582KfRtx8dWVMJs2ULHJRPz0Im6dD70lw9d2lHY1IsTh1CHlwcPAlt9jYWAYOHFjs4G0RFRVF5cqVrQkOQP369TEMw7oi+n+NGTOG5ORk63bw4EGnxnjF8Q6Ewb/DVQPAMMOf4y0zI6efdMrb9WgcxdxH2tOqegXSc/J5fMYmHpkex4m0bKe8n0i5l5sF23+xPN/0XenGIlLCbC48njJlijPjsEm7du2YMWMGaWlpBAQEAPDvv//i5uZGlSpVLniNt7c33t7eJRnmlcfLD/p+CLFt4bfRsGcRfHQt3DoFql7j8LeLDvHlu/uvYfLi3by9cBdzNh3hr53HeKRzbQa1rYane7nqhRVxrv3LLF3KAAdWQsoRCIou3ZhESkip/jVIS0sjLi6OuLg4APbt20dcXBzx8fGApRXm3NahO++8k4oVK3LPPfewbds2li5dyhNPPMGQIUMu2FUlJazZXXD/IqhYC1KPwJQbYOX7TqkDcHczMfz62sx8qA0No4NIzcpj/G/b6f7OUpb8e9zh7ydSbu38/ZwXBvwzu7QiESlxpZrkrFu3jmbNmtGsWTMAHnvsMZo1a8Zzzz0HQEJCgjXhAQgICGDBggUkJSXRokULBgwYQK9evXjvPcdOSieXoVJDeGAxNOoHRj788T+YPgAyTzvl7ZpVDWXO8GuZcHNjKvh7sed4OoO+WMt9X/7NPhUmy5XObIadcy3Pa/2f5fGfn0ovHpESZvfaVeWd1q4qIYYB6z6HeWMgP8eykvmtX0Llq532lsmZuby3aBdfrtxPntnA093EkGurM+L62gR4l+qUUCKl4/B6+PR68AqAoSvh3aaAAaO2WP4/KVKOlMjaVSI2MZmg5X1w7x8QEgtJ8fBFN1j7qdOGsQb7evJszwbMG9WeDnXCyc03+HjJXq57YzEz1x/CbL6i8nmRc1pxOkNoLFS71vL6n1mlF5NICVKSI84V3QweXAr1elpadH4fDTPvsczb4SS1IgL58p6WfD6oBdUq+nE8NZvRMzZx0+SVbIx3TreZSJm040w9Tt0bLI8Nb7I8blWXlVwZlOSI8/mGwG3fQLdXLKuY/zMLPukEiVuc9pYmk4nO9Ssx/9EOPN2jHv5e7mw6mMRNH67ksR/iOJaS5bT3FikTTu+HY/+AyR1qd7Xsa9DH8johDk7uKc3oREqEkhwpGSYTtBkG98yFoMpwag981gU2fOXUWVi9Pdx5qGNN/nqiE7c0t0wz8NOGw1z3xmI+XLyb7LziTV4pUuYVdFVVbQN+lrX+8A+D6h0sz1WALFcAJTlSsmJawYPLLCM98rJgzgiYPRRynDsSKiLQhzdubcrsYe24KiaE9Jx8Js7bSde3lzJn0xGthSWuZ8dvlsd6NxTe3+jM7PVbVZcjrk9JjpQ8/4pw5w/Q+TnLUhCbvoP3msGajyHPuTMXXxUTwk9D2/LmrU0JD/TmwMkMRn63kQ4T/2Ly4j0kZeQ49f1FSkTmacvEfwB1exQ+Vq+npdv42D9wfGfJxyZSgpTkSOlwc4P2j8OgXyC4KqQdhblPWpKddV9AnvOSDTc3E/2aV+Gv0Z0Y1aU2Ff29SEjO4rV5O7hmwiLGztrC7mOpTnt/EafbtcAyT1V4fahQo/AxvwpQs7PluQqQxcUpyZHSVe1aGLEebnwTAqMh5TD8+ihMag4bv4H8PKe9dYC3B6O61GHF09fz+i1NqB8VRFaumWlr4uny1lIGfbGWJf8e5wqbSkpcwcW6qgoUdFn985NWJheXpskApezIzYL1U2HZm5B+zLKvQg3o+DQ0vgXc3J369oZhsHrvKb5YsY+F249a/9tfM9yfe9pV5+arK+PnpUkFpYzLy4aJNSEnFe5bBFVanH9OVgq8Xgvys+GhFRDZqOTjFLFTcf5+K8mRsicnwzJb8vK3IePMauZhdaDT09DgJktXl5MdOJnOlysP8MO6g6RlW1qTgn09uaNVVQa2iSU6RGulSRm1eyF80w8CIuGx7Rf//8v0AbDjV0u3cefnSjZGkWJQkmMDJTnlSHYarP0YVrwHWUmWfREN4boxluJJk8npIaRm5TJj3SGmrtxP/KkMwLI4aI9GkQy5tjpXVw11egwidvntcfj7M2g+GHq9e/HztsyEH++F0GowMq5E/v8kcjmU5NhASU45lJUMqyfDqg8g+8xMyZFN4LqxUKdbifzHOd9ssGj7Uaas2M+qvSet+5tWCebmq6vQo3EkEYE+To9D5JIMA95uaKltu/MHy/8/LiY7zdJllZcJ9//l1HXlRBxBSY4NlOSUY5mnYeUkWPMR5KRZ9lVuYWnZqdm5xP4luu1IClNW7OPnuCPknJlfx80E19SoSM8m0fRoFEmov1eJxCJSyJGNltnEPf3gyX3gWUTiPWOwZQbytiOg6/iSiFCk2JTk2EBJjgtIPwkr34U1n1j+FQoQVhda3Q9NbwfvwBIJ43hqNj/HHeaXzQlsOphk3e/hZqJdrTB6NY2ma8NKBPl4lkg8Ivz1Cix5zdKde/u0os/fNgd+uBuCYywrk6vLSsowJTk2UJLjQtKOWYqT138JuWdmTPYKhKvutKyAHl6nxEKJP5nBr1uO8OumBLYlnF181MvdjY51w+nZJIou9Svh763RWeJEk6+Fo1ug72TL/w+Kkptp6bLKSYN7F1hmJBcpo5Tk2EBJjgvKSoZN02HtJ3By99n9NTpBqwegTnenDz8/157jafy6KYFfNh9h97E0634fTzc616tEr6ZRdKobgY9nycUkV4CkeHinsWUW8dG7LTOL2+LH+2HLD9B6KPR49YKnzP8nkVA/L1pVr+DAgEXsoyTHBkpyXJjZDPsWw9pPzyxOeOanHVwVWg6BZgNt/w+/AxiGwc6jqfyy6Qi/bk7gwMkM6zF/L3e6NozkxsZRXFs7TAmPXL41H1tmDa/aFobMtf26nXPhu9vPDDnfdt4/CH7dfITh327EzQRfDG5Jp7oRDg5cxDZKcmygJOcKcfqAZXmIDV9aCpYB3L0tkwq2uh+im5VoOIZhsOVwMr9uTuDXTUc4kpxlPebv5c519SLo0SiKTnXD1aUlxfNlb9i3xFJA3HaE7dflZcPrtSE7GQb/ZpmF/IxjqVl0fXspSRm5AAR6ezBrWFtqRZRM3ZvIuZTk2EBJzhUmN9OyPs/ajyFh09n9VVpCy/uhYV/w8C7RkMxmg40HT/PLpgTmbU0kMeVswuPt4UaHOuF0bxhJl/qVCPZT0bLYIDMJXq8J5jwYsQEq1rTv+tnDIO4baHEv9HwLsCTm9325jkU7jtEgKgh/b3f+3n+a2Ip+zH64nUYQSolTkmMDJTlXKMOAQ+ssdTv/zAKz5V+m+IVBo35Qq4vlX7BefiUaltlssOlQEvO2JjJ3a6J1wkGwjNJqU7MiPRpF0bVhJcICSjYZk3KkYGK/sLowfK391xfMkuwXBo/vBHcPflh3kCdnbsbL3Y05I9oRHuBNnw9WcOh0Jm1rVuTLIa3wdNfyh1JylOTYQEmOkHbMMiJr3ReQeuTsfndviG1jmXOnVheIqF+iQ2oNw2B7Qirz/klk3tYE/j16tmjZzQQtqlWgR6NIujWM1LISUtjMIbD1R7j2Uejygv3X5+fCG3Ug8xTcPZtDFVrT/Z1lpGXn8VT3egztZGkZ2pGYQr8PV5Kek8+A1lUZ37cRJg07lxKiJMcGSnLEKj8Xdv0B/86HPX9C8sHCxwOjodb1lqSnRifwK9mRJXuOpzH/n0TmbU1k86HkQseaxoTQrWElOtQOp35UEO5u+kNzxcrLsXRVZadc3jDwXx6B9VMxmg1kwLEBrNxzkqurhjDjobaFfl8Ltx3l/q/XYRgwrndDBrWt5pjPIVIEJTk2UJIjF2QYcOJf2L0I9iyC/csh72ytDCY3qNzc0sJTs7NlCvwSHJZ+6HQG8/85yrytCaw7cJpz/18b6ONB6+oVuKZGRa6pUVFJz5Vmz5/w9U3gHw6P/1v8BWz3LoGvepPlGUzj1Pfx8PTm90faUz3M/7xTP1qyh1fn7sDdzcTUe1rSvnb4ZX4IkaIpybGBkhyxSW4mHFh5Nuk5vqPwcZ8QqHmdJemp9X8QWKnEQjuWmsWCbUdZtP0Yf+87ReqZVdILBPp40Kra2aSnQbSSHpf2+xOWWrNmd0OfScW/jzmfvNfr4pF5nME5T3B9r7sY2KbaBU81DIPHZ2zipw2HCfTxYPawdtQMDyj+e4vYQEmODZTkSLEkHzqb8OxZbBlue67oZpZJB+t0g8imxf/XtJ3yzQbbjqSweu9JVu89yVolPVcWw4C3G0HKIbhjOtTtUexb5eWb+eONu7kh81eW+nXh2tEzcbvE7yQ7L587P13D+gOnqRHmz6yH22k0oDiVkhwbKMmRy5afB4fXWxKeXX9YFkU8V0AlqN3VkvTU6ATeJfcvXHuSnlbVK9CiWiiNKgfj7aHJCMulhE3wcQfw8IWn9oFn8QvSP1y8mz/n/8xM7xcxewXi9sTuIhf4PJ6aTd8PVnA4KZNra4Ux5Z6WGnElTqMkxwZKcsThUhNh1wL4dx7sXXx2hXQAdy/L0PQ63S2JT4XqJRqaLUmPl7sbjasE0zw21LppuHo5sfhVWDwB6t4Id3xb7NvsSEyh1/vLycvP55+Qx/HLOgq3fwv1bizy2m1HUrjlo5Vk5OQzsE0sL/ZpVOw4RC5FSY4NlOSIU+Vlw4EVlhFb/86D0/sLHw+ra+nSqtMdYlqDe8nOblyQ9Kzae4J1+0+z/sBpTqbnnHdetYp+XB0bSotYS2tPrfCAS3ZdSCn5qD0kboY+H0Czu4p1i5w8M30+WMH2hBS61K/Ep5V+xLT6Q2h0C9zyuU33mP9PIg9+vR6Al/o24u5rYosVi8ilKMmxgZIcKTGGASd2WZKdf+dD/Cow8s8e9wm2dGdVaw+x7SC8XonV8pwN0eDAyQzWHbAkPOsPnCo0P0+BIB8Pro4NpXnVUJpXC+WqmBD8vLT8RKlKPgRvNwRMMHoXBBRvhNObf+zk/T93E+rnyfxHOxCRvBU+6wye/vDEbpsnyPzgr928Pn8n7m4mvh7Sira1wooVj8jFKMmxgZIcKTWZpy3Dff+db6nlKVhTq4BfRYhta0l6ql0L4fVLPOkBSM7IZcPB02w4cJp1+08TdzCJzNz8Que4u5moGe5P/aggGkQF0SA6iPpRQermKklrP4XfR0PMNXDv/GLdIu5gEv0mryTfbPDhgKu5oXGUJTl/t4llVfNbv7QsfWIDwzB47IdNzNp4mGBfT2YPa3fB4ecixaUkxwZKcqRMMOdblpnYv9QyJ8/BtZCbUfgc31BLC0+1ay1bRMNSSXry8s3sSExl3f5TrI9PYv3+U4UWGD1XRKA3DaILJz7VKvprNJczfNUX9v4F//citHvE7suzcvO54b1l7D2eTu+m0bx3xzmL1i54Dla8C/V7w21f23XP2z9ZTdzBJGqEnxlx5asRV+IYSnJsoCRHyqS8HMsorQPLLUlP/Orzkx6fkHOSnnZQqVGJTkh4rqMpWWw7ksK2hBS2HUlhe0IK+06mc6H/mvh6ulMvKrBQq0+9yEB1d12OrGSYWNOyBtvwdRBW2+5bvPjLNr5YsY+IQG/+eLQDIX7nLLhpHbXlY+my8rZ91fFjqVn0mbSChOQs2tcOY8rglnhoxJU4gJIcGyjJkXIhPxeOxMH+ZZZC5vjVhUdtAXgHQ6WGUKkBRBRs9cE3pDQiJj07jx2JqWxLsCQ9246ksCMxhaxc83nnmkxQtYIfdSsFUi8ykLqRQdSLClSrj622/mhZr6pibRixzu7LV+05yR2frgZgyuCWXFcvovAJhgHvN4dTe+Dmz6DJrfaFdziZWz9aRWZuPoPbVuOF3g3tjlHkv5Tk2EBJjpRL+XmWf10XJD0HVkFO6oXPDapyJvGpb+niqtQAwuqAR8nXy+SbDfadSLckPWcSn20JKRxPzb7g+d4ebtSuFEDdSkFnkh9LEhQe6K2FIM/1432wZQa0HQldX7Lr0rTsPLq/s5RDpzO5o1UME25ucuET/xwPS1+HujfAHd/ZHeK8rQk89M0GAF65qTF3tq5q9z1EzqUkxwZKcsQl5OfB8e1wdBsc++fM4zZIOXzh803uULHW+a0+IVXBveRrJk6mZbMzMZXtiansTExhZ2Iq/x5NO6/AuUAFfy/qVjqb9FQP8ye2oj8Rgd5X3tD2/FzLgpxZyTBkPlS9xq7Lx/y0me/WHqRKqC/zRnUgwPsi3YZHt8HkNpa5nkbvKlYL4fuLdvHmgn/xcDNx77XV6dkkmkaVg5SwSrEoybGBkhxxaZlJcGz7OYnPmedZyRc+3+QOodWgYk1LElSxJlQ48zyocokWOpvNBvGnMtiRmMqOM4nPzsRU9p9Mx3yR/0p5e7gRU8GP2Ap+lseKlq1qBX+qhPri4+mCMznvXQxf9QG/MBj9r111WX/tOMY9U/8GYPoD13BNjYqXvuCD1pZ12/pOhqvutDtUwzAY9X0cP8cdse6rVtGPG5tEcWPjaOpHBSrhEZspybGBkhy54hgGpByxtPQc23a29efEbsjLvPh1Hj5QoUbhxKcgEfIPtxTWlICs3Hx2HU07m/gcTeXAyQwOJ2WSf7HsB0t4kUE+VK3gR9UzCVDViv7EVvCjWkX/8rvO0tynYM1HcNVd0PcDmy9Lysih69tLOZaazZB21XmuV4OiL1r8Gix+xbIQ7V0/FivcfLPBH/8k8uvmBBbtOFqoRqtGuD89G0fRs2k0dSrZXtwsVyYlOTZQkiNyhtkMqQlwcrelwPRkwbbbMlOzOffi13oHQUgshMZe4LEqeDl/fpS8fDNHkrI4cCqdAyczOHgqgwMnMzhwKoP4k+mk51y466tAqJ8n1cL8qV7R0vVVLcyP6mH+VAvzJ8injCZAhgHvNIHkeJuXXSgw8ruNzNl0hBrh/vw+sr1trVwndsGkFuDmYemy8qtwGcFbitMX7TjGr5uOsPjf4+TknU146lQKoGeTaG5sEqUVzeWCyl2Ss3TpUl5//XXWr19PQkICs2bNom/fvjZdu2LFCjp27EijRo2Ii4uz+T2V5IjYID/P8of05F5L0mNNhHZD0kGgiP9s+IVdJAGKheAY8PC69PWXyTAMTqXncODUOcnPmURo/8l0jl2k8LlABX8vqlX0O5sEnXmsFuZHYGkmQIlb4aN2lla2J/cWmUzmmw02HUrit80JfL58H+5uJn4c2parYkJsf8+ProXELdDrXWg++LLCP1dqVi4Ltx/l100JLN11nNz8s7+p+lFB9GwSRc8mUcRW1ISCYlGcv9+lOlFFeno6TZs2ZciQIdx88802X5eUlMTAgQPp3LkzR48edWKEIlcodw9LV1WFGlC7S+FjuVmWlp6kA5ZZcQuenz5gecxKhowTlu3w+gvc3GRZqd0/3LIUgX/BFgb+Eec8P7O/iJWwL8RkMlExwJuKAd5cXTX0vOPp2XnsP2lpAdp3Ip39J848P5nO8dRsTqXncCo9hw3xSeddG+rnSZVQP6qE+hJTwfJo2SzPnTr/z87fLY81rrtogpOckcuSXcdZvOMYi/89zqlz1iZ7uFNN+xIcgIY3W5Kc1R+BVwBU7wABEUVfV4RAH09ualaFm5pVITkz19qltWL3CbafmYbg9fk7aVw5mC71K9GkSjCNKgcTHqhZtcV2Zaa7ymQy2dySc/vtt1O7dm3c3d2ZPXu2WnJEypLMpMJJT6HH+EvXAV2Id1DhpOfcLSD8bGIUEG6ZMPEya4XSsvM4cDKd/ScsrT77TqRz4GQ6+05kcCLt0i1AABX9vc4mPRXOJj8xZ/ZdVjH0xx0hIQ56vw9XDwQsrVY7j6by147j/LXjGOvjTxeqVQr09qBDnXC6NqxErybR9o9GO70f3ru68LprYXWhevuzS5D4O26dqtPpOcw/k/Cs3HPivKLzyCAfGlcJpnFly6bE58pR7rqrzmVrkjNlyhQmT57MypUrGT9+vJIckfLEMCDtmKUWKP144S3t3NcnLI+Xqgu6EDfP/7QQRRR+7h9mqSvx9AdPX/D0szx6+Ng0kiwtO49DpzM4eCqTQ6czOHQ6k4OnLI+HTmeQkpVX5D0q+ntROdSXyiGWLTrE1/q6Sqgvwb6eFx5xlHwY3m4AmMgcuZ2VR934c8cxFu88zuGkwolj7YgArq8XwXX1ImgeG4rn5c44fGAVbP/FsgxJ4lbO666MaGBJeKqfWWz2Mmt3CpxIy2b+P4n8ve8UWw4ns/fEhWfVjgzyodGZpKdxlSAaVQ4mItD+FkAp28pdd5W9du3axdNPP82yZcvw8LAt9OzsbLKzz/7rKyUlxVnhiUhRTCYIrGTZimIYkJV0NuFJP25JkKxJ0LHCiVF2iiUpSj1i2ezl4Vs48Sn03PIY4OlLPS9/6nn6WbqLwvwh2t/y3NOfNPw4mulOQqY7B9MgPtWNvSlwICmXQ6czScvO42R6DifTc9h86MLD+v293Kkceib5OScBapIwk+rALq/63PhWXKGiXW8PN9rWrMj19SLoVDeCmAq2rRxus9g2lg0g45RlQsp9yyxLkBz75+zIvbUfAybLkiMFLT2xbYs9C3dYgDcDWscyoHUsYEkytx1JYcvhZLYcSrImPokpWSSmZLFw+9nyhXMTn+rhllF1VSv4EeJ3kSRSXFK5SXLy8/O58847GTduHHXq1LH5ugkTJjBu3DgnRiYiTmEyWRYp9Q21bW2m3KwLtA4dO5sQFbQWZZ62dJnlZkLeOQuN5mVatsxTxQ454MxW878H3DwwfP0xggLI9gwmwyOIFFMgp8wBHM3z50iOL/GZPsRn+ZCUG8jpYwFsOBrAEvwwsLTCfOk5k+ru8FN6E3LyzVQO8eX6ehFcXy+Ca2pUxNerhOYE8qsA9XtZNrB8v/uXW2bj3rcMTuyEo1ss2+oPARNENrLUdwVGQ1DBVhmCoiAwyubZuAO8PWhVvQKtqp9tKTo38dl6OJkth5PZczztgokPWLrvCuZUqnpmfqWCaQYqh/pefquXlCnlprsqKSmJ0NBQ3N3P/h/ZbDZjGAbu7u788ccfXH/99eddd6GWnJiYGHVXiYhlNfjcMwlPbsZ/Hi+0L93ymJNeeMvNsKwtlpNxZl+aZV9+TtExXCo83MhwDySZQCrlJ+BBPt+3msnVza+hVkRA2WyRSDt2NuHZv8wyIq8o/uGWZCeo8pkE6Mzzgn2BkZZFQm38vOnZeWxLSGHLoWS2JaQQfzKDA6fSOZpy6ZoqNxNEh/hak56YM3MqVQ+zbCWWSMoFuXRNjtlsZtu2bYX2ffjhh/z555/MnDmT6tWr4+9f9FBD1eSISInJzy2cCGWlWFqKMk5d4vG05TE3/fz7RTSEoStKbCJGh0g5Yhlll3LEsuxIypHCW37RxdyApW7KWmN15jGg0n/2nSlC9wm+4HeUlZvPodMZxJ+ZViD+zBQD8We2Cy0me67KIb7UCPenxpmkp0Z4ADXC/YkO9r3ylhcpBeWuJictLY3du89m+fv27SMuLo4KFSpQtWpVxowZw+HDh/nqq69wc3OjUaNGha6PiIjAx8fnvP0iImWCu6elHqU4NSl52YWTn6wkqNyifCU4cLZ76kIMw/LZUo+ckwQlnH2emmApuM5JtXQtJsdbtqK4e58pQD+T+PiEgE8wPj7B1DqzERoMUcGWhMinKoZPMMdzvIlPyrYmPfFnphXYezyd5MxcDidlcjgpk2W7ThR6Ox9PN6pV9KfmmaTn3ASozE4seYUo1SRn3bp1XHfdddbXjz32GACDBg1i6tSpJCQkEB9vww9aRMTVeHif6baJKu1InMdkAv+Kli2y8cXPy0k/W3SeduycovNj5+w/atmXk2ppHUo5ZNlsDQWIACK8AmnhU5D8BENwAEZFb3LwIiXPnaQcd05lmziRZeJoBhzNhAyzJ9nHPck+5skuvNiKJ1l4kW14kufpj4dfCJ4BFQjwD6LCmfmbwgK8qODvZZnPyd+Limdee3uoS8yRykx3VUlRd5WIiAvLzTw/IcpKPrOlnPP8P9uFugcdHZrhTgp+pBh+pOB/5tGPFMOf5DOvczwDMbyDcPMNwcMvBB//IAICgwkMCiY4OISQoGDCgnwJC/AmyMejbNZlOUm5664SERFxKE9fyxIiobH2XZefeyYJSiqc/OSkWboO87LPjMDLtnSdnfuYe+H95pwMjOwU3LKSMRl5eJryqUgqFU2pl44l+8yWdOHD6YY3GXhzEB+y3XzJc/cl38MPw9Mfk3cA7t7+ePgE4OPtha+nGz4e7vh6mnA/Nx8yDKzzHRU8t7Z5FDyawORmaXEznRl1ZnL7z77/nFOwzzsArhlq3/8GTqAkR0RExN3zbNeZg1gHoxuGJRHKSj4/iTqzz8hMJjftFDkZp8nPsOwzZafilpeBR14GXuZM3M4kH/6mbPzJBlIs+UjemS0LKCJ/KjEBkUpyREREXJ7JBF5+lu0iNVYmwOvMdkEFidKZKQqyM9NITj5Nakoy6alJZKSlkJWRSk5GKnlZqZiz0sjOzSMz10xWbj5mTFjaa0xw5jlnXhvWR1Ohuaw9TJaJJn08THi7m/C2PoK3uwkvdxPe7uDl7oaXO3h5mPByA0838PQLxtcx395lUZIjIiJS1p2bKBGONxBR2VIsXRSz2SA5M5eTZxaePZWebXmelnPOvoLnlgVqravC27mySoEQP0/iehbvWkdSkiMiIuLC3NxMhPp7Eep/0XaiQgzDIC07j9Ssgi2XlKxcUrPySDnzOjUrj5TMXOvxwufmEexbNobOK8kRERERK5PJRKCPJ4GXMceP+b/Lx5cSLdIhIiIiDlVWZoBWkiMiIiIuSUmOiIiIuCQlOSIiIuKSlOSIiIiIS1KSIyIiIi5JSY6IiIi4JCU5IiIi4pKU5IiIiIhLUpIjIiIiLklJjoiIiLgkJTkiIiLikpTkiIiIiEtSkiMiIiIuyaO0AyhphmFZ/j0lJaWUIxERERFbFfzdLvg7bosrLslJTU0FICYmppQjEREREXulpqYSHBxs07kmw56UyAWYzWaOHDlCYGAgJpPJofdOSUkhJiaGgwcPEhQU5NB7uzJ9b/bTd1Y8+t6KR99b8eh7s9+lvjPDMEhNTSU6Oho3N9uqba64lhw3NzeqVKni1PcICgrSD7oY9L3ZT99Z8eh7Kx59b8Wj781+F/vObG3BKaDCYxEREXFJSnJERETEJSnJcSBvb2+ef/55vL29SzuUckXfm/30nRWPvrfi0fdWPPre7Ofo7+yKKzwWERGRK4NackRERMQlKckRERERl6QkR0RERFySkhwRERFxSUpyHOSDDz6gWrVq+Pj40Lp1a9auXVvaIZVpL7zwAiaTqdBWr1690g6rzFm6dCm9evUiOjoak8nE7NmzCx03DIPnnnuOqKgofH196dKlC7t27SqdYMuQor63wYMHn/f76969e+kEW0ZMmDCBli1bEhgYSEREBH379mXnzp2FzsnKymLYsGFUrFiRgIAA+vXrx9GjR0sp4rLBlu+tU6dO5/3eHnrooVKKuGyYPHkyTZo0sU7616ZNG+bOnWs97qjfmpIcB/j+++957LHHeP7559mwYQNNmzalW7duHDt2rLRDK9MaNmxIQkKCdVu+fHlph1TmpKen07RpUz744IMLHp84cSLvvfceH330EWvWrMHf359u3bqRlZVVwpGWLUV9bwDdu3cv9Pv77rvvSjDCsmfJkiUMGzaM1atXs2DBAnJzc+natSvp6enWcx599FF++eUXZsyYwZIlSzhy5Ag333xzKUZd+mz53gDuv//+Qr+3iRMnllLEZUOVKlV49dVXWb9+PevWreP666+nT58+/PPPP4ADf2uGXLZWrVoZw4YNs77Oz883oqOjjQkTJpRiVGXb888/bzRt2rS0wyhXAGPWrFnW12az2YiMjDRef/11676kpCTD29vb+O6770ohwrLpv9+bYRjGoEGDjD59+pRKPOXFsWPHDMBYsmSJYRiW35anp6cxY8YM6znbt283AGPVqlWlFWaZ89/vzTAMo2PHjsYjjzxSekGVE6GhocZnn33m0N+aWnIuU05ODuvXr6dLly7WfW5ubnTp0oVVq1aVYmRl365du4iOjqZGjRoMGDCA+Pj40g6pXNm3bx+JiYmFfnvBwcG0bt1avz0bLF68mIiICOrWrcvQoUM5efJkaYdUpiQnJwNQoUIFANavX09ubm6h31u9evWoWrWqfm/n+O/3VmDatGmEhYXRqFEjxowZQ0ZGRmmEVybl5+czffp00tPTadOmjUN/a1fcAp2OduLECfLz86lUqVKh/ZUqVWLHjh2lFFXZ17p1a6ZOnUrdunVJSEhg3LhxtG/fnq1btxIYGFja4ZULiYmJABf87RUckwvr3r07N998M9WrV2fPnj0888wz9OjRg1WrVuHu7l7a4ZU6s9nMqFGjaNeuHY0aNQIsvzcvLy9CQkIKnavf21kX+t4A7rzzTmJjY4mOjmbz5s089dRT7Ny5k59++qkUoy19W7ZsoU2bNmRlZREQEMCsWbNo0KABcXFxDvutKcmRUtGjRw/r8yZNmtC6dWtiY2P54YcfuPfee0sxMrkS3H777dbnjRs3pkmTJtSsWZPFixfTuXPnUoysbBg2bBhbt25VnZydLva9PfDAA9bnjRs3Jioqis6dO7Nnzx5q1qxZ0mGWGXXr1iUuLo7k5GRmzpzJoEGDWLJkiUPfQ91VlyksLAx3d/fzqr6PHj1KZGRkKUVV/oSEhFCnTh12795d2qGUGwW/L/32Ll+NGjUICwvT7w8YPnw4v/76K3/99RdVqlSx7o+MjCQnJ4ekpKRC5+v3ZnGx7+1CWrduDXDF/968vLyoVasWzZs3Z8KECTRt2pR3333Xob81JTmXycvLi+bNm7No0SLrPrPZzKJFi2jTpk0pRla+pKWlsWfPHqKioko7lHKjevXqREZGFvrtpaSksGbNGv327HTo0CFOnjx5Rf/+DMNg+PDhzJo1iz///JPq1asXOt68eXM8PT0L/d527txJfHz8Ff17K+p7u5C4uDiAK/r3diFms5ns7GzH/tYcWxt9ZZo+fbrh7e1tTJ061di2bZvxwAMPGCEhIUZiYmJph1ZmPf7448bixYuNffv2GStWrDC6dOlihIWFGceOHSvt0MqU1NRUY+PGjcbGjRsNwHjrrbeMjRs3GgcOHDAMwzBeffVVIyQkxPj555+NzZs3G3369DGqV69uZGZmlnLkpetS31tqaqoxevRoY9WqVca+ffuMhQsXGldffbVRu3ZtIysrq7RDLzVDhw41goODjcWLFxsJCQnWLSMjw3rOQw89ZFStWtX4888/jXXr1hlt2rQx2rRpU4pRl76ivrfdu3cbL774orFu3Tpj3759xs8//2zUqFHD6NChQylHXrqefvppY8mSJca+ffuMzZs3G08//bRhMpmMP/74wzAMx/3WlOQ4yPvvv29UrVrV8PLyMlq1amWsXr26tEMq02677TYjKirK8PLyMipXrmzcdtttxu7du0s7rDLnr7/+MoDztkGDBhmGYRlG/uyzzxqVKlUyvL29jc6dOxs7d+4s3aDLgEt9bxkZGUbXrl2N8PBww9PT04iNjTXuv//+K/4fJRf6vgBjypQp1nMyMzONhx9+2AgNDTX8/PyMm266yUhISCi9oMuAor63+Ph4o0OHDkaFChUMb29vo1atWsYTTzxhJCcnl27gpWzIkCFGbGys4eXlZYSHhxudO3e2JjiG4bjfmskwDKOYLUsiIiIiZZZqckRERMQlKckRERERl6QkR0RERFySkhwRERFxSUpyRERExCUpyRERERGXpCRHREREXJKSHBERwGQyMXv27NIOQ0QcSEmOiJS6wYMHYzKZztu6d+9e2qGJSDnmUdoBiIgAdO/enSlTphTa5+3tXUrRiIgrUEuOiJQJ3t7eREZGFtpCQ0MBS1fS5MmT6dGjB76+vtSoUYOZM2cWun7Lli1cf/31+Pr6UrFiRR544AHS0tIKnfPFF1/QsGFDvL29iYqKYvjw4YWOnzhxgptuugk/Pz9q167NnDlznPuhRcSplOSISLnw7LPP0q9fPzZt2sSAAQO4/fbb2b59OwDp6el069aN0NBQ/v77b2bMmMHChQsLJTGTJ09m2LBhPPDAA2zZsoU5c+ZQq1atQu8xbtw4+vfvz+bNm7nhhhsYMGAAp06dKtHPKSIO5Lg1RUVEimfQoEGGu7u74e/vX2h7+eWXDcOwrPT80EMPFbqmdevWxtChQw3DMIxPPvnECA0NNdLS0qzHf/vtN8PNzc26unh0dLQxduzYi8YAGP/73/+sr9PS0gzAmDt3rsM+p4iULNXkiEiZcN111zF58uRC+ypUqGB93qZNm0LH2rRpQ1xcHADbt2+nadOm+Pv7W4+3a9cOs9nMzp07MZlMHDlyhM6dO18yhiZNmlif+/v7ExQUxLFjx4r7kUSklCnJEZEywd/f/7zuI0fx9fW16TxPT89Cr00mE2az2RkhiUgJUE2OiJQLq1evPu91/fr1Aahfvz6bNm0iPT3denzFihW4ublRt25dAgMDqVatGosWLSrRmEWkdKklR0TKhOzsbBITEwvt8/DwICwsDIAZM2bQokULrr32WqZNm8batWv5/PPPARgwYADPP/88gwYN4oUXXuD48eOMGDGCu+++m0qVKgHwwgsv8NBDDxEREUGPHj1ITU1lxYoVjBgxomQ/qIiUGCU5IlImzJs3j6ioqEL76taty44dOwDLyKfp06fz8MMPExUVxXfffUeDBg0A8PPzY/78+TzyyCO0bNkSPz8/+vXrx1tvvWW916BBg8jKyuLtt99m9OjRhIWFccstt5TcBxSREmcyDMMo7SBERC7FZDIxa9Ys+vbtW9qhiEg5opocERERcUlKckRERMQlqSZHRMo89aqLSHGoJUdERERckpIcERERcUlKckRERMQlKckRERERl6QkR0RERFySkhwRERFxSUpyRERExCUpyRERERGXpCRHREREXNL/A2PHzEUb3XeVAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "with open(HISTORY_PATH, \"rb\") as infile:\n",
    "    history = pickle.load(infile)\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot(history[\"loss\"], label=\"Training Loss\")\n",
    "ax.plot(history[\"val_loss\"], label=\"Validation Loss\")\n",
    "ax.set_xlabel(\"Epoch\")\n",
    "ax.set_ylabel(\"Loss (Categorical Crossentropy)\")\n",
    "ax.set_title(\"Loss by Epoch\")\n",
    "ax.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loaded model\n",
      "loaded vectorizer\n"
     ]
    }
   ],
   "source": [
    "lstm_model = keras.models.load_model(MODEL_PATH)\n",
    "print(\"loaded model\")\n",
    "\n",
    "with open(VECTORIZER_PATH, \"rb\") as infile:\n",
    "    vectorizer = pickle.load(infile)\n",
    "print(\"loaded vectorizer\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5 Evaluate the model with per-class accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiate dictionaries to keep track of errors\n",
    "num_correct_by_idx = {k:0 for k in vectorizer.int_to_token_mapping.keys()}\n",
    "total_count_by_idx = {k:0 for k in vectorizer.int_to_token_mapping.keys()}\n",
    "\n",
    "# Iterate over the batches in the testing data, and count\n",
    "# how many predictions are correctly made\n",
    "for batch_samples, batch_labels in testing_dataset:\n",
    "    gold_labels = batch_labels.numpy().argmax(axis=1)\n",
    "    predictions = lstm_model(batch_samples).numpy().argmax(axis=1)\n",
    "    for i in range(len(gold_labels)):\n",
    "        label = gold_labels[i]\n",
    "        total_count_by_idx[label] += 1\n",
    "        # If the model prediction is correct, count it\n",
    "        if label == predictions[i]:\n",
    "            num_correct_by_idx[label] += 1\n",
    "\n",
    "# Compute how many predictions of each character were actually correct           \n",
    "pct_correct_by_idx = {\n",
    "    idx:(num_correct_by_idx[idx]/total_count_by_idx[idx])\n",
    "    for idx in num_correct_by_idx.keys()\n",
    "    if total_count_by_idx[idx] != 0\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAHWCAYAAABACtmGAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABaFUlEQVR4nO3deXiM1///8Vf2EIktIaQIIdZYGqW2omiqaLVqq4qldLOnpbS1t6jWUq3WhxZdFLW1WrUvtRZFWkvtYt9VYg/J+f3RX+ZrJNyZSEzwfFzXXJc5c59zv2cyM+Y1577PuBhjjAAAAAAAt+Xq7AIAAAAAILMjOAEAAACABYITAAAAAFggOAEAAACABYITAAAAAFggOAEAAACABYITAAAAAFggOAEAAACABYITAAAAAFggOAG4r9SqVUu1atVydhkAnKxt27bKli1bqrZ1cXHRgAEDMrYgAA88ghMAm8mTJ8vFxcV28fb2VmhoqDp37qyTJ086u7z7wm+//SYXFxflz59fiYmJzi7nvrRixQq5uLgoJiYm3fvc+hx3d3dXUFCQ2rZtq6NHj95d4Q+w3377Ld2CR0JCgvz8/PTcc88lu23UqFFycXFRmzZtkt3Wr18/ubi4aPfu3Xddw9q1azVgwACdP3/+rscC8PBwd3YBADKfQYMGqXDhwrp69apWr16tL7/8Ur/99pu2bdumrFmzOru8TG3KlCkKDg5WTEyMli1bprp16zq7JKTg5uf4H3/8ocmTJ2v16tXatm2bvL29nV1epvPbb79p7Nix6RKe3Nzc9Pjjj2vt2rXJbluzZo3c3d21Zs2aFG/LkyePQkNDHd7nlStX5O7+fx951q5dq4EDB6pt27bKkSOHw+MBeDgx4wQgmfr16+vll19Whw4dNHnyZHXv3l0HDhzQzz//fNdjX758OR0qzJwuXbqkn3/+WVFRUapQoYKmTJni7JJu69KlS84uwalufo5/9dVXevvtt7Vv3z7NnTv3ntZhjNGVK1fu6T4zg+rVq+vMmTP6559/7NrXrFmjZs2aad++fTpx4oSt/caNG1q/fr2qVauWpv15e3vbBScASAuCEwBLTz75pCTpwIEDtrbvv/9e4eHhypIli3LlyqUWLVro8OHDdv1q1aqlMmXKaNOmTXriiSeUNWtWvfvuu3fc1/fff69KlSopa9asypkzp5544gktWrTottvHx8erX79+Cg8PV/bs2eXj46MaNWpo+fLlybadNm2awsPD5evrKz8/P4WFhenTTz+13X79+nUNHDhQxYoVk7e3t3Lnzq3q1atr8eLFqXqc5syZoytXrqhp06Zq0aKFZs+eratXrybb7urVqxowYIBCQ0Pl7e2tfPny6YUXXtC+ffts2yQmJurTTz9VWFiYvL29FRAQoKefflp//vmnJCkmJkYuLi6aPHlysvFvPZ9jwIABcnFx0Y4dO/TSSy8pZ86cql69uiTp77//Vtu2bVWkSBF5e3srMDBQ7du319mzZ5ONe/ToUb3yyivKnz+/vLy8VLhwYb3xxhuKj4/X/v375eLiolGjRiXrt3btWrm4uGjq1KmpehydoUaNGpJk9zeQpJ07d+rFF19Urly55O3trYoVKyYLV0mH/61cuVKvvfaacufOLT8/P0VGRurff/+12zY4OFgNGzbUwoULVbFiRWXJkkX/+9//JEnnz59X9+7dVaBAAXl5ealo0aL66KOPkh3yafU8Tu1YSc+hTz75ROPHj1dISIi8vLz02GOPaePGjbbt2rZtq7Fjx0qS3WGOjtRzq6Tn380zS/v379eJEyfUuXNneXt7290WHR2tS5cu2frd7OjRo2rcuLGyZcumgIAAvf3220pISLDb5ubXxIABA9SzZ09JUuHChW335+bDPFPz/rZnzx41adJEgYGB8vb21iOPPKIWLVooNjbWbr+dO3fWlClTVLx4cXl7eys8PFwrV660G+vgwYN68803Vbx4cWXJkkW5c+dW06ZNUzz09Pz58+rRo4eCg4Pl5eWlRx55RJGRkTpz5oxtm2vXrql///4qWrSovLy8VKBAAfXq1UvXrl1L6c8BIJX4+gWApaQPk7lz55Ykffjhh+rbt6+aNWumDh066PTp0/rss8/0xBNPaMuWLXaHvpw9e1b169dXixYt9PLLLytv3ry33c/AgQM1YMAAVa1aVYMGDZKnp6fWr1+vZcuW6amnnkqxT1xcnL766iu1bNlSHTt21IULF/T1118rIiJCGzZsUPny5SVJixcvVsuWLVWnTh199NFHkqR//vlHa9asUbdu3ST994Fq6NCh6tChgypVqqS4uDj9+eef2rx5s+rVq2f5OE2ZMkW1a9dWYGCgWrRood69e+uXX35R06ZNbdskJCSoYcOGWrp0qVq0aKFu3brpwoULWrx4sbZt26aQkBBJ0iuvvKLJkyerfv366tChg27cuKFVq1bpjz/+UMWKFS1rSUnTpk1VrFgxDRkyRMYY2+Oyf/9+tWvXToGBgdq+fbvGjx+v7du3648//rB9QD527JgqVaqk8+fP69VXX1WJEiV09OhRzZw5U5cvX1aRIkVUrVo1TZkyRT169Ej2uPj6+qZ4TktmkfQBNWfOnLa27du3q1q1agoKClLv3r3l4+OjH3/8UY0bN9asWbP0/PPP243RuXNn5ciRQwMGDNCuXbv05Zdf6uDBg7bzr5Ls2rVLLVu21GuvvaaOHTuqePHiunz5smrWrKmjR4/qtddeU8GCBbV27Vr16dNHx48f1+jRoyWl7nmc2rGS/PDDD7pw4YJee+01ubi4aPjw4XrhhRe0f/9+eXh46LXXXtOxY8e0ePFifffdd3Z9U1NPSh5//HG5u7tr9erV6tChg6T/QpSPj48ee+wxVaxYUWvWrFGTJk1st0lKFpwSEhIUERGhypUr65NPPtGSJUs0YsQIhYSE6I033khx3y+88IJ2796tqVOnatSoUfL395ckBQQESErd+1t8fLwiIiJ07do1denSRYGBgTp69Kh+/fVXnT9/XtmzZ7ft7/fff9f06dPVtWtXeXl56YsvvtDTTz+tDRs2qEyZMpKkjRs3au3atWrRooUeeeQRxcTE6Msvv1StWrW0Y8cO2yHSFy9eVI0aNfTPP/+offv2evTRR3XmzBnNnTtXR44ckb+/vxITE/Xss89q9erVevXVV1WyZElt3bpVo0aN0u7du/XTTz/d9u8CwIIBgP9v0qRJRpJZsmSJOX36tDl8+LCZNm2ayZ07t8mSJYs5cuSIiYmJMW5ububDDz+067t161bj7u5u116zZk0jyYwbN85y33v27DGurq7m+eefNwkJCXa3JSYm2o1Zs2ZN2/UbN26Ya9eu2W3/77//mrx585r27dvb2rp162b8/PzMjRs3bltDuXLlTIMGDSxrTcnJkyeNu7u7mTBhgq2tatWq5rnnnrPbbuLEiUaSGTlyZLIxku7nsmXLjCTTtWvX225z4MABI8lMmjQp2TaSTP/+/W3X+/fvbySZli1bJtv28uXLydqmTp1qJJmVK1fa2iIjI42rq6vZuHHjbWv63//+ZySZf/75x3ZbfHy88ff3N23atEnWzxlSeo7PnDnTBAQEGC8vL3P48GHbtnXq1DFhYWHm6tWrtrbExERTtWpVU6xYsWRjhoeHm/j4eFv78OHDjSTz888/29oKFSpkJJkFCxbY1TV48GDj4+Njdu/ebdfeu3dv4+bmZg4dOmSMSd3zOLVjJT2HcufObc6dO2fb7ueffzaSzC+//GJr69Spk0npI0Nq6rmdxx57zISEhNiuv/baa6Z27drGGGN69eplHnvsMdttL774osmaNau5fv26ra1NmzZGkhk0aJDduBUqVDDh4eF2bbe+Jj7++GMjyRw4cMBuu9S+v23ZssVIMjNmzLjjfZRkJJk///zT1nbw4EHj7e1tnn/+eVtbSq/DdevWGUnm22+/tbX169fPSDKzZ89Otn3S6/C7774zrq6uZtWqVXa3jxs3zkgya9asuWPNAG6PQ/UAJFO3bl0FBASoQIECatGihbJly6Y5c+YoKChIs2fPVmJiopo1a6YzZ87YLoGBgSpWrFiyQ+S8vLzUrl07y33+9NNPSkxMVL9+/eTqav/WdPO39bdyc3OTp6enpP8Obzt37pxu3LihihUravPmzbbtcuTIoUuXLt3xsLscOXJo+/bt2rNnj2W9t5o2bZpcXV1t35BLUsuWLTV//ny7w7VmzZolf39/denSJdkYSfdz1qxZcnFxUf/+/W+7TVq8/vrrydqyZMli+/fVq1d15swZPf7445Jke/wSExP1008/qVGjRinOdiXV1KxZM3l7e9ud27Vw4UKdOXNGL7/8cprrzgg3P8dffPFF+fj4aO7cuXrkkUckSefOndOyZcvUrFkzXbhwwfY8P3v2rCIiIrRnz55kq/C9+uqr8vDwsF1/44035O7urt9++81uu8KFCysiIsKubcaMGapRo4Zy5sxp97qqW7euEhISbId2peZ5nNqxkjRv3txupi3psMX9+/dbPo6pqed2qlevbncu05o1a1S1alVJUrVq1bRlyxbbOZFr1qxR5cqVUzxP6dbndY0aNVJVe0pS+/6WNKO0cOFCy/M2q1SpovDwcNv1ggUL6rnnntPChQtthxTe/Dq8fv26zp49q6JFiypHjhx272OzZs1SuXLlks12Sv/3OpwxY4ZKliypEiVK2N2HpEOuUzqMGUDqEJwAJDN27FgtXrxYy5cv144dO7R//37bB709e/bIGKNixYopICDA7vLPP//o1KlTdmMFBQXZgo0kxcbG6sSJE7bLuXPnJP13OKCrq6tKlSrlcL3ffPONypYtazsvKSAgQPPmzbM71+DNN99UaGio6tevr0ceeUTt27fXggUL7MYZNGiQzp8/r9DQUIWFhalnz576+++/U1VD0rlZZ8+e1d69e7V3715VqFBB8fHxmjFjhm27ffv2qXjx4nc8UX3fvn3Knz+/cuXK5eAjcWeFCxdO1nbu3Dl169ZNefPmVZYsWRQQEGDbLunxO336tOLi4myHFd1Ojhw51KhRI/3www+2tilTpigoKMj2oS2zSHqOz5w5U88884zOnDkjLy8v2+179+6VMUZ9+/ZN9jxPCrS3PteLFStmdz1btmzKly9fsvNUUvo77NmzRwsWLEi2r6RVGZP2lZrncWrHSlKwYEG760kh6tbzs1KSmnpu5+bznM6fP287NFKSqlatqhs3bmjDhg06cOCAjh8/nuL5TUnn/91af2pqT0lq398KFy6sqKgoffXVV/L391dERITGjh1r956T5NbnhSSFhobq8uXLOn36tKT/Vv3r16+f7Zw0f39/BQQE6Pz583Zj7tu3z/J1uGfPHm3fvj1Z/UmrEd769weQepzjBCCZSpUq3fY8msTERLm4uGj+/Plyc3NLdvutP0h58zepktStWzd98803tus1a9bUihUr0lzr999/r7Zt26px48bq2bOn8uTJIzc3Nw0dOtTuRP88efIoOjpaCxcu1Pz58zV//nxNmjRJkZGRtnqeeOIJ7du3Tz///LMWLVqkr776SqNGjdK4ceNs52GkZM+ePbaT6VP6kDRlyhS9+uqrab6PKbndzNOtJ8Xf7Na/hfTfLNHatWvVs2dPlS9fXtmyZVNiYqKefvrpNP0OVWRkpGbMmKG1a9cqLCxMc+fO1ZtvvplsFtHZbn6ON27cWNWrV9dLL72kXbt22R4DSXr77beTzQ4lKVq0aJr2ndLfITExUfXq1VOvXr1S7JP0oTc1z+PUjpUkpdexJNt5cHeSmnpuJykIrV692nYOT5UqVSRJ/v7+KlasmFavXm1blCGl4HS72tPKkfe3ESNGqG3btrb3i65du2ro0KH6448/bDOXqdWlSxdNmjRJ3bt3V5UqVZQ9e3a5uLioRYsWDr8OExMTFRYWppEjR6Z4e4ECBRwaD8D/ITgBcEhISIiMMSpcuHCafk+lV69edodtJX27HRISosTERO3YscO2oENqzJw5U0WKFNHs2bPtwkRKh7l5enqqUaNGatSokRITE/Xmm2/qf//7n/r27Wv7EJwrVy61a9dO7dq108WLF/XEE09owIABdwxOU6ZMkYeHh7777rtkH7ZWr16tMWPG6NChQypYsKBCQkK0fv16Xb9+3e6wrpuFhIRo4cKFOnfu3G1nnZIet1t/wPPgwYO3rfNW//77r5YuXaqBAweqX79+tvZbD1UMCAiQn5+ftm3bZjnm008/rYCAAE2ZMkWVK1fW5cuX1bp161TX5AxJQbt27dr6/PPP1bt3bxUpUkSS5OHhkerf4tqzZ49q165tu37x4kUdP35czzzzjGXfkJAQXbx4MVX7snoeOzJWat3pENHUvK5SkidPHls48vHxUalSpewWlqlatarWrFmjI0eOyM3NzRaqMvL+OPr+FhYWprCwML3//vtau3atqlWrpnHjxumDDz6wbZPSob+7d+9W1qxZbbNlM2fOVJs2bTRixAjbNlevXk32+g4JCbF8HYaEhOivv/5SnTp17urQXgDJZa6vAAFkei+88ILc3Nw0cODAZN9IG2NSXMb6ZqVKlVLdunVtl6Rj/xs3bixXV1cNGjQo2Tesd/rmOymo3LzN+vXrtW7dOrvtbq3L1dVVZcuWlSTbEr23bpMtWzYVLVrUcgnfKVOmqEaNGmrevLlefPFFu0vSssdJS3E3adJEZ86c0eeff55snKT70KRJExljNHDgwNtu4+fnJ39//2Tnq3zxxRd3rPVmKT12kpKtuubq6qrGjRvrl19+sS2HnlJNkuTu7q6WLVvqxx9/1OTJkxUWFmZ7nI8fP66dO3fq+vXrqa7xXqlVq5YqVaqk0aNH6+rVq8qTJ49q1aql//3vfzp+/Hiy7ZMOsbrZ+PHj7e7bl19+qRs3bqh+/fqW+2/WrJnWrVunhQsXJrvt/PnzunHjhqTUPY9TO5YjfHx8bP1vlpp67qR69eqKjo7WokWLbOc3JalatarWrVunVatWqWzZsvL19XW47tu53f1J7ftbXFxcsscxLCxMrq6uye73unXr7M5TOnz4sH7++Wc99dRTttegm5tbsv199tlnyWaQmzRpor/++ktz5sxJdp+S+jdr1kxHjx7VhAkTkm1z5cqVh/433IC7wYwTAIeEhITogw8+UJ8+fRQTE6PGjRvL19dXBw4c0Jw5c/Tqq6/q7bffdnjcokWL6r333tPgwYNVo0YNvfDCC/Ly8tLGjRuVP39+DR06NMV+DRs21OzZs/X888+rQYMGOnDggMaNG6dSpUrp4sWLtu06dOigc+fO6cknn9QjjzyigwcP6rPPPlP58uVVsmRJSf+Fulq1aik8PFy5cuXSn3/+qZkzZ6pz5863rXv9+vXau3fvbbcJCgrSo48+qilTpuidd95RZGSkvv32W0VFRWnDhg2qUaOGLl26pCVLlujNN9/Uc889p9q1a6t169YaM2aM9uzZYztsbtWqVapdu7ZtXx06dNCwYcPUoUMHVaxYUStXrtTu3btT/Zj7+fnpiSee0PDhw3X9+nUFBQVp0aJFdr/XlWTIkCFatGiRatasaVvi+Pjx45oxY4ZWr15tN1MQGRmpMWPGaPny5bYlqiWpT58++uabb3TgwAEFBwenus57pWfPnmratKkmT56s119/XWPHjlX16tUVFhamjh07qkiRIjp58qTWrVunI0eO6K+//rLrHx8frzp16qhZs2batWuXvvjiC1WvXl3PPvtsqvY9d+5cNWzYUG3btlV4eLguXbqkrVu3aubMmYqJiZG/v3+qnsepHcsRSV9wdO3aVREREXJzc1OLFi1SVc+dVK9eXZMmTdLGjRvVqVMnu9uqVq2q2NhYxcbGpriYyt1Iuj/vvfeeWrRoIQ8PDzVq1CjV72/Lli1T586d1bRpU4WGhurGjRu2GeebF4iRpDJlyigiIsJuOXJJdl+MNGzYUN99952yZ8+uUqVKad26dVqyZIntJyCS9OzZUzNnzlTTpk3Vvn17hYeH69y5c5o7d67GjRuncuXKqXXr1vrxxx/1+uuva/ny5apWrZoSEhK0c+dO/fjjj7bfEAOQBvd4FT8AmVjSssopLTl9q1mzZpnq1asbHx8f4+PjY0qUKGE6depkdu3aZdumZs2apnTp0g7VMHHiRFOhQgXj5eVlcubMaWrWrGkWL15sN+bNy5EnJiaaIUOGmEKFChkvLy9ToUIF8+uvv5o2bdqYQoUK2babOXOmeeqpp0yePHmMp6enKViwoHnttdfM8ePHbdt88MEHplKlSiZHjhwmS5YspkSJEubDDz+0W2L6Vl26dDGSzL59+267zYABA4wk89dffxlj/lt6+L333jOFCxc2Hh4eJjAw0Lz44ot2Y9y4ccN8/PHHpkSJEsbT09MEBASY+vXrm02bNtm2uXz5snnllVdM9uzZja+vr2nWrJk5derUbZcjP336dLLajhw5Yp5//nmTI0cOkz17dtO0aVNz7NixZGMY898yypGRkbalu4sUKWI6deqUbDl4Y4wpXbq0cXV1NUeOHLG1JS0ffesS0PfSnZ7jCQkJJiQkxISEhNiW1963b5+JjIw0gYGBxsPDwwQFBZmGDRuamTNnJhvz999/N6+++qrJmTOnyZYtm2nVqpU5e/as3T4KFSp02yXvL1y4YPr06WOKFi1qPD09jb+/v6latar55JNPbM/B1DyPUztW0nLkH3/8cbJabv3737hxw3Tp0sUEBAQYFxcX29Lkqa3ndnbt2mVbsvvW5dMTExNNjhw5jCQzffr0ZH3btGljfHx8krUnPd/vdH+M+W/Z9qCgIOPq6prseWn1/rZ//37Tvn17ExISYry9vU2uXLlM7dq1zZIlS5Ltt1OnTub77783xYoVs71HLV++3G67f//917Rr1874+/ubbNmymYiICLNz505TqFChZEv5nz171nTu3NkEBQUZT09P88gjj5g2bdqYM2fO2LaJj483H330kSldurTtvTQ8PNwMHDjQxMbGJnvMAKSOizGpOPsTAAAHVKhQQbly5dLSpUudXUqGmzx5stq1a6eNGzfyTT7suLi4qFOnTikemgvg/sM5TgCAdPXnn38qOjpakZGRzi4FAIB0wzlOAIB0sW3bNm3atEkjRoxQvnz51Lx587se88qVKyn+Ns7NcuXKZfdbYQAAZARmnAAA6WLmzJlq166drl+/rqlTp8rb2/uux5w+fbry5ct3x8vatWvToXoAAO6Mc5wAAJnW8ePHtX379jtuEx4ebvtdKwAAMgrBCQAAAAAscKgeAAAAAFh46BaHSExM1LFjx+Tr6ysXFxdnlwMAAADASYwxunDhgvLnzy9X1zvPKT10wenYsWMqUKCAs8sAAAAAkEkcPnxYjzzyyB23eeiCk6+vr6T/Hhw/Pz8nVwMAAADAWeLi4lSgQAFbRriThy44JR2e5+fnR3ACAAAAkKpTeFgcAgAAAAAsEJwAAAAAwALBCQAAAAAsEJwAAAAAwALBCQAAAAAsEJwAAAAAwALBCQAAAAAsEJwAAAAAwALBCQAAAAAsEJwAAAAAwALBCQAAAAAsODU4rVy5Uo0aNVL+/Pnl4uKin376ybLPihUr9Oijj8rLy0tFixbV5MmTM7xOAAAAAA83pwanS5cuqVy5cho7dmyqtj9w4IAaNGig2rVrKzo6Wt27d1eHDh20cOHCDK4UAAAAwMPM3Zk7r1+/vurXr5/q7ceNG6fChQtrxIgRkqSSJUtq9erVGjVqlCIiIjKqTAAAAAAPufvqHKd169apbt26dm0RERFat27dbftcu3ZNcXFxdhcAAAAAcIRTZ5wcdeLECeXNm9euLW/evIqLi9OVK1eUJUuWZH2GDh2qgQMH3qsS0yS49zyH+8QMa5ABlQAAAABIyX0145QWffr0UWxsrO1y+PBhZ5cEAAAA4D5zX804BQYG6uTJk3ZtJ0+elJ+fX4qzTZLk5eUlLy+ve1EeAAAAgAfUfTXjVKVKFS1dutSubfHixapSpYqTKgIAAADwMHBqcLp48aKio6MVHR0t6b/lxqOjo3Xo0CFJ/x1mFxkZadv+9ddf1/79+9WrVy/t3LlTX3zxhX788Uf16NHDGeUDAAAAeEg4NTj9+eefqlChgipUqCBJioqKUoUKFdSvXz9J0vHjx20hSpIKFy6sefPmafHixSpXrpxGjBihr776iqXIAQAAAGQoF2OMcXYR91JcXJyyZ8+u2NhY+fn5ObscSayqBwAAADiDI9ngvjrHCQAAAACcgeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABbcnV0A7l5w73lp6hczrEE6VwIAAAA8mJhxAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALTg9OY8eOVXBwsLy9vVW5cmVt2LDhjtuPHj1axYsXV5YsWVSgQAH16NFDV69evUfVAgAAAHgYOTU4TZ8+XVFRUerfv782b96scuXKKSIiQqdOnUpx+x9++EG9e/dW//799c8//+jrr7/W9OnT9e67797jygEAAAA8TJwanEaOHKmOHTuqXbt2KlWqlMaNG6esWbNq4sSJKW6/du1aVatWTS+99JKCg4P11FNPqWXLlpazVAAAAABwN5wWnOLj47Vp0ybVrVv3/4pxdVXdunW1bt26FPtUrVpVmzZtsgWl/fv367ffftMzzzxz2/1cu3ZNcXFxdhcAAAAAcIS7s3Z85swZJSQkKG/evHbtefPm1c6dO1Ps89JLL+nMmTOqXr26jDG6ceOGXn/99Tseqjd06FANHDgwXWsHAAAA8HBx+uIQjlixYoWGDBmiL774Qps3b9bs2bM1b948DR48+LZ9+vTpo9jYWNvl8OHD97BiAAAAAA8Cp804+fv7y83NTSdPnrRrP3nypAIDA1Ps07dvX7Vu3VodOnSQJIWFhenSpUt69dVX9d5778nVNXkO9PLykpeXV/rfAQAAAAAPDafNOHl6eio8PFxLly61tSUmJmrp0qWqUqVKin0uX76cLBy5ublJkowxGVcsAAAAgIea02acJCkqKkpt2rRRxYoVValSJY0ePVqXLl1Su3btJEmRkZEKCgrS0KFDJUmNGjXSyJEjVaFCBVWuXFl79+5V37591ahRI1uAAgAAAID05tTg1Lx5c50+fVr9+vXTiRMnVL58eS1YsMC2YMShQ4fsZpjef/99ubi46P3339fRo0cVEBCgRo0a6cMPP3TWXQAAAADwEHAxD9kxbnFxccqePbtiY2Pl5+fn7HIkScG95zncJ2ZYg7vqf+sYAAAAwMPGkWxwX62qBwAAAADOQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACw4O7sApA5BPeel6Z+McMapHMlAAAAQObDjBMAAAAAWGDGCemGWSsAAAA8qJhxAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsODu7AKAmwX3nudwn5hhDTKgEgAAAOD/MOMEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggR/AxQOHH9EFAABAemPGCQAAAAAsEJwAAAAAwAKH6gEp4HA/AAAA3IwZJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACw4O7sAoAHUXDveWnqFzOsQTpXAgAAgPTAjBMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFpwensWPHKjg4WN7e3qpcubI2bNhwx+3Pnz+vTp06KV++fPLy8lJoaKh+++23e1QtAAAAgIeRuzN3Pn36dEVFRWncuHGqXLmyRo8erYiICO3atUt58uRJtn18fLzq1aunPHnyaObMmQoKCtLBgweVI0eOe188AAAAgIeGU4PTyJEj1bFjR7Vr106SNG7cOM2bN08TJ05U7969k20/ceJEnTt3TmvXrpWHh4ckKTg4+F6WDAAAAOAh5HBwCg4OVvv27dW2bVsVLFgwzTuOj4/Xpk2b1KdPH1ubq6ur6tatq3Xr1qXYZ+7cuapSpYo6deqkn3/+WQEBAXrppZf0zjvvyM3NLcU+165d07Vr12zX4+Li0lwzcC8F956Xpn4xwxqkcyUAAABw+Byn7t27a/bs2SpSpIjq1aunadOm2QWT1Dpz5owSEhKUN29eu/a8efPqxIkTKfbZv3+/Zs6cqYSEBP3222/q27evRowYoQ8++OC2+xk6dKiyZ89uuxQoUMDhWgEAAAA83NIUnKKjo7VhwwaVLFlSXbp0Ub58+dS5c2dt3rw5I2q0SUxMVJ48eTR+/HiFh4erefPmeu+99zRu3Ljb9unTp49iY2Ntl8OHD2dojQAAAAAePGleVe/RRx/VmDFjdOzYMfXv319fffWVHnvsMZUvX14TJ06UMeaO/f39/eXm5qaTJ0/atZ88eVKBgYEp9smXL59CQ0PtDssrWbKkTpw4ofj4+BT7eHl5yc/Pz+4CAAAAAI5Ic3C6fv26fvzxRz377LN66623VLFiRX311Vdq0qSJ3n33XbVq1eqO/T09PRUeHq6lS5fa2hITE7V06VJVqVIlxT7VqlXT3r17lZiYaGvbvXu38uXLJ09Pz7TeFQAAAAC4I4cXh9i8ebMmTZqkqVOnytXVVZGRkRo1apRKlChh2+b555/XY489ZjlWVFSU2rRpo4oVK6pSpUoaPXq0Ll26ZFtlLzIyUkFBQRo6dKgk6Y033tDnn3+ubt26qUuXLtqzZ4+GDBmirl27Ono3AAAAACDVHA5Ojz32mOrVq6cvv/xSjRs3ti0LfrPChQurRYsWlmM1b95cp0+fVr9+/XTixAmVL19eCxYssC0YcejQIbm6/t+kWIECBbRw4UL16NFDZcuWVVBQkLp166Z33nnH0bsBAAAAAKnmcHDav3+/ChUqdMdtfHx8NGnSpFSN17lzZ3Xu3DnF21asWJGsrUqVKvrjjz9SNTYAAAAApAeHz3E6deqU1q9fn6x9/fr1+vPPP9OlKAAAAADITByecerUqZN69eqlypUr27UfPXpUH330UYqhCoBz8CO6AAAA6cPhGacdO3bo0UcfTdZeoUIF7dixI12KAgAAAIDMxOHg5OXlley3lyTp+PHjcnd3eAILAAAAADI9h4PTU089pT59+ig2NtbWdv78eb377ruqV69euhYHAAAAAJmBw1NEn3zyiZ544gkVKlRIFSpUkCRFR0crb968+u6779K9QAAAAABwNoeDU1BQkP7++29NmTJFf/31l7JkyaJ27dqpZcuWKf6mEwAAAADc79J0UpKPj49effXV9K4FAAAAADKlNK/msGPHDh06dEjx8fF27c8+++xdFwUAAAAAmYnDwWn//v16/vnntXXrVrm4uMgYI0lycXGRJCUkJKRvhQAAAADgZA6vqtetWzcVLlxYp06dUtasWbV9+3atXLlSFStW1IoVKzKgRAAAAABwLodnnNatW6dly5bJ399frq6ucnV1VfXq1TV06FB17dpVW7ZsyYg6AQAAAMBpHJ5xSkhIkK+vryTJ399fx44dkyQVKlRIu3btSt/qAAAAACATcHjGqUyZMvrrr79UuHBhVa5cWcOHD5enp6fGjx+vIkWKZESNAAAAAOBUDgen999/X5cuXZIkDRo0SA0bNlSNGjWUO3duTZ8+Pd0LBAAAAABnczg4RURE2P5dtGhR7dy5U+fOnVPOnDltK+sBAAAAwIPEoXOcrl+/Lnd3d23bts2uPVeuXIQmAAAAAA8sh4KTh4eHChYsyG81AQAAAHioOLyq3nvvvad3331X586dy4h6AAAAACDTcfgcp88//1x79+5V/vz5VahQIfn4+Njdvnnz5nQrDgAAAAAyA4eDU+PGjTOgDAAAAADIvBwOTv3798+IOgAAAAAg03L4HCcAAAAAeNg4POPk6up6x6XHWXEPAAAAwIPG4eA0Z84cu+vXr1/Xli1b9M0332jgwIHpVhgAAAAAZBYOB6fnnnsuWduLL76o0qVLa/r06XrllVfSpTAAmUNw73lp6hczrEE6VwIAAOA86XaO0+OPP66lS5em13AAAAAAkGmkS3C6cuWKxowZo6CgoPQYDgAAAAAyFYcP1cuZM6fd4hDGGF24cEFZs2bV999/n67FAQAAAEBm4HBwGjVqlF1wcnV1VUBAgCpXrqycOXOma3EAAAAAkBk4HJzatm2bAWUAAAAAQObl8DlOkyZN0owZM5K1z5gxQ9988026FAUAAAAAmYnDwWno0KHy9/dP1p4nTx4NGTIkXYoCAAAAgMzE4UP1Dh06pMKFCydrL1SokA4dOpQuRQF4sKTlt6D4HSgAAJCZODzjlCdPHv3999/J2v/66y/lzp07XYoCAAAAgMzE4eDUsmVLde3aVcuXL1dCQoISEhK0bNkydevWTS1atMiIGgEAAADAqRw+VG/w4MGKiYlRnTp15O7+X/fExERFRkZyjhMAAACAB5LDwcnT01PTp0/XBx98oOjoaGXJkkVhYWEqVKhQRtQHAAAAAE7ncHBKUqxYMRUrViw9awEAAACATMnhc5yaNGmijz76KFn78OHD1bRp03QpCgAAAAAyE4eD08qVK/XMM88ka69fv75WrlyZLkUBAAAAQGbicHC6ePGiPD09k7V7eHgoLi4uXYoCAAAAgMzE4eAUFham6dOnJ2ufNm2aSpUqlS5FAQAAAEBm4vDiEH379tULL7ygffv26cknn5QkLV26VD/88INmzpyZ7gUCgCQF957ncJ+YYQ0yoBIAAPAwcjg4NWrUSD/99JOGDBmimTNnKkuWLCpXrpyWLVumXLlyZUSNAAAAAOBUaVqOvEGDBmrQ4L9vcuPi4jR16lS9/fbb2rRpkxISEtK1QABIL8xaAQCAtHL4HKckK1euVJs2bZQ/f36NGDFCTz75pP7444/0rA0AAAAAMgWHZpxOnDihyZMn6+uvv1ZcXJyaNWuma9eu6aeffmJhCAAAAAAPrFQHp0aNGmnlypVq0KCBRo8eraefflpubm4aN25cRtYHAJlGWg71kzjcDwCAB0Gqg9P8+fPVtWtXvfHGGypWrFhG1gQAAAAAmUqqz3FavXq1Lly4oPDwcFWuXFmff/65zpw5k5G1AQAAAECmkOoZp8cff1yPP/64Ro8erenTp2vixImKiopSYmKiFi9erAIFCsjX1zcjawWA+x6H+wEAcH9yeFU9Hx8ftW/fXqtXr9bWrVv11ltvadiwYcqTJ4+effbZjKgRAAAAAJwqzcuRS1Lx4sU1fPhwHTlyRFOnTk2vmgAAAAAgU7mr4JTEzc1NjRs31ty5c9NjOAAAAADIVNIlOAEAAADAg4zgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAW3J1dAADAMcG956WpX8ywBulcCQAAD49MMeM0duxYBQcHy9vbW5UrV9aGDRtS1W/atGlycXFR48aNM7ZAAAAAAA81pwen6dOnKyoqSv3799fmzZtVrlw5RURE6NSpU3fsFxMTo7fffls1atS4R5UCAAAAeFg5PTiNHDlSHTt2VLt27VSqVCmNGzdOWbNm1cSJE2/bJyEhQa1atdLAgQNVpEiRe1gtAAAAgIeRU4NTfHy8Nm3apLp169raXF1dVbduXa1bt+62/QYNGqQ8efLolVdesdzHtWvXFBcXZ3cBAAAAAEc4NTidOXNGCQkJyps3r1173rx5deLEiRT7rF69Wl9//bUmTJiQqn0MHTpU2bNnt10KFChw13UDAAAAeLg4/VA9R1y4cEGtW7fWhAkT5O/vn6o+ffr0UWxsrO1y+PDhDK4SAAAAwIPGqcuR+/v7y83NTSdPnrRrP3nypAIDA5Ntv2/fPsXExKhRo0a2tsTEREmSu7u7du3apZCQELs+Xl5e8vLyyoDqAQAAADwsnDrj5OnpqfDwcC1dutTWlpiYqKVLl6pKlSrJti9RooS2bt2q6Oho2+XZZ59V7dq1FR0dzWF4AAAAADKE038ANyoqSm3atFHFihVVqVIljR49WpcuXVK7du0kSZGRkQoKCtLQoUPl7e2tMmXK2PXPkSOHJCVrBwAAAID04vTg1Lx5c50+fVr9+vXTiRMnVL58eS1YsMC2YMShQ4fk6npfnYoFAAAA4AHj9OAkSZ07d1bnzp1TvG3FihV37Dt58uT0LwgAHnDBveelqV/MsAbpXAkAAPcHpnIAAAAAwEKmmHECANx/0jJrxYwVAOB+xYwTAAAAAFggOAEAAACABYITAAAAAFjgHCcAgNNwnhQA4H7BjBMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFd2cXAADA3QjuPc/hPjHDGmRAJQCABxkzTgAAAABggeAEAAAAABYITgAAAABggXOcAAAPtbScIyVxnhQAPGyYcQIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAAC+7OLgAAgPtdcO95aeoXM6xBOlcCAMgozDgBAAAAgAVmnAAAyASYtQKAzI0ZJwAAAACwwIwTAAAPCGatACDjMOMEAAAAABaYcQIAADZpmbVixgrAw4AZJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwkCmC09ixYxUcHCxvb29VrlxZGzZsuO22EyZMUI0aNZQzZ07lzJlTdevWveP2AAAAAHC3nB6cpk+frqioKPXv31+bN29WuXLlFBERoVOnTqW4/YoVK9SyZUstX75c69atU4ECBfTUU0/p6NGj97hyAAAAAA8LpwenkSNHqmPHjmrXrp1KlSqlcePGKWvWrJo4cWKK20+ZMkVvvvmmypcvrxIlSuirr75SYmKili5deo8rBwAAAPCwcGpwio+P16ZNm1S3bl1bm6urq+rWrat169alaozLly/r+vXrypUrV4q3X7t2TXFxcXYXAAAAAHCEU4PTmTNnlJCQoLx589q1582bVydOnEjVGO+8847y589vF75uNnToUGXPnt12KVCgwF3XDQAAAODh4vRD9e7GsGHDNG3aNM2ZM0fe3t4pbtOnTx/FxsbaLocPH77HVQIAAAC437k7c+f+/v5yc3PTyZMn7dpPnjypwMDAO/b95JNPNGzYMC1ZskRly5a97XZeXl7y8vJKl3oBAAAAPJycOuPk6emp8PBwu4UdkhZ6qFKlym37DR8+XIMHD9aCBQtUsWLFe1EqAAAAgIeYU2ecJCkqKkpt2rRRxYoVValSJY0ePVqXLl1Su3btJEmRkZEKCgrS0KFDJUkfffSR+vXrpx9++EHBwcG2c6GyZcumbNmyOe1+AAAAAHhwOT04NW/eXKdPn1a/fv104sQJlS9fXgsWLLAtGHHo0CG5uv7fxNiXX36p+Ph4vfjii3bj9O/fXwMGDLiXpQMAAAB4SDg9OElS586d1blz5xRvW7Fihd31mJiYjC8IAAAAAG5yX6+qBwAAAAD3QqaYcQIAAA+O4N7zHO4TM6xBBlQCAOmHGScAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAAL7s4uAAAA4FbBvec53CdmWIMMqAQA/sOMEwAAAABYIDgBAAAAgAWCEwAAAABYIDgBAAAAgAWCEwAAAABYIDgBAAAAgAWCEwAAAABYIDgBAAAAgAWCEwAAAABYIDgBAAAAgAWCEwAAAABYcHd2AQAAAOktuPe8NPWLGdYgnSsB8KBgxgkAAAAALBCcAAAAAMACwQkAAAAALHCOEwAAQAo4TwrAzZhxAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAAL7s4uAAAA4EEV3HtemvrFDGuQzpUAuFvMOAEAAACABWacAAAAMrG0zFoxYwWkP4ITAADAA47wBdw9ghMAAAAsEb7wsMsU5ziNHTtWwcHB8vb2VuXKlbVhw4Y7bj9jxgyVKFFC3t7eCgsL02+//XaPKgUAAADwMHJ6cJo+fbqioqLUv39/bd68WeXKlVNERIROnTqV4vZr165Vy5Yt9corr2jLli1q3LixGjdurG3btt3jygEAAAA8LJwenEaOHKmOHTuqXbt2KlWqlMaNG6esWbNq4sSJKW7/6aef6umnn1bPnj1VsmRJDR48WI8++qg+//zze1w5AAAAgIeFU89xio+P16ZNm9SnTx9bm6urq+rWrat169al2GfdunWKioqya4uIiNBPP/2U4vbXrl3TtWvXbNdjY2MlSXFxcXdZffpJvHbZ4T4315+W/g/SGLf+LTPDGPfz45lZxuDv+mCOwd/1wRyDv2vmGyMz/k2AzCjpOWqMsd7YONHRo0eNJLN27Vq79p49e5pKlSql2MfDw8P88MMPdm1jx441efLkSXH7/v37G0lcuHDhwoULFy5cuHDhkuLl8OHDltnlgV9Vr0+fPnYzVImJiTp37pxy584tFxcXJ1Z2Z3FxcSpQoIAOHz4sPz8/xsgEY2SGGhgj/cfIDDUwRuargTHSf4zMUANjpP8YmaEGxkj/MTJDDfeKMUYXLlxQ/vz5Lbd1anDy9/eXm5ubTp48add+8uRJBQYGptgnMDDQoe29vLzk5eVl15YjR460F32P+fn53fWTjTHSd4zMUANjpP8YmaEGxsh8NTBG+o+RGWpgjPQfIzPUwBjpP0ZmqOFeyJ49e6q2c+riEJ6engoPD9fSpUttbYmJiVq6dKmqVKmSYp8qVarYbS9Jixcvvu32AAAAAHC3nH6oXlRUlNq0aaOKFSuqUqVKGj16tC5duqR27dpJkiIjIxUUFKShQ4dKkrp166aaNWtqxIgRatCggaZNm6Y///xT48ePd+bdAAAAAPAAc3pwat68uU6fPq1+/frpxIkTKl++vBYsWKC8efNKkg4dOiRX1/+bGKtatap++OEHvf/++3r33XdVrFgx/fTTTypTpoyz7kKG8PLyUv/+/ZMdZsgYzhsjM9TAGOk/RmaogTEyXw2Mkf5jZIYaGCP9x8gMNTBG+o+RGWrIjFyMSc3aewAAAADw8HL6D+ACAAAAQGZHcAIAAAAACwQnAAAAALBAcMpkrl+/rvHjx6tu3boKCgpSYGCgqlatqk8++USXL192dnkOqVWrlrp37+7sMjI1HqMHgzFGr776qnLlyiUXFxdFR0enum96Pgd4Ptnj8cDNMuPzoW3btmrcuPFd9Qdw7xCcMpH9+/fr0Ucf1dixY/Xiiy9qxowZWrRokbp3766lS5eqdOnS2r17t7PLfGhlxv90kTksWLBAkydP1q+//qrjx48/cKt83q9mz56twYMHO7sM3ORBeR9dsWKFXFxcdPDgwbsa59NPP9XkyZPTp6iH2LBhw1S6dGllzZpVoaGh+uGHH9I81tmzZ5UnTx7FxMQ43LdFixYaMWJEmvedWRw6dEjBwcEyxujHH39Us2bNHB4j6TUSExOjAQMGKDg4OP0LdQKCUyYRGxuriIgIPf/884qOjtbrr7+uqlWrqmzZsmrWrJnmz5+vd999V0899ZT+/fdfZ5ebqRUvXlw///yzs8t4qMTHxzu7BKfat2+f8uXLp6pVqyowMFDu7k7/pQdIypUrl3x9fZ1dBh5Au3fvVqlSpVSgQIG7Gid79uzKkSOHQ33OnDmjNm3aqGDBgpo6daqKFi2qpk2bPtTvw6tWrdKoUaO0bds2vfzyy4qMjNT+/fvTNNaHH36o5557Lk0f9N9//319+OGHio2NTdO+M4v3339ftWrV0o0bN/Tuu+/qmWeecXiMmJgYFS1aVEFBQTpw4IBq1aqV/oU6AcEpkxg2bJjCw8M1aNAgxcbGqlWrVrbD9MaMGaP69eurY8eOqlGjhkaPHp3h9SQmJmro0KEqXLiwsmTJonLlymnmzJlpGqdXr17KlSuXAgMDNWDAAIf616pVS127dnVojOeee05z5851uNY7adu2rX7//Xd9+umncnFxsX2L4ohLly4pMjJS2bJlU758+Rz+VurChQtq1aqVfHx8lC9fPo0aNSpN397OnDlTYWFhypIli3Lnzq26devq0qVLDo1Rq1Ytde7cWd27d5e/v78iIiIc6n/t2jV17dpVefLkkbe3t6pXr66NGzemun9wcHCy10H58uUden6NHz9e+fPnV2Jiol37c889p/bt26d6nLZt26pLly46dOiQXFxc7vpbtXnz5il79uyaMmXKXY1zr3377bfKnTu3rl27ZtfeuHFjtW7dOtXjLFiwQNWrV1eOHDmUO3duNWzYUPv27UtTTWmd3YiJibG9zm++OPIff9JrpHPnzsqePbv8/f3Vt29fWf0CSK1atdSlSxd1795dOXPmVN68eTVhwgTbD8P7+vqqaNGimj9/vsP3624kJiZq+PDhKlq0qLy8vFSwYEF9+OGHGb7fX3/9VTly5FBCQoIkKTo6Wi4uLurdu7dtmw4dOujll19O1Xg3btxw+G+SkoULF2rw4MF2vzOZFmk5VK9Hjx76448/9N133+mZZ57RhAkTVKRIkWTvZVZS+j/JkdfM6dOnFRgYqCFDhtja1q5dK09PTy1dutShWu7WvHnz9NRTT6lIkSLq3LmzEhISdOzYMYfHuXz5sr7++mu98soraaqjTJkyCgkJ0ffff5+m/pnFypUrFRwcrKNHj2rfvn1p+n9twYIFGjJkiNzd3bVixYoHZ/bfIFMICgoyW7duNcYY0759e1OlShXzxx9/mLlz55rAwEBTs2ZNY4wxS5YsMZUrV87wej744ANTokQJs2DBArNv3z4zadIk4+XlZVasWJHqMWrWrGn8/PzMgAEDzO7du80333xjXFxczKJFizJ0jFWrVpm8efOahIQEW9uBAweMJDNr1ixTq1YtkyVLFlO2bFmzdu3aVNVx/vx5U6VKFdOxY0dz/Phxc/z4cXPjxo1U3w9jjHnjjTdMwYIFzZIlS8zff/9tGjZsaHx9fU23bt1S1b9Dhw6mUKFCZsmSJWbr1q3m+eefd6i/McYcO3bMuLu7m5EjR5oDBw6Yv//+24wdO9ZcuHDBoftSs2ZNky1bNtOzZ0+zc+dOs3PnTof6d+3a1eTPn9/89ttvZvv27aZNmzYmZ86c5uzZs6nqX6hQITNq1Ci7tnLlypn+/funuoZz584ZT09Ps2TJElvb2bNnk7VZOX/+vBk0aJB55JFHzPHjx82pU6dS3deY/x7LpL/hlClTjK+vr/nll18cGiOlse61y5cvm+zZs5sff/zR1nby5Enj7u5uli1blupxZs6caWbNmmX27NljtmzZYho1amTCwsLsXs+pldbH48aNG7bX+fHjx82WLVtM7ty5Td++fR3ad7Zs2Uy3bt3Mzp07zffff2+yZs1qxo8fb9nP19fXDB482OzevdsMHjzYuLm5mfr165vx48eb3bt3mzfeeMPkzp3bXLp0yeH7lla9evUyOXPmNJMnTzZ79+41q1atMhMmTHB4HEf/JufPnzeurq5m48aNxhhjRo8ebfz9/e3+HyxatGiqaknr3yQjtWnTxjz33HMO9SldurT54IMPbP3T6m7/TzLGmHnz5hkPDw+zceNGExcXZ4oUKWJ69OiR5pruVmJiomnbtq0pU6aMuXbtmsP9Z8yYYQICAu6qhoEDB5rq1aunuf+kSZNMenw8T69xYI9HNBM4e/as8fPzs10PCAgwa9assV3/4IMPbMFp+/btJjQ0NEPruXr1qsmaNWuyUPHKK6+Yli1bpnqcmjVrJnvzeOyxx8w777yToWMkJCSYgIAAu/qTglOJEiXMr7/+anbt2mVefPFFU6hQIXP9+vVU15LWD6UXLlwwnp6edh8qz549a7JkyZKqMePi4oyHh4eZMWOGre38+fMma9asDtW0adMmI8nExMQ4Un4yNWvWNBUqVEhT34sXLxoPDw8zZcoUW1t8fLzJnz+/GT58eKrGSI/gZIwxzz33nGnfvr3t+v/+9z+TP39+hz+kjxo1yhQqVMihPkmSnleff/65yZ49u0NfTtxuLGd54403TP369W3XR4wYYYoUKWISExPTPObp06eNJNsXS45Ij8fjypUrpnLlyqZhw4YOPS9q1qxpSpYsaXff33nnHVOyZEnLfje/5924ccP4+PiY1q1b29qOHz9uJJl169alqpbvv//e+Pj42C4rV65M9f0w5r/3Hy8vrzQFpVul5W/y6KOPmo8//tgYY0zjxo3Nhx9+aDw9Pc2FCxfMkSNHjCSze/fuVO07LX+TjJSW4PTqq6+akJAQ88svv6Q5ON3t/0k3e/PNN01oaKh56aWXTFhYmLl69WqaakoP7du3N6GhoebIkSNp6t+1a1fz9NNP31UN8+fPN56enml+HGbPnm2KFy9+VzWk5ziwx6F6mcCNGzfk7e1tux4fHy8fHx/b9WzZstn+vXnzZhUtWjRD69m7d68uX76sevXqKVu2bLbLt99+6/AhM2XLlrW7ni9fPp06dSpDx3B1dVXDhg1TPM/p7bffVoMGDRQaGqqBAwfq4MGD2rt3r0P1pMW+ffsUHx+vypUr29py5cql4sWLp6r//v37df36dVWqVMnWlj179lT3T1KuXDnVqVNHYWFhatq0qSZMmJDmc+bCw8PT1G/fvn26fv26qlWrZmvz8PBQpUqV9M8//6RpzLRq1aqVZs2aZTu8bMqUKWrRosVdH37jqJkzZ6pHjx5avHixataseU/3nZ46duyoRYsW6ejRo5KkyZMnq23btnJxcUn1GHv27FHLli1VpEgR+fn52Q4ROXToUEaUbKl9+/a6cOGCfvjhB4efF48//rjdfa9SpYr27NljO+zsdm5+z3Nzc1Pu3LkVFhZma8ubN68kpfq99Nlnn1V0dLTtUrFiRUfuhv755x9du3ZNderUcahfeqlZs6ZWrFghY4xWrVqlF154QSVLltTq1av1+++/K3/+/CpWrFiqxkrr3yQzGTlypJo3b64ePXro22+/Vfny5TVu3DiHxrjb/5Nu9sknn+jGjRuaMWOGpkyZIi8vL4fHSA8bN27UxIkTNXfuXAUFBaVpjIMHDyp//vx3VUf+/PkVHx+vEydOpKn/888/r507d95VDek5DuwRnDIBf39/xcfH6+TJk5Kk6tWra/jw4bpy5YqOHj2qCRMmSPrv2OH33ntPUVFRGVrPxYsXJf13zPDN/9nu2LHD4fOcPDw87K67uLg4fBx2WsZ49tlnUzzP6eYPJPny5ZOU+g8fDwI3NzctXrxY8+fPV6lSpfTZZ5+pePHiOnDggMNj3Rzu7zVXV9dk5yVcv37d4XEaNWokY4zmzZunw4cPa9WqVWrVqlV6lZlqFSpUUEBAgCZOnJim8y0yiwoVKqhcuXL69ttvtWnTJm3fvt3h5ZIbNWqkc+fOacKECVq/fr3Wr18vyTkLkHzwwQdauHCh5s6de08XmUjpPe/mtqQP/ql9L006LyrpkiVLFofqcXT79FarVi2tXr1af/31lzw8PFSiRAnVqlVLK1as0O+//35ff9mQFj4+Pvrwww+1Z88ePfvss3rjjTcUFRWl8ePHO6Weffv26dixY0pMTEzTSnTpJemcprSEvyRXrlyx+yI7LZJeL/fbT8ggdQhOmYCrq6ueffZZffHFF5L+W550y5YtypYtm8LCwlSvXj39/vvvat++vT799NMM/9avVKlS8vLy0qFDh+z+sy1atOhdryB0rzz11FOKiYlJNpt0Nx8+PD090/ytZEhIiDw8PGwfAiXp33//TfXy8kWKFJGHh4fdAgqxsbFpWp7excVF1apV08CBA7VlyxZ5enpqzpw5Do+TViEhIfL09NSaNWtsbdevX9fGjRtVqlSpVI0REBCg48eP267HxcWlKfx5e3vrhRde0JQpUzR16lQVL15cjz76qMPj3K2QkBAtX75cP//8s7p06XLP95+eOnTooMmTJ2vSpEmqW7euQ+8ZZ8+e1a5du/T++++rTp06KlmypNNWEZ01a5YGDRqkH3/8USEhIWka4+bXuyT98ccfKlasmNzc3NKjxHumWLFiypIlS7qc8L9ixQqHFziqUaOGLly4oFGjRtlCUlJwWrFihUOLdjwof5MkOXLk0Guvvab69etr1apVqe53t/8nJYmPj9fLL7+s5s2ba/DgwerQoYPTvoysWbOmQ4sMpcTf3/+u33POnTsn6b//p/DgIThlEv369dPnn3+u+fPnKyQkRDt27NDRo0d1/PhxDRkyRKdPn9bOnTvv6ofyUsvX11dvv/22evTooW+++Ub79u3T5s2b9dlnn+mbb77J8P2nh6xZs6pOnTrpurpecHCw1q9fr5iYGJ05c8ahmbNs2bLplVdeUc+ePbVs2TJt27ZNbdu2TfWhP76+vmrTpo169uyp5cuXa/v27XrllVfk6urq0GFQ69ev15AhQ/Tnn3/q0KFDmj17tk6fPq2SJUumeoy75ePjozfeeEM9e/bUggULtGPHDnXs2FGXL19O9UpGTz75pL777jutWrVKW7duVZs2bdL8wadVq1aaN2+eJk6c6JTZpiShoaFavny5Zs2adV//zs1LL72kI0eOaMKECQ6tTihJOXPmVO7cuTV+/Hjt3btXy5Yty/AZ9pRs27ZNkZGReuedd1S6dGmdOHFCJ06csH0gSq1Dhw4pKipKu3bt0tSpU/XZZ5+pW7duGVR1xvH29tY777yjXr162Q7Z/uOPP/T11187PFadOnU0dOhQh/rkzJlTZcuW1ZQpU2wh6YknntDmzZu1e/duh2acHoS/SY8ePfT7778rNjZWCQkJWr58uX7//XeHDp++2/+Tkrz33nuKjY3VmDFj9M477yg0NNTh132Szz///K6+GF6+fHmqV1e8nQoVKmjHjh13Nca2bdv0yCOPyN/f/67GuVtz5sxRiRIlnFrDg4jglEmEhIRo6tSpatWqlbp27aqtW7cqT5488vDw0O7du9WjR497+gFi8ODB6tu3r4YOHaqSJUvq6aef1rx581S4cOF7VsPdeu6559L195zefvttubm5qVSpUgoICHD4nIuPP/5YNWrUUKNGjVS3bl1Vr17dof/oRo4cqSpVqqhhw4aqW7euqlWrppIlSzp0WIGfn59WrlypZ555RqGhoXr//fc1YsQI1a9f36H7creGDRumJk2aqHXr1nr00Ue1d+9eLVy4UDlz5kxV/z59+qhmzZpq2LChGjRooMaNG6d5VuDJJ59Urly5tGvXLr300ktpGiO9FC9eXMuWLdPUqVP11ltvOaWGyZMnOxTGb5U9e3Y1adJE2bJlc/iLHldXV02bNk2bNm1SmTJl1KNHD3388cdpriWt/vzzT12+fFkffPCB8uXLZ7u88MILDo0TGRmpK1euqFKlSurUqZO6deumV199NYOqzlh9+/bVW2+9pX79+qlkyZJq3rx5mmYW9u3bZzss3RE1a9ZUQkKCLTjlypVLpUqVUmBgoEOHZj0If5OCBQsqKipKBQoU0A8//KDIyEi1b9/e4dnqu/0/KWn28LvvvpOfn59cXV1tX2h9+eWXjt4tnTlzJs0/PSD9dxTGrl270txfkiIiIrR9+/a7mnVatWqVnnrqqTT3v9v34CTp8XggORdzPx9Q/wA6cOCABg0apDlz5tjONcqTJ4/atGmjPn36yM/Pz8kV3j9OnjypAgUK6MSJE4qLi1PhwoW1ZcsWlS9fXpJ0/vx55cyZU8uXL78vf5jt0qVLCgoK0ogRI9L8mxPArfr376/ff/9dK1asSPMYderUUenSpTVmzJj0K+w+U6tWLZUvX/6e/O4e7l8tW7aUm5tbmn/3p23btpo8eXK61cPzVqpcubLat2+v1157zeG+V69eVWBgoBYsWKDHH388TftPj/dgZBx+3j6TKVy4sCZNmqSvv/5aJ0+elKurq20FpdQ4dOjQHc8T2bFjhwoWLJgepWZ6efPmVXh4uObNm6fWrVsnO+k+R44c99WJ+Fu2bNHOnTtVqVIlxcbGatCgQZL+m1kD0sv8+fP1+eefp6nvv//+azvvJOmcTQDJ3bhxQ7t379a6devS9AEdGadfv37q2bOnOnbs6PChi5MmTVKlSpXSHJqku3sPRsYjOGVSrq6utlXfHJE/f35FR0ff8faHycCBA52yGldG+eSTT7Rr1y55enoqPDxcq1atcvpx1HiwbNiwIc19K1SooH///VcfffTRXa1sBTzotm3bpqpVq6p27dp6/fXX0zxOes424T8NGjTQnj17dPToUYcXxPLw8NBnn312V/u/m/dgZDwO1QMAAAAACywOAQAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AgDRzcXHRTz/95LT979q1S4GBgbpw4UKqtq9Vq5a6d++esUVlEr1791aXLl2cXQYAPDAITgCAFJ04cUJdunRRkSJF5OXlpQIFCqhRo0ZaunSps0uz6dOnj7p06SJfX19nl3LXJk+erBw5cqTbeG+//ba++eYb7d+/P93GBICHGcEJAJBMTEyMwsPDtWzZMn388cfaunWrFixYoNq1a6tTp07OLk+SdOjQIf36669q27atU+tISEhQYmKiU2u4WVI9/v7+ioiI0JdffunskgDggUBwAgAk8+abb8rFxUUbNmxQkyZNFBoaqtKlSysqKkp//PHHbfu98847Cg0NVdasWVWkSBH17dtX169ft93+119/qXbt2vL19ZWfn5/Cw8P1559/SpIOHjyoRo0aKWfOnPLx8VHp0qX122+/3XZfP/74o8qVK6egoCC79jVr1qhWrVrKmjWrcubMqYiICP3777+22xMTE9WrVy/lypVLgYGBGjBggF3/kSNHKiwsTD4+PipQoIDefPNNXbx40XZ70szQ3LlzVapUKXl5eenQoUPauHGj6tWrJ39/f2XPnl01a9bU5s2b7cY+f/68XnvtNeXNm1fe3t4qU6aMfv31V61YsULt2rVTbGysXFxc5OLiYqvr2rVrevvttxUUFCQfHx9VrlxZK1assKxHkho1aqRp06bd9jEEAKSeu7MLAABkLufOndOCBQv04YcfysfHJ9ntdzqczNfXV5MnT1b+/Pm1detWdezYUb6+vurVq5ckqVWrVqpQoYK+/PJLubm5KTo6Wh4eHpKkTp06KT4+XitXrpSPj4927NihbNmy3XZfq1atUsWKFe3aoqOjVadOHbVv316ffvqp3N3dtXz5ciUkJNi2+eabbxQVFaX169dr3bp1atu2rapVq6Z69epJklxdXTVmzBgVLlxY+/fv15tvvqlevXrpiy++sI1x+fJlffTRR/rqq6+UO3du5cmTR/v371ebNm302WefyRijESNG6JlnntGePXvk6+urxMRE1a9fXxcuXND333+vkJAQ7dixQ25ubqpatapGjx6tfv36adeuXZJku++dO3fWjh07NG3aNOXPn19z5szR008/ra1bt6pYsWK3rUeSKlWqpCNHjigmJkbBwcG3fSwBAKlgAAC4yfr1640kM3v2bMttJZk5c+bc9vaPP/7YhIeH2677+vqayZMnp7htWFiYGTBgQKrrLFeunBk0aJBdW8uWLU21atVu26dmzZqmevXqdm2PPfaYeeedd27bZ8aMGSZ37ty265MmTTKSTHR09B3rS0hIML6+vuaXX34xxhizcOFC4+rqanbt2pXi9pMmTTLZs2e3azt48KBxc3MzR48etWuvU6eO6dOnj2U9sbGxRpJZsWLFHWsFAFhjxgkAYMcYk+a+06dP15gxY7Rv3z5dvHhRN27ckJ+fn+32qKgodejQQd99953q1q2rpk2bKiQkRJLUtWtXvfHGG1q0aJHq1q2rJk2aqGzZsrfd15UrV+Tt7W3XFh0draZNm96xxlvHzJcvn06dOmW7vmTJEg0dOlQ7d+5UXFycbty4oatXr+ry5cvKmjWrJMnT0zPZOCdPntT777+vFStW6NSpU0pISNDly5dth81FR0frkUceUWho6B3ru9nWrVuVkJCQrM+1a9eUO3du2/WU6pGkLFmySPpvRgoAcHc4xwkAYKdYsWJycXHRzp07Heq3bt06tWrVSs8884x+/fVXbdmyRe+9957i4+Nt2wwYMEDbt29XgwYNtGzZMpUqVUpz5syRJHXo0EH79+9X69attXXrVlWsWFGfffbZbffn7+9vd+6S9H9B4U6SDg1M4uLiYlvcISYmRg0bNlTZsmU1a9Ysbdq0SWPHjpUku/uRJUsWubi42I3Tpk0bRUdH69NPP9XatWsVHR2t3Llz2/qlprZbXbx4UW5ubtq0aZOio6Ntl3/++UeffvrpHeuR/jvsUpICAgIc3jcAwB7BCQBgJ1euXIqIiNDYsWN16dKlZLefP38+xX5r165VoUKF9N5776lixYoqVqyYDh48mGy70NBQ9ejRQ4sWLdILL7ygSZMm2W4rUKCAXn/9dc2ePVtvvfWWJkyYcNs6K1SooB07dti1lS1b9q6WS9+0aZMSExM1YsQIPf744woNDdWxY8dS1XfNmjXq2rWrnnnmGZUuXVpeXl46c+aMXW1HjhzR7t27U+zv6elpdy6W9N99TEhI0KlTp1S0aFG7S2BgoGVN27Ztk4eHh0qXLp2q+wAAuD2CEwAgmbFjxyohIUGVKlXSrFmztGfPHv3zzz8aM2aMqlSpkmKfYsWK6dChQ5o2bZr27dunMWPG2GaTpP8OrevcubNWrFihgwcPas2aNdq4caNKliwpSerevbsWLlyoAwcOaPPmzVq+fLnttpRERERo3bp1dmGjT58+2rhxo9588039/fff2rlzp7788ku7AHMnRYsW1fXr1/XZZ59p//79+u677zRu3LhU9S1WrJi+++47/fPPP1q/fr1atWplN8tUs2ZNPfHEE2rSpIkWL16sAwcOaP78+VqwYIEkKTg4WBcvXtTSpUt15swZXb58WaGhoWrVqpUiIyM1e/ZsHThwQBs2bNDQoUM1b948y5pWrVqlGjVqpGm2CwBgj+AEAEimSJEi2rx5s2rXrq233npLZcqUUb169bR06dLb/i7Qs88+qx49eqhz584qX7681q5dq759+9pud3Nz09mzZxUZGanQ0FA1a9ZM9evX18CBAyX99/tDnTp1UsmSJfX0008rNDTUbiW7W9WvX1/u7u5asmSJrS00NFSLFi3SX3/9pUqVKqlKlSr6+eef5e6eulN6y5Urp5EjR+qjjz5SmTJlNGXKFA0dOjRVfb/++mv9+++/evTRR9W6dWt17drVtrpdklmzZumxxx5Ty5YtVapUKfXq1csW/KpWrarXX39dzZs3V0BAgIYPHy5JmjRpkiIjI/XWW2+pePHiaty4sTZu3KiCBQta1jRt2jR17NgxVfUDAO7MxdzNWcAAADjR2LFjNXfuXC1cuNDZpWQ68+fP11tvvaW///471cERAHB7vJMCAO5br732ms6fP68LFy7I19fX2eVkKpcuXdKkSZMITQCQTphxAgAAAAALnOMEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABYITgAAAABggeAEAAAAABb+HwNNyISeq9WMAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1000x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Compute how many predictions of each character were actually correct    \n",
    "pct_correct_by_char = {\n",
    "    vectorizer.int_to_token(i):pct for i,pct in pct_correct_by_idx.items()\n",
    "}\n",
    "\n",
    "labels = list(sorted(pct_correct_by_char, key=pct_correct_by_char.get, reverse=True))\n",
    "pct_correct = [pct_correct_by_char[char] for char in labels]\n",
    "labels[labels.index(\" \")] = \"_\"\n",
    "labels[labels.index(\"\\n\")] = \"\\\\n\"\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10, 5))\n",
    "ax.bar(labels, pct_correct)\n",
    "ax.set_title(\"Per-class Accuracy. '_' Represents Whitespace\")\n",
    "ax.set_xlabel(\"Class (character)\")\n",
    "ax.set_ylabel(\"Accuracy\")\n",
    "\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.6 Generate a poem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_poem(model, seed, vectorizer, max_length):\n",
    "    \"\"\"Generate a poem using a LSTM neural language model\n",
    "    \n",
    "    Parameters:\n",
    "        model (keras.Sequential):\n",
    "            Trained LSTM neural network\n",
    "        seed (str):\n",
    "            The beginning of a poem, to get the model started\n",
    "        vectorizer (Vectorizer):\n",
    "            Vectorizer fit to the vocabulary of the model and seed\n",
    "        max_length (int):\n",
    "            Maximum number of characters in the poem\n",
    "    \n",
    "    Returns:\n",
    "        str: A poem\n",
    "    \"\"\"\n",
    "    seed_chars = list(seed)\n",
    "    for i in range(0, max_length):\n",
    "        # Vectorize the seed phrase and give it the shape:\n",
    "        # (batch_size, seq_length, vocab_size)\n",
    "        vec = (vectorizer\n",
    "               .tokens_to_vectors(seed_chars)\n",
    "               .reshape(1, len(seed_chars), vectorizer.vocab_size()))\n",
    "\n",
    "        # Ask the model for a prediction on the current seed phrase\n",
    "        pred_distribution = model(vec)[0].numpy()\n",
    "\n",
    "        # Make a choice over the probability distribution, like in the Shannon method\n",
    "        options = list(range(0, vectorizer.vocab_size()))\n",
    "        idx = np.random.choice(options, p=pred_distribution)\n",
    "\n",
    "        # Turn the chosen value into a character and append it to the seed\n",
    "        char = vectorizer.int_to_token(idx)\n",
    "        seed_chars.append(char)\n",
    "\n",
    "        # If the character is one of the poem segmenting symbols, end early\n",
    "        if char in [\"@\", \"$\"]:\n",
    "            break\n",
    "    return \"\".join(seed_chars)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## 4. Comparing Generated Poems from N-gram LM and Recurrent Neural LM\n",
    "\n",
    "Generate 3 poems from each type of model and conduct a qualitative analysis on their differences"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### N-Gram Language Model Poems"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create poems from BIGRAM model\n",
    "bigram_lm_poems = [\n",
    "    postprocess_for_ngram_lm(bigram_model.generate_poem(max_words=100))\n",
    "    for _ in range(3)\n",
    "]\n",
    "\n",
    "# Create poems from TRIGRAM model\n",
    "trigram_lm_poems = [\n",
    "    postprocess_for_ngram_lm(trigram_model.generate_poem(max_words=100))\n",
    "    for _ in range(3)\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----- BIGRAM POEMS -----\n",
      "Poem 1\n",
      "a generation playing its myriad waves of proof of the north , \n",
      " by axes ? \n",
      " are , \n",
      " to himself , dress with my arms i hear walt whitman ? is little breaths , \n",
      " i have the neighboring shore to have loved ! \n",
      " torches shine the paths and eidolons.\n",
      "\n",
      " and elude the younger by under her she looks back most to thee , \n",
      " soon shall be happy with the states , but one of that makes minute , \n",
      " but the drummers , to merge in bays , i\n",
      "\n",
      "Poem 2\n",
      "wandering at table , \n",
      " the buildings and evil propels me seek ’ d all the crew of these chants each and strong divine woman that\n",
      " from it is , themselves so that long roll alarming , and\n",
      " growth of the way through the male and on my son to call even we wander ’ d scum , \n",
      " my chair by luck veering , more .\n",
      "\n",
      "Poem 3\n",
      "1\n",
      " what wants , large , the ’ s bay , -- what do less immortal even when the fields of which i heard over the air , full of his brothers mine , , , you hastily walk ’ en at daybreak , thy ultimate human race.\n",
      "\n",
      " 9\n",
      " i have we may filter and light over , \n",
      " ever the from the islands , hunger to the the cool night , \n",
      " how ye is there while thou , bathing myself -- and your fires , \n",
      " i return ’\n",
      "\n",
      "----- TRIGRAM POEMS -----\n",
      "Poem 1\n",
      "the world chaste , temperate , an eidolon.\n",
      "\n",
      " thy very songs not in that thou sat ’ st where washington sat , \n",
      " , criticisms , \n",
      " the measur ’ d , \n",
      " the railroads , with laws , courts , jury and judge , the suez canal , \n",
      " to be work ’ d neck and passionately kiss ’ d person , how solemn they look there is something that shall endure longer even than the mother of all by them , \n",
      " girls , my consolation what they have weight , form upright , death\n",
      "\n",
      "Poem 2\n",
      "city of the sands and drift it in their tents the general western world , \n",
      " a brood of the north atlantic , \n",
      " my own way of the real words are in building the of thy teeming womb thy giant babes in wombs , latent within thyself , \n",
      " through him flights , whirls , screams , , \n",
      " see , great war for you a programme of chants.\n",
      "\n",
      " chants of the rest and tacking the pours copiously out of itself , which i think o tongues ye tell this heart , my\n",
      "\n",
      "Poem 3\n",
      "unseen buds , infinite , teeming , intricate whirl , the sleepers sleep ? have they them , in flutes ’ clear notes and sounding harps ’ for us , \n",
      " passing sweet hours , months , or finish ’ d to powder and pour ’ d after the his thread , \n",
      " i see in you , \n",
      " with the mothers of mothers , \n",
      " to fly in the clouds of dust the to the growth of thousands of globes ! thou hot , and over with its devilish winnings and\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\"----- BIGRAM POEMS -----\")\n",
    "for i, bigram_lm_poem in enumerate(bigram_lm_poems):\n",
    "    print(f\"Poem {i+1}\")\n",
    "    print(bigram_lm_poem)\n",
    "    print()\n",
    "\n",
    "print(\"----- TRIGRAM POEMS -----\")\n",
    "for i, trigram_lm_poem in enumerate(trigram_lm_poems):\n",
    "    print(f\"Poem {i+1}\")\n",
    "    print(trigram_lm_poem)\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### LSTM Language Model Poems"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create poems from LSTM NEURAL model\n",
    "seed_phrases = [\n",
    "    \"i sing a song of myself \", \n",
    "    \"cool afternoon breeze \", \n",
    "]\n",
    "neural_lm_poems = [\n",
    "    postprocess_for_neural_lm(generate_poem(lstm_model, s, vectorizer, max_length=200))\n",
    "    for s in seed_phrases\n",
    "] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----- LSTM POEMS -----\n",
      "Poem 1\n",
      "i sing a song of myself in itself,\n",
      " to walk-sion vines of holls scorlings, fast, all sweet, for dream,\n",
      " the young past and bold, yet it tears. o the songs, or no earth,\n",
      " in sponks in these root and fisherous emigran timie\n",
      "\n",
      "Poem 2\n",
      "cool afternoon breeze for you,\n",
      " see the looking--the value of music, exhibited refire’d,\n",
      " which flowering touch--red pork, centre of invitation,\n",
      " are the cries when we will shoot rear’d audy as if shapes, the fearless,\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\"----- LSTM POEMS -----\")\n",
    "for i, lstm_lm_poem in enumerate(neural_lm_poems):\n",
    "    print(f\"Poem {i+1}\")\n",
    "    print(lstm_lm_poem)\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 Discussion\n",
    "\n",
    "Unsurprisingly, the bigram model is noticeably worse than the trigram and LSTM models. None of the three models achieve global coherency (coherency through an entire poem), though the trigram and LSTM models have relatively good grammar and create locally coherent/believable snippets.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6 (default, Oct 18 2022, 12:41:40) \n[Clang 14.0.0 (clang-1400.0.29.202)]"
  },
  "vscode": {
   "interpreter": {
    "hash": "8b3bcbfdbe2ea4559ba2b087259f245128234fbe81df24eef219cf04c2909952"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}