active_learning_single_annotator/active_learning_single_annotator.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A standard active learning pipeline to efficiently label data for classification\n",
    "Train accurate classifier models with minimal data labeling (and minimal code) via active learning and AutoML."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/cleanlab/examples/blob/master/active_learning_single_annotator/active_learning_single_annotator.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook demonstrates a practical approach to efficiently label data for training an accurate image classifier via active learning and AutoML. We consider standard active learning settings with a pool of unlabeled examples, where we label a batch of examples at a time and collect **at most one label** per example. If your data labeling may be noisy (imperfect), then consider our **active_learning_multiannotator** notebook instead, which helps you decide what data to re-label during active learning. The active learning method demonstrated here works for any classification data (text, tabular, audio, etc.) as long as you are able to train a ML model on the labeled subset of the data.\n",
    "\n",
    "In **Active Learning**, we aim to construct a labeled dataset by collecting the fewest labels that still allow us to train an accurate classifier model. Here we assume data labeling is done in **batches**, and between these data labeling rounds, we retrain our classifier to decide what previously unlabeled examples (i.e. datapoints) to label next round.\n",
    "\n",
    "\n",
    "This notebook demonstrates how to compute these scores easily for use in sequential active learning, showing how a classification model iteratively improves after labeling more examples for multiple rounds with the following steps:\n",
    "\n",
    "1. Establish an initially labeled dataset, `df_labeled` to train the model on. This is a small subset of our training data, `df_train`. The rest of the training data is marked as `df_unlabeled`.\n",
    "2. Train the model on the available labeled data and get predictions for the unlabeled data, `pred_probs_unlabeled`.\n",
    "3. Compute active learning scores for all unlabeled examples and select which samples to collect labels for.\n",
    "4. Label the selected samples and add them to current training set.\n",
    "5. Repeat steps 2-4 to collect as many labels as your budget permits.\n",
    "\n",
    "The accuracy of the model trained on the resulting dataset will generally match that of the same model trained on a much larger set of randomly selected examples -- i.e. this is the cheapest way to grow a dataset for training an accurate classifier!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import dependencies and data\n",
    "\n",
    "Here we use images from the [Caltech-256](https://data.caltech.edu/records/nyy15-4j048) [1] classification dataset. Any image dataset in the same format can be substituted instead and the same code should work.  Please install the dependencies specified in this [requirements.txt](https://github.com/cleanlab/examples/blob/master/active_learning_single_annotator/requirements.txt) file before running this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/.local/lib/python3.8/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from autogluon.multimodal import MultiModalPredictor\n",
    "from gluoncv.auto.data.dataset import ImageClassificationDataset\n",
    "from sklearn.model_selection import train_test_split\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from cleanlab.multiannotator import get_label_quality_scores, get_active_learning_scores\n",
    "from utils.model_training_autogluon import train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File ‘256_ObjectCategories.zip’ already there; not retrieving.\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!wget -nc 'https://cleanlab-public.s3.amazonaws.com/ActiveLearning/Caltech256/256_ObjectCategories.zip' && unzip -o -q 256_ObjectCategories.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Construct a small initial labeled dataset\n",
    "We load our data file into a variable called `dataset`. This is a DataFrame containing labels and file paths for each image (i.e. example) from Caltech-256\n",
    "\n",
    "We then randomly split the dataset into train and test splits. Test data are just used here to demonstrate the accuracy in our model after each active learning round (you may not have any test data in your applications)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = ImageClassificationDataset.from_folder('./256_ObjectCategories/')\n",
    "dataset = dataset.replace(257, 256) # no class class in dataset is labeled as 257, we need to reindex\n",
    "\n",
    "# Split data into train and test\n",
    "df_train, df_test = train_test_split(dataset, test_size=0.33, random_state=123)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The train data are further split into labeled and unlabeled pools. `df_labeled` represents a small initial labeled dataset which we use to train an initial classifier model (in your application, this would be all the labeled data you have). `df_unlabeled` represents our unlabeled pool of examples we could consider labeling. In this example, we technically know all the labels for these images too -- given they all come from Caltech-256 -- but we demonstrate how active learning would work in your applications by assuming we don't know their labels. In each active learning round, we only reveal the label of specific images the algorithm decides to collect labels for.\n",
    "\n",
    "In this demonstration, our initial training set (`df_labeled`) has 8 labeled images from each class, which is not enough data to train a good classifier. The goal is to grow this dataset with the fewest number of additional labeled examples that suffice to train an accurate model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_labeled(dataset,  num_labeled_per_class=8):\n",
    "    \"\"\"Splits provided dataset into two datasets. With df_labeled containing num_labeled_per_class labeles for \n",
    "    each class and df_unlabeled containing the rest of the rows in dataset\"\"\"\n",
    "    \n",
    "    df_labeled = dataset.groupby(\"label\").sample(n=num_labeled_per_class, random_state=123)\n",
    "    labeled_index = list(df_labeled.index)\n",
    "    unlabeled_index = [i for i in range(len(dataset)) if i not in labeled_index]\n",
    "    df_unlabeled = dataset.iloc[unlabeled_index]\n",
    "    df_unlabeled = df_unlabeled.reset_index(drop=True)\n",
    "    df_labeled = df_labeled.reset_index(drop=True)    \n",
    "    return df_labeled, df_unlabeled\n",
    "\n",
    "# Split the train data into labeled and unlabeled with 8 labeled per each class\n",
    "df_labeled, df_unlabeled = get_labeled(df_train, num_labeled_per_class=8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train model on labeled data and get predicted class probabilites for unlabeled data\n",
    "\n",
    "The first step of the active learning pipeline is to train your model on the available labeled data. Next we ask the trained model for its predictions on the unlabeled data -- specifically the predicted probability of each class for each unlabeled example. The `train()` function below returns our `predictor` fitted to `df_labeled`. To use a different type of model, modify this `train()` function as needed. All you need to run active learning with cleanlab is code to: (1) train your model on the labeled data, (2) get its predicted class probabilities for the unlabeled data, (3) collect labels for the examples with the lowest active learning scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = train(df_labeled, out_folder=None, time_limit=30)\n",
    "pred_probs_unlabeled = predictor.predict_proba(df_unlabeled)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Obtain active learning scores for the unlabeled data\n",
    "\n",
    "Using these predicted class probabilities, you should next compute active learning scores that estimate the informativeness of labeling each datapoint. Since we will collect at most one annotation per example in this pipeline, we only care about scoring the unlabeled data.\n",
    "\n",
    "These active learning scores represent how confident our model is about an example's true label based on the currently obtained annotations; examples with the lowest scores are those for which additional labels should be collected (i.e. likely the most informative). These scores are estimated via [ActiveLab](https://arxiv.org/abs/2301.11856) [2], an algorithm developed by the Cleanlab team. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# compute active learning scores\n",
    "_, active_learning_scores_unlabeled = get_active_learning_scores(\n",
    "    df_labeled['label'].to_numpy(), pred_probs_unlabeled=pred_probs_unlabeled\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.00742915, 0.00528911, 0.00507766, 0.00588142, 0.00631748])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# print active learning scores for the first 5 examples in the unlabeld pool:\n",
    "active_learning_scores_unlabeled[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get index of examples to collect labels for\n",
    "\n",
    "Subsequently, rank the unlabeled examples by their active learning scores, and obtain the indices of examples with the lowest scores. These are the **unlabeled** examples whose true label our current model is least confident about. You should prioritize these examples for labeling next."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_idx_to_label(active_learning_scores_unlabeled, batch_size_to_label):\n",
    "    \"\"\"Function to get indices of examples with the lowest active learning score to collect more labels for.\"\"\"\n",
    "    \n",
    "    return np.argsort(active_learning_scores_unlabeled)[:batch_size_to_label]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([ 7928,  9244,  3141, 11455,  1645]),\n",
       " array([0.00469176, 0.00472476, 0.00474545, 0.00476181, 0.00477689]))"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size_to_label = 100  # you can pick how many examples to collect more labels for at each round, depending on your setup\n",
    "\n",
    "# get next idx to label based on batch_size_to_label and magnitude of each example's active learning score\n",
    "next_idx_to_label = get_idx_to_label(active_learning_scores_unlabeled, batch_size_to_label=batch_size_to_label)\n",
    "next_idx_to_label[:5],active_learning_scores_unlabeled[next_idx_to_label[:5]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run 20 rounds of active learning (collecting new labels) to iteratively improve model \n",
    "\n",
    "The code below shows a full demonstration of how we can **repeatedly** use the above methods to: select which examples to collect labels for next, add their labels to the current training dataset, and train an improved classifier model.\n",
    "\n",
    "Here we run 20 rounds of this active learning loop, choosing 100 new unlabeled examples to label in each round. In your applications, you will need to replace the code we used here to reveal the labels of new examples.\n",
    "\n",
    "[Optional step] After each round, we also report the current model's accuracy on our held-out test dataset (you may not have test data in your applications)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def setup_next_iter_data(df_labeled, df_unlabeled, relabel_idx_unlabeled):\n",
    "    \"\"\"Updates inputs after additional labels have been collected in a single active learning round,\n",
    "    this ensures that the inputs will be well formatted for the next round of active learning.\"\"\"\n",
    "\n",
    "    df_labeled = pd.concat([df_labeled,df_unlabeled.iloc[relabel_idx_unlabeled]], ignore_index=True)\n",
    "    df_unlabeled = df_unlabeled.drop(relabel_idx_unlabeled)\n",
    "    df_unlabeled = df_unlabeled.reset_index(drop=True)\n",
    "    df_labeled = df_labeled.reset_index(drop=True)  \n",
    "    return df_labeled, df_unlabeled"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_rounds = 20\n",
    "batch_size_to_label = 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_accuracy_arr = np.full(num_rounds, np.nan)\n",
    "\n",
    "for i in range(num_rounds):\n",
    "    # train model and obtain predicted class probabilities for the unlabeled data\n",
    "    print('fitting model')\n",
    "    predictor = train(df_labeled, out_folder=None, time_limit=60*60)\n",
    "    \n",
    "    print('obtaining predicted class probabilities for the unlabeled data')\n",
    "    pred_probs_unlabeled = predictor.predict_proba(df_unlabeled)\n",
    "        \n",
    "    print('computing active learning scores')\n",
    "    # compute active learning scores\n",
    "    _, active_learning_scores_unlabeled = get_active_learning_scores(\n",
    "        df_labeled['label'].to_numpy(), pred_probs_unlabeled=pred_probs_unlabeled\n",
    "    )\n",
    "    \n",
    "    print('getting idx to relabel')\n",
    "    # get the indices of examples to collect more labels for\n",
    "    relabel_idx_unlabeled = get_idx_to_label(\n",
    "        active_learning_scores_unlabeled=active_learning_scores_unlabeled,\n",
    "        batch_size_to_label=batch_size_to_label,\n",
    "    )\n",
    "    \n",
    "    print('setting up next iter')\n",
    "    # format the data for the next round of active learning, ie. moving some unlabeled \n",
    "    # examples to the labeled pool because we are collecting labels for them\n",
    "    df_labeled, df_unlabeled = setup_next_iter_data(df_labeled, df_unlabeled, relabel_idx_unlabeled)\n",
    "    \n",
    "    # evaluate model accuracy for the current round on held-out test data. This is an optional step \n",
    "    # for demonstration purposes, in practical applications you may not have ground truth labels\n",
    "    print('predicting class labels for test split')\n",
    "    pred_labels = predictor.predict(data=df_test)\n",
    "    true_labels_test = np.array(df_test['label'].tolist())\n",
    "    model_accuracy_arr[i] = np.mean(pred_labels == true_labels_test)\n",
    "    print('test round: ', i, 'accuracy: ', np.mean(pred_labels == true_labels_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Results\n",
    "\n",
    "Below, we can see that the model accuracy increases steadily with each additional round of data labeling and model training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial model test accuracy: 0.845\n",
      "Final model test accuracy (after 20 rounds of active learning): 0.938\n"
     ]
    }
   ],
   "source": [
    "print(f\"Initial model test accuracy: {model_accuracy_arr[0]:.3}\")\n",
    "print(f\"Final model test accuracy (after 20 rounds of active learning): {model_accuracy_arr[-1]:.3}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAGwCAYAAABB4NqyAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABX6klEQVR4nO3deVxU5f4H8M/MwDDDLrIji6CCIIKKkEupSW6FWt4yNUVtuZqWS1dTc+lWiraYZqZpbmmm3Z9LaaUZJWUiKohpbiAKiKyiLINsM+f3BzJJojIwcBjm83695nXlcOY73+k68PE5z3keiSAIAoiIiIiMiFTsBoiIiIiaGgMQERERGR0GICIiIjI6DEBERERkdBiAiIiIyOgwABEREZHRYQAiIiIio2MidgPNkUajwfXr12FlZQWJRCJ2O0RERFQHgiCgqKgIrq6ukEofPMbDAFSL69evw93dXew2iIiIqB7S09PRpk2bB57DAFQLKysrAFX/Aa2trUXuhoiIiOqisLAQ7u7u2t/jD8IAVIvqy17W1tYMQERERAamLtNXOAmaiIiIjE6zCECrV6+Gl5cXFAoFwsLCcPz48fueW1FRgXfeeQc+Pj5QKBQICgrCgQMH7nv+0qVLIZFIMH369EbonIiIiAyR6AFo586dmDlzJhYtWoSEhAQEBQVh4MCByMnJqfX8+fPn4/PPP8eqVatw7tw5TJo0CU8//TROnTp1z7knTpzA559/js6dOzf22yAiIiIDInoAWr58OV5++WVMmDAB/v7+WLt2LczNzbFx48Zaz9+6dSvmzZuHIUOGwNvbG5MnT8aQIUPw0Ucf1TivuLgYY8aMwfr169GqVaumeCtERERkIEQNQOXl5YiPj0d4eLj2mFQqRXh4OGJjY2t9TllZGRQKRY1jSqUSR44cqXFsypQpePLJJ2vUvp+ysjIUFhbWeBAREVHLJWoAysvLg1qthpOTU43jTk5OyMrKqvU5AwcOxPLly5GUlASNRoNDhw5h9+7dyMzM1J6zY8cOJCQkICoqqk59REVFwcbGRvvgGkBEREQtm+iXwHS1cuVKtG/fHn5+fpDL5Zg6dSomTJigXfExPT0d06ZNw1dffXXPSNH9zJ07FwUFBdpHenp6Y74FIiIiEpmoAcje3h4ymQzZ2dk1jmdnZ8PZ2bnW5zg4OGDv3r1QqVRITU3FhQsXYGlpCW9vbwBAfHw8cnJy0LVrV5iYmMDExAQxMTH45JNPYGJiArVafU9NMzMz7Zo/XPuHiIio5RM1AMnlcnTr1g3R0dHaYxqNBtHR0ejRo8cDn6tQKODm5obKykrs2rULw4YNAwD0798fZ86cQWJiovYREhKCMWPGIDExETKZrFHfExERETV/oq8EPXPmTERGRiIkJAShoaFYsWIFVCoVJkyYAAAYN24c3NzctPN54uLikJGRgeDgYGRkZODtt9+GRqPB7NmzAVRtY9GpU6car2FhYYHWrVvfc5yIiIiMk+gBaOTIkcjNzcXChQuRlZWF4OBgHDhwQDsxOi0trcaOrqWlpZg/fz5SUlJgaWmJIUOGYOvWrbC1tRXpHRAREZGhkQiCIIjdRHNTWFgIGxsbFBQUcD4QERGRgdDl97foI0BERERkPNQaAVmFpZAAcLVVitYHAxARERHpVWmFGmn5JUi9UYLUGyqk55cgNb8EaTdKcO3mbZSrNRgV6o6oZ8TbqooBiIiIiHQiCALyVeXaUJN6owRp+SVIy1ch9UYJcorKHvh8U5kEZRWaJuq2dgxAREREdI9KtQbXb5Ui9U6oSa8e0cmv+nNxWeUDn2+lMIFna3N42lnAo7U5POzM4WlnDo/W5nCxUUImlTTRO6kdAxARERFpHb+Sj8U/nMfZjAKoNfe/T0oiAZytFVXB5k7A8WhtAc87X9soTSGRiBtyHoQBiIiIiJCvKkfUD+fxv/hr2mNyE6l25Mb9TrCpCjsWaNNKCYWp4S4uzABERERkxARBwP/iryHqh/O4WVIBABgV6oEp/XzgaqOEVORLVY2FAYiIiMhIXcouwvw9Z3H8aj4AwM/ZCoufDkQ3z1Yid9b4GICIiIiMzO1yNVb9koR1v6WgUiNAaSrDzCc6YHwvL5jKRN0mtMkwABERERmRXy/kYMG3Z3Ht5m0AwBP+Tnh7aADcRFyUUAwMQEREREYgs+A23tl3Dj+ezQIAuNoo8PbQAAwIcBa5M3EwABEREbVglWoNvoxNxUc/XYSqXA2ZVIIXe7fFtP7tYWFmvDHAeN85ERFRC5eYfgtv7TmDv64XAgC6ethi8dOB6OjCjb4ZgIiIiFqYwtIKfHjwIrYeS4UgANYKE8wZ3BHPd3dvsbe164oBiIiIqIUQBAH7/szEu/vPIffOflzPdHHDvCc7wt7STOTumhcGICIiohbgap4KC749i9+T8gAA3vYWeG94J/RsZy9yZ80TAxAREZEBK6tUY+3hFKw+nIzySg3kJlJM7dcO/+7jDTMTw92qorExABERERmoo8l5mL/3LFLyVACAR9vb491hneBlbyFyZ80fAxAREZGBySsuw+Lvz2PPqQwAgIOVGRY85Y+Izi7Negf25oQBiIiIyEAIgoDdCRl4Z/85FNyugEQCjH3EE28M8IWN0lTs9gwKAxAREZEBuHazBPP2nMVvl3IBAP4u1ljyTCCC3W3FbcxAMQARERE1YxqNgG1xqVj24wWoytWQm0gxPbw9Xn7U22g2Lm0MDEBERETNVHJOMebs+hMnU28CALp7tcLSEZ3h42ApcmeGjwGIiIiomalQa7DutxSs/DkJ5WoNLOQyzBnshzFhnlzJWU8YgIiIiJqRsxkFmPV/f+J8ZtX+XX06OGDJM4Fws1WK3FnLwgBERETUDJRWqLHi5ySs/z0Fao0AW3NTLIrwx/BgN97a3ggYgIiIiEQWl3IDc3afwZU7Cxo+1dkFbw8N4P5djYgBiIiISCRFpRVYduACth1LAwA4WZvh3WGdMCDAWeTOWj4GICIiIhH8eiEH8/acQWZBKQBgVKg75gzuyAUNmwgDEBERURPKV5XjnX1/YW/idQCAh505lj4TyF3bmxgDEBERURMQBAH7/szE29/9hXxVOaQS4MXebTHzCV8o5dy1vakxABERUYsnCALyistxJU+FlNxiXMlTIS2/BFYKE7jaKuFmq4RbKyXa2JrD2UYBuYl+V1jOLLiNBXvP4ufzOQAAP2crLBvRGUHcxkI0DEBERNRi3C5XV4WcvGJcyVUhJe/OI7cYRaWVdaohkQBOVgq4taoKRq7acKTUHrMwq9uvT41GwI4T6Yj64TyKyiphKpPgtcfbY1IfH72HLNINAxARERkUtUbA9Vu3tcEmJVelHdm5fmdCcW0kEqBNKyXa2lvC294Cnq3NUVxaiYxbt6seN6v+t6xSg6zCUmQVliL+zhYU/2Rrbvp3OLJVok2rv0eR3GyVsLOQI/VGCebs/hPHUvIBAMHutnj/X53RwcmqUf67kG4YgIiIqFkqLqvExawi7SWrlNyqkZ2rN0pQXqm57/NamZuirb0FvB0s0dbeAj4OVX/2sDOHwvTBc20EQcANVbk2DFX/7zXt1yUoLK3ErZIK3CqpwF/XC2utozSVQa0RUK7WQGkqw38G+mJ8Ty/IuI1Fs8EAREREzcq564X4MvYq9iZmoLSi9qAjl0nhZW8Ob3tLtHWwgLe9BbwdLOBtb4lWFvJ6v7ZEIoG9pRnsLc3uOz+nqLQCGbdu4/qdgHTtrqCUcfM2corKcLtCDQDo3c4eUc8Ewt3OvN49UeNgACIiItFVqDU4cDYLX8ZexYmrf192crI2QztHy6qgcyfk+DhYwtVWKdpoipXCFH7OpvBztq71+2WVamTeKkVZpQYdnCy5jUUzxQBERESiySkqxddx6fgqLhU5RWUAABOpBIMDXRDZwxPdPFsZXIAwM5HBy95C7DboIRiAiIioSQmCgIS0m9hyNBU/ns1EhVoAADhYmWF0qAfGhHnA0VohcpfU0jEAERFRkyitUOO7xOvYEnu1xuThEM9WGNfTC4MCnHlrODUZBiAiImpU6fkl2HYsFTtPpuNWSQUAwMxEiuHBbhjbwxOd3GxE7pCMEQMQERHpnUYj4EhyHr6MvYroCzkQqq5yoU0rJcb18MRzIe6wNa//3VpEDcUAREREelNUWoH/i7+GrbGpSMlTaY8/2t4e43t6oa+vI9fCoWaBAYiIiBosKbsIX8amYnfCNajKq9bAsTIzwYhubTC2hyd8HCxF7pCoJgYgIiKqF1VZJX5PqrrMdfTyDe3x9o6WGNfTC093cYNlHffMImpq/JtJREQAqm5PLy6rRF5xOfKKy5BXVIa84jLkFpUht/pY9aOoXLvaMQBIJcAAf2eM6+mJHt6tDW7tHjI+DEBERC2YIAgoLK28K9DUDDK5ReXIvSvslD1gj63a2Fua4bmQNhjziCfcbJWN9C6I9I8BiIioBapQa7Dql2Rs+D1FOyenrizkMthbmd3ZE0sOB+2fqx4OVnLtny14iYsMFP/mEhG1MMk5xZixMxFnMgq0xyzNTGoNM/Z3hRnHO99Tyh+8YzpRS8AARETUQmg0Ar6MvYqoHy+grFIDG6Up3h3eCQP8naAwZaghuhsDEBFRC5BVUIpZ/3cavyflAahad+eDfwXB2YZ7ahHVhgGIiMjAfXf6OubvOYPC0kooTKWYN6Qjxj7iyTuxiB6AAYiIyEAVlFRg/rdnse/0dQBAUBsbLB8ZzEUHieqAAYiIyAD9npSLWf/7E1mFpZBJJXjt8XaY0q8dTGXcTZ2oLhiAiIgMyO1yNZYduIDNR68CALztLbB8ZDCC3W1F7YvI0DAAEREZiD+v3cL0nYlIya3aZHRcD0/MHdyRt60T1QMDEBFRM1ep1mD1r5ex6pckVGoEOFqZ4YNng9Cng4PYrREZLAYgIqJmLCW3GDO/OY3E9FsAgCcDXfDe8E5oZSEXtzEiA8cARETUDAmCgG1xaVjy/XncrlDDSmGCd4d1wrBgV97eTqQHDEBERM1MTmEpZu/6E4cv5gIAevq0xofPBsGVm40S6Q0DEBFRM/LDmUzM23MGt0oqYGYixZuD/DC+pxekUo76EOkTAxARUTNQWFqBt7/9C7tPZQAAOrlZ4+PngtHeyUrkzohapmaxYtbq1avh5eUFhUKBsLAwHD9+/L7nVlRU4J133oGPjw8UCgWCgoJw4MCBGudERUWhe/fusLKygqOjI4YPH46LFy829tsgIqqXo5fzMOjj37D7VAakEmBqv3bYPbkXww9RIxI9AO3cuRMzZ87EokWLkJCQgKCgIAwcOBA5OTm1nj9//nx8/vnnWLVqFc6dO4dJkybh6aefxqlTp7TnxMTEYMqUKTh27BgOHTqEiooKDBgwACqVqqneFhHRQ5VVqvHe/nMYvT4O1wtK4dnaHP+b1BP/GegLuYnoP56JWjSJIAiCmA2EhYWhe/fu+PTTTwEAGo0G7u7ueO211zBnzpx7znd1dcVbb72FKVOmaI+NGDECSqUS27Ztq/U1cnNz4ejoiJiYGDz22GP3fL+srAxlZWXarwsLC+Hu7o6CggJYW1s39C0SEd0j7UYJpmxPwJmMAgDA6DAPvDWkIyzMODOBqL4KCwthY2NTp9/fov4To7y8HPHx8QgPD9cek0qlCA8PR2xsbK3PKSsrg0KhqHFMqVTiyJEj932dgoKqHzB2dna1fj8qKgo2Njbah7u7u65vhYiozg6czcKTq37HmYwC2JqbYkNkCJY8HcjwQ9SERA1AeXl5UKvVcHJyqnHcyckJWVlZtT5n4MCBWL58OZKSkqDRaHDo0CHs3r0bmZmZtZ6v0Wgwffp09OrVC506dar1nLlz56KgoED7SE9Pb9gbIyKqRXmlBv/d9xcmbYtHUWklunrY4ofXH0X/jk4PfzIR6ZXB/XNj5cqVePnll+Hn5weJRAIfHx9MmDABGzdurPX8KVOm4OzZsw8cITIzM4OZmVljtUxEhPT8Ekz9+hRO31nR+ZXHvDFroC93bycSiaifPHt7e8hkMmRnZ9c4np2dDWdn51qf4+DggL1790KlUiE1NRUXLlyApaUlvL297zl36tSp2L9/P3799Ve0adOmUd4DEdHDHDqXjSc/+R2n02/BRmmKL8aFYN6Qjgw/RCIS9dMnl8vRrVs3REdHa49pNBpER0ejR48eD3yuQqGAm5sbKisrsWvXLgwbNkz7PUEQMHXqVOzZswe//PIL2rZt22jvgYjofirUGiz+/hxe/vIkCksrEeRui+9f741wf17yIhKb6JfAZs6cicjISISEhCA0NBQrVqyASqXChAkTAADjxo2Dm5sboqKiAABxcXHIyMhAcHAwMjIy8Pbbb0Oj0WD27NnamlOmTMH27dvx7bffwsrKSjufyMbGBkoll5InosZ3/dZtTN2egIS0WwCAib3aYs5gP97eTtRMiB6ARo4cidzcXCxcuBBZWVkIDg7GgQMHtBOj09LSIJX+/QOjtLQU8+fPR0pKCiwtLTFkyBBs3boVtra22nPWrFkDAOjbt2+N19q0aRPGjx/f2G+JiIzcrxdyMOObRNwqqYCVwgQf/CsIgzrVflmfiMQh+jpAzZEu6wgQEVWrVGvw4U+XsDbmMgAg0M0Gq0d3hUdrc5E7IzIOuvz+Fn0EiIioJcgqKMVrXyfgxNWbAIDIHp6Y92RHmJnIRO6MiGrDAERE1EAxl3IxY2ci8lXlsDQzwbIRnfFkZxex2yKiB2AAIiKqp0q1Bit+TsLqw8kQBMDfxRqfjekKL3sLsVsjoodgACIiqoecwlK89vUpxF3JBwC88IgH5j/pD4UpL3kRGQIGICIiHf2RnIdpO04hr7gcFnIZokZ0xtAgV7HbIiIdMAAREdWRWiPgk+gkfPJLEgQB8HO2wuoxXeHjYCl2a0SkIwYgIqI6yC0qw/Sdp/BH8g0AwPPd3fH20ABe8iIyUAxAREQPEXv5Bl7fcQq5RWVQmsqw5JlOeLoL9xckMmQMQERE91FwuwLrf0vBZ4eToRGADk6W+GxMV7RztBK7NSJqIAYgIqJ/yC4sxYYjV/DVsVSoytUAgGe7tcE7wzpBKeclL6KWgAGIiOiOy7nFWBeTgj2nMlCu1gAAfJ2s8Hr/9lzYkKiFYQAiIqN3Ov0W1hy+jIPnslC9O2J3r1aY3NcH/XwdIZFIxG2QiPSOAYiIjJIgCDiSnIc1hy/j6OUb2uPhHR0xqY8PQrzsROyOiBobAxARGRW1RsCPZzOx5vBl/HW9EABgIpVgaLAr/v2YD3ydOcGZyBgwABGRUSitUGNXwjWs+y0FqTdKAABKUxlGdnfHS4+2RZtW5iJ3SERNiQGIiFq0wtIKbDuWio1HriKvuAwAYGtuisgeXojs6QU7C7nIHRKRGBiAiKhFyiksxYY/rmD7sTQUlVUCAFxtFHjpUW+M7O4OCzP++CMyZvwJQEQtytU8FT7/LQW74q9pb2Vv72iJf/fxwdAgV8hNpCJ3SETNAQMQEbUIZ64VYG3MZfx4NhOaO7eyd/WwxeS+7dDfzxFSKW9lJ6K/MQARkUE7fiUfq35Jwu9Jedpj/XwdMLlvO3T3asU1fIioVgxARGSQkrKLsOzABfx8PgcAIJNKENHZBf/u44OOLtYid0dEzR0DEBEZlKyCUnx86BL+F58OjVAVfJ4LccerfX3gbsdb2YmobhiAiMggFJZWYO3hy9j4xxWUVlRNbh4Y4IRZA/3QztFS5O6IyNAwABFRs1ZWqca2Y2n49Jck3CypAACEeLbC3CF+6ObJ7SqIqH4YgIioWdJoBOz78zo+OHgR127eBgD4OFjgzUF+eMLfiZObiahBGICIqNk5kpSHqB/Pa/fqcrQyw4wnOuDZbm1gIuM6PkTUcAxARNRs/HW9AEt/vKC9pd3SzAST+nhjYu+2MJfzxxUR6Q9/ohCR6NLzS7D80CXsOZUBADCVSTAmzBOvPd4OrS3NRO6OiFoiBiAiEs1NVTlW/5qML2NTtdtWRAS5YtYAX3i05i3tRNR4GICIqMmVVqix6Y+r+OxwMopKqzYq7enTGnMG+6FzG1txmyMio8AARERNRq0RsCvhGj4+dAmZBaUAAD9nK8wZ7Ic+HRx4ZxcRNRkGICJqdIIg4NeLOVj64wVcyi4GALjZKvHGgA4YHuzGjUqJqMkxABFRozpxNR8fHryIuCv5AAAbpSmm9muHsT08oTCVidwdERkrBiAiahTHr+Rjxc+XcPTyDQCA3ESKCb288GqfdrAxNxW5OyIydgxARKRXx1JuYOXPSYhNqQo+pjIJ/tXNHa893g6utkqRuyMiqsIARER6EXv5BlZGX8KxlKpLXaayql3aJ/f1QZtWvKWdiJoXBiAiqjdBEBB7Z8Sneo6PqUyCkd3dMblvO7hxxIeImikGICLSmSAIiL18Ayuik3D8TvCRy6R3go8PL3URUbPHAEREdSYIAo5evoEVP1/Cias3AVQFn+dDq4KPiw2DDxEZBgYgInooQRDwR3JV8DmZeif4mEgxqrs7JjH4EJEBYgAiovsSBAFHkvOw4uckxN8VfEaHemBSHx842yhE7pCIqH4YgIjoHoIg4LekPKz8+RIS0m4BAMxMpBgdVhV8nKwZfIjIsDEAEZGWIAiIuZSLldFJOHVX8BkT5olJfbzhyOBDRC0EAxARQRAEHL6Ui5U/JyEx/RYAQGFaFXz+3ccbjlYMPkTUsugcgFQqFSwsLBqjFyISwdU8FWZ8k6gd8VGYSvFCmCdeYfAhohZM5wDk5OSE5557DhMnTkTv3r0boyciaiI/n8vGjG8SUVRaCYWpFON6eOHlR73hYGUmdmtERI1KqusTtm3bhvz8fDz++OPo0KEDli5diuvXrzdGb0TUSNQaAR/9dBEvfXkSRaWV6Ophi1//0xfzhnRk+CEio6BzABo+fDj27t2LjIwMTJo0Cdu3b4enpyeeeuop7N69G5WVlY3RJxHpya2SckzcfAKrfkkGAIzr4Ykdr/TgWj5EZFQkgiAIDS2yatUqzJo1C+Xl5bC3t8ekSZMwZ84cmJsb5gaIhYWFsLGxQUFBAaytrcVuh0hvzmYUYNK2eFy7eRsKUymWPB2IZ7q2EbstIiK90OX3d73vAsvOzsaWLVuwefNmpKam4l//+hdefPFFXLt2DcuWLcOxY8fw008/1bc8EenZ/8Vfw1t7zqCsUgMPO3OsfaEb/F0Z8InIOOkcgHbv3o1Nmzbh4MGD8Pf3x6uvvooXXngBtra22nN69uyJjh076rNPIqqnsko13tl3Dl/FpQEA+vk6YMXILrAxNxW5MyIi8egcgCZMmIDnn38ef/zxB7p3717rOa6urnjrrbca3BwRNUxmwW1M3paAxPRbkEiAaf3b4/XH20MqlYjdGhGRqHSeA1RSUmKwc3vqinOAqCU4ejkPr20/hRuqclgrTLDy+S7o5+codltERI2mUecAHT58GDKZDAMHDqxx/ODBg9BoNBg8eLCuJYlIjwRBwPrfU7DswEWoNQI6ulhj7Qtd4dmaC5gSEVXT+Tb4OXPmQK1W33NcEATMmTNHL00RUf0Ul1Vi6vZTWPLDBag1Ap7p4obdk3sy/BAR/YPOI0BJSUnw9/e/57ifnx+Sk5P10hQR6e5ybjH+vTUeyTnFMJFKsDDCH2Mf8YREwvk+RET/pHMAsrGxQUpKCry8vGocT05O5h5hRCI5cDYT//nfnyguq4STtRk+G9MV3TztxG6LiKjZ0vkS2LBhwzB9+nRcvnxZeyw5ORlvvPEGhg4dqtfmiOjBKtUaLP3xAiZtS0BxWSVC29ph32u9GX6IiB5C5wD0/vvvw8LCAn5+fmjbti3atm2Ljh07onXr1vjwww8bo0ciqsWN4jJEbjqOtTFV/xh5qXdbfPVSGHdwJyKqg3pdAjt69CgOHTqE06dPQ6lUonPnznjssccaoz8ig/Dd6evYFpsKV1sFvB0s0dbeAt4OFmhrbwFzeb0XXL+v0+m3MHlbPK4XlMJcLsOyEZ0REeSq99chImqp9LIXWEvDdYBIFz+eycSU7QnQ3OeT5GKjgLeDBbzt/w5G3vaWcGulhKweCxJ+fTwNi779C+VqDdraW+Dzsd3Qwcmqge+CiMjwNfpeYCqVCjExMUhLS0N5eXmN773++us611u9ejU++OADZGVlISgoCKtWrUJoaGit51ZUVCAqKgpbtmxBRkYGfH19sWzZMgwaNKjeNYnq6/ekXEzbkQiNAAwNcoWfixVSclW4kqdCSm4xbpZUILOgFJkFpfgj+UaN58pNpPBqbX4nFFWFI5874aiVhfye1yqtUGPRt39h58l0AMAT/k746LkgWCu4pQURka50DkCnTp3CkCFDUFJSApVKBTs7O+Tl5cHc3ByOjo46B6CdO3di5syZWLt2LcLCwrBixQoMHDgQFy9ehKPjvavWzp8/H9u2bcP69evh5+eHgwcP4umnn8bRo0fRpUuXetUkqo/41Jt45ct4lKs1eDLQBR+PDL5nROemqhwpd8JQVSiqCkdXbqhQXqnBpexiXMouBpBd43m25qbwtrdAW3tLeDtYwMPOHOt+S8GZjAJIJcAbA3wxuY8Pt7QgIqonnS+B9e3bFx06dMDatWthY2OD06dPw9TUFC+88AKmTZuGZ555RqcGwsLC0L17d3z66acAAI1GA3d3d7z22mu1LqxYvc/YlClTtMdGjBgBpVKJbdu21avmP/ESGD3M+cxCjPw8FoWllXisgwO+GBcCuUnd7ylQawRcv3W71nCUcev2fZ/XytwUn4zqgkfbO+jjbRARtSiNegksMTERn3/+OaRSKWQyGcrKyuDt7Y33338fkZGROgWg8vJyxMfHY+7cudpjUqkU4eHhiI2NrfU5ZWVlUChq3uWiVCpx5MiRBtUsKyvTfl1YWFjn90DG52qeCmM3HEdhaSVCPFth7QtddQo/ACCTSuBuZw53O3P06VAzzNwuV1eNEt0Vji7nqeBoZYZFEf5o06pl78VHRNQUdA5ApqamkEqrftg7OjoiLS0NHTt2hI2NDdLT03WqlZeXB7VaDScnpxrHnZyccOHChVqfM3DgQCxfvhyPPfYYfHx8EB0djd27d2u356hPzaioKPz3v//VqXcyTlkFpRjzRRzyisvQ0cUaG8Z31/tdXkq5DP6u1vB35egjEVFj0XkdoC5duuDEiRMAgD59+mDhwoX46quvMH36dHTq1EnvDf7TypUr0b59e/j5+UEul2Pq1KmYMGGCNpTVx9y5c1FQUKB96BrkyDjkq8rxwoY4ZNy6jbb2FvhyYihslJyATERkiHRODUuWLIGLiwsAYPHixWjVqhUmT56M3NxcrFu3Tqda9vb2kMlkyM6uOQE0Ozsbzs7OtT7HwcEBe/fuhUqlQmpqKi5cuABLS0t4e3vXu6aZmRmsra1rPIjuVlRagfGbjiM5pxguNgpsfTEUDlZmYrdFRET1pFMAEgQBjo6O6NGjB4CqS2AHDhxAYWEh4uPjERQUpNOLy+VydOvWDdHR0dpjGo0G0dHR2te4H4VCATc3N1RWVmLXrl0YNmxYg2sS1aa0Qo2XtpzEn9cKYGchx9YXwzgPh4jIwOkcgNq1a6fXS0QzZ87E+vXrsWXLFpw/fx6TJ0+GSqXChAkTAADjxo2rMaE5Li4Ou3fvRkpKCn7//XcMGjQIGo0Gs2fPrnNNorqqUGsw5asExF3Jh5WZCb6cGIp2jpZit0VERA2k0+xNqVSK9u3b48aNG2jfvr1eGhg5ciRyc3OxcOFCZGVlITg4GAcOHNBOYk5LS6sxv6e0tBTz589HSkoKLC0tMWTIEGzduhW2trZ1rklUFxqNgP/87zSiL+TAzESKLyJD0MnNRuy2iIhID3ReB2jfvn14//33sWbNmiaZ9CwGrgNEgiBg4bd/YeuxVJhIJVg/LgT9/LiIJhFRc9ao6wCNGzcOJSUlCAoKglwuh1KprPH9/Px8XUsSNTsf/XQJW4+lQiIBlo8MZvghImphdA5AK1asaIQ2iJqP9b+l4NNfkwEA7w3vhKHcZZ2IqMXROQBFRkY2Rh9EzcLOE2lY/MN5AMDsQb4YE+YpckdERNQYdA5AaWlpD/y+h4dHvZshEtMPZzIxd/cZAMC/+3jj1b7tRO6IiIgai84ByMvLCxLJ/Xegrt6SgsiQxFzKxbQdp6ARgFGhHpgzyE/sloiIqBHpHIBOnTpV4+uKigqcOnUKy5cvx+LFi/XWGFFTOXk1H//eehIVagFPdXbBe8M7PTDkExGR4dM5ANW22nNISAhcXV3xwQcf6LQbPJHY/rpegAmbT6C0QoO+vg5Y/lwwZFKGHyKilq7+O4j+g6+vr3aTVCJDkJJbjMiNx1FUWolQLzusGdMNchO9fSSIiKgZ03kEqLCwsMbXgiAgMzMTb7/9tt5WhyZqbNdv3cbYDceRV1yOAFdrfDE+BEq5TOy2iIioiegcgGxtbe+ZHyEIAtzd3bFjxw69NUbUWG4Ul+GFDXHIuHUb3vYW2DIxFNYKU7HbIiKiJqRzAPrll19qBCCpVAoHBwe0a9cOJiY6lyNqUoWlFYjcdBwpuSq42iiw9aUw2Fuaid0WERE1MZ0TS9++fRuhDaLGV1qhxktbTuJsRiFaW8ix7aUwuNkqH/5EIiJqcXSe8RkVFYWNGzfec3zjxo1YtmyZXpoi0rcKtQavfpWA41fyYWVmgi0TQ+HtYCl2W0REJBKdA9Dnn38OP797F4kLCAjA2rVr9dIUkb59GZuKXy7kQGEqxcYJ3dHJzUbsloiISEQ6B6CsrCy4uLjcc9zBwQGZmZl6aYpIn9QaAZv+uAIAeOtJf3T3shO5IyIiEpvOAcjd3R1//PHHPcf/+OMPuLpy12xqfn4+n41rN2/D1twU/+raRux2iIioGdB5EvTLL7+M6dOno6KiAo8//jgAIDo6GrNnz8Ybb7yh9waJGqp69GdUqAfX+iEiIgD1CECzZs3CjRs38Oqrr6K8vBwAoFAo8Oabb2LOnDl6b5CoIc5nFuJYSj5kUgnGPuIpdjtERNRM6ByAJBIJli1bhgULFuD8+fNQKpVo3749zMy4lgo1P9WjP4M6OcOVt7wTEdEdOgeggoICqNVq2NnZoXv37trj+fn5MDExgbW1tV4bJKqvG8Vl2Jt4HQAwsZeXuM0QEVGzovMk6Oeff77WLS+++eYbPP/883ppikgfvj6ehvJKDTq3sUFXj1Zit0NERM2IzgEoLi4O/fr1u+d43759ERcXp5emiBqqQq3B1mOpAIAJvbzu2b+OiIiMm84BqKysDJWVlfccr6iowO3bt/XSFFFD/XAmE9mFZXCwMsOTgVyegYiIatI5AIWGhmLdunX3HF+7di26deuml6aIGmrTH1cBAC+EeUJuovNfcyIiauF0ngT93nvvITw8HKdPn0b//v0BVK0DdOLECfz00096b5BIV6fSbiIx/RbkMilGh3mI3Q4RETVDOv/TuFevXoiNjYW7uzu++eYb7Nu3D+3atcOff/6JRx99tDF6JNJJ9ehPRJArHKy4PAMREd1L5xEgAAgODsZXX31V45hGo8H+/fvx1FNP6aUxovrIKijFD2eq9qSbwFvfiYjoPuoVgO6WnJyMjRs3YvPmzcjNzUVFRYU++iKql23HUlGpERDqZccd34mI6L7qNTv09u3b+PLLL/HYY4/B19cXR48excKFC3Ht2jV990dUZ6UVamw/ngaAoz9ERPRgOo0AnThxAl988QV27NgBHx8fjBkzBkePHsVnn30Gf3//xuqRqE6+S7yOfFU53GyVeMLfSex2iIioGatzAOrcuTMKCwsxevRoHD16FAEBAQDADVCpWRAEARvv7Ps1tocnTGS89Z2IiO6vzr8lLl68iMceewz9+vXjaA81O8dS8nEhqwgKUyme7+4udjtERNTM1TkApaSkwNfXF5MnT0abNm3wn//8B6dOneIWA9QsVO/6/kzXNrA1l4vcDRERNXd1DkBubm546623kJycjK1btyIrKwu9evVCZWUlNm/ejEuXLjVmn0T3lZ5fgkPnswEAE3p6idsMEREZhHpNlHj88cexbds2ZGZm4tNPP8Uvv/wCPz8/dO7cWd/9ET3Ul7FXIQjAo+3t0d7JSux2iIjIADRopqiNjQ1effVVnDx5EgkJCejbt6+e2iKqG1VZJXacSAfAW9+JiKju9HarTHBwMD755BN9lSOqk90J11BUWom29hbo28FR7HaIiMhA8F5hMlgajYBNR68CACJ7eEIq5YR8IiKqGwYgMli/JeUiJVcFKzMT/CuEt74TEVHdMQCRware9f3ZEHdYmjV4WzsiIjIiDEBkkJJzihFzKRcSCTCet74TEZGO6vTPZl0mN7/++uv1boaorrbcmfvT388JHq3NxW2GiIgMTp0C0Mcff1ynYhKJhAGIGl3B7QrsSrgGAJjIW9+JiKge6hSArly50th9ENXZNyfSUVKuhq+TFXr4tBa7HSIiMkD1ngNUXl6OixcvorKyUp/9ED2QWiNgS+xVAFULH3IvOiIiqg+dA1BJSQlefPFFmJubIyAgAGlpaQCA1157DUuXLtV7g0R3O3QuG9du3kYrc1MM7+ImdjtERGSgdA5Ac+fOxenTp3H48GEoFArt8fDwcOzcuVOvzRH9U/Wu76NCPaAwlYncDRERGSqdF0/Zu3cvdu7ciUceeaTG5YeAgABcvnxZr80R3e2v6wWIu5IPmVSCsT08xW6HiIgMmM4jQLm5uXB0vHfPJZVKxfkY1Kg231n4cHAnZ7jYKMVthoiIDJrOASgkJATff/+99uvq0PPFF1+gR48e+uuM6C43isvw7enrAIAJvdqK3A0RERk6nS+BLVmyBIMHD8a5c+dQWVmJlStX4ty5czh69ChiYmIao0cibI9LQ3mlBkFtbNDVw1bsdoiIyMDpPALUu3dvJCYmorKyEoGBgfjpp5/g6OiI2NhYdOvWrTF6JCNXXqnB1mOpAKpGf3iplYiIGqpeO0j6+Phg/fr1+u6FqFY/ns1ETlEZHK3MMCTQRex2iIioBahTACosLKxzQWtr63o3Q1SbjXcmP7/wiCfkJty/l4iIGq5OAcjW1rbOlx3UanWDGiK6W0LaTZxOvwW5TIrRYR5it0NERC1EnQLQr7/+qv3z1atXMWfOHIwfP15711dsbCy2bNmCqKioxumSjNamO6M/Q4NdYW9pJm4zRETUYtQpAPXp00f753feeQfLly/HqFGjtMeGDh2KwMBArFu3DpGRkfrvkoxSVkEpfjyTCaBq3y8iIiJ90XlCRWxsLEJCQu45HhISguPHj+ulKSIA2HrsKio1AkLb2iHA1UbsdoiIqAXROQC5u7vXegfYF198AXd3d700RVRaocb2uKqNdidy9IeIiPRM59vgP/74Y4wYMQI//vgjwsLCAADHjx9HUlISdu3apfcGyTh9m5iBmyUVcLNV4gl/Z7HbISKiFkbnEaAhQ4YgKSkJERERyM/PR35+PiIiInDp0iUMGTKkMXokIyMIgnbyc2RPT8ikXPiQiIj0q16LqrRp0wZLlizB7t27sXv3bixevLjel79Wr14NLy8vKBQKhIWFPXQe0YoVK+Dr6wulUgl3d3fMmDEDpaWl2u+r1WosWLAAbdu2hVKphI+PD959910IglCv/qjpxabcwIWsIihNZRgZwlvfiYhI/+q1EvStW7ewYcMGnD9/HgAQEBCAiRMnwsZGt4mqO3fuxMyZM7F27VqEhYVhxYoVGDhwIC5evFjrjvPbt2/HnDlzsHHjRvTs2ROXLl3C+PHjIZFIsHz5cgDAsmXLsGbNGmzZsgUBAQE4efIkJkyYABsbG7z++uv1ebt0x43iMlzJU6G9kxVslKaN9jrVoz8jurnBxrzxXoeIiIyXRNBxaOTkyZMYOHAglEolQkNDAQAnTpzA7du38dNPP6Fr1651rhUWFobu3bvj008/BQBoNBq4u7vjtddew5w5c+45f+rUqTh//jyio6O1x9544w3ExcXhyJEjAICnnnoKTk5O2LBhg/acESNGQKlUYtu2bXXqq7CwEDY2NigoKODK1ndoNAIGrfwNl7KLAQCerc3RydUGAW7W6ORqg05uNrCzkDf4ddJulKDPh79CEICfZz6Gdo5WDa5JRETGQZff3zqPAM2YMQNDhw7F+vXrYWJS9fTKykq89NJLmD59On777bc61SkvL0d8fDzmzp2rPSaVShEeHo7Y2Nhan9OzZ09s27YNx48fR2hoKFJSUvDDDz9g7NixNc5Zt24dLl26hA4dOuD06dM4cuSIdoSoNmVlZSgrK9N+rcvWH8YiNuUGLmUXQyIBBAFIvVGC1Bsl+P7OOj0A4GarRICrNTq52aDTnWDkaK3Q6XW2xF6FIACPtrdn+CEiokajcwA6efJkjfADACYmJpg9e3at6wPdT15eHtRqNZycnGocd3JywoULF2p9zujRo5GXl4fevXtDEARUVlZi0qRJmDdvnvacOXPmoLCwEH5+fpDJZFCr1Vi8eDHGjBlz316ioqLw3//+t869G6PqW9JfCPPEGwM64GxGIc5eL8DZjAL8db0QV/JUyLh1Gxm3buOnc9na5zlamVUFIldrBLjZINDNBi42ilq3Vikuq8Q3J9IBABN7tW2aN0ZEREZJ5wBkbW2NtLQ0+Pn51Tienp4OK6vG/Rf74cOHsWTJEnz22WcICwtDcnIypk2bhnfffRcLFiwAAHzzzTf46quvsH37dgQEBCAxMRHTp0+Hq6vrfVepnjt3LmbOnKn9urCwkGsa3SW3qAwH/8oCAIwK9YCtuRy929ujd3t77TlFpRX463qhNhCdzSjA5dxi5BSV4ZcLOfjlQo72XDsL+d8jRa5Vo0UedubYFX8NRWWV8La3QJ8ODk3+PomIyHjoHIBGjhyJF198ER9++CF69uwJAPjjjz8wa9asGttjPIy9vT1kMhmys7NrHM/Ozoazc+3rvixYsABjx47FSy+9BAAIDAyESqXCK6+8grfeegtSqRSzZs3CnDlz8Pzzz2vPSU1NRVRU1H0DkJmZGczMuM/U/exKuIZKjYBgd1v4u9Z+TdVKYYpHvFvjEe/W2mMl5ZU4n1lYNVqUUYCz1wuRlF2EfFU5fk/Kw+9JeXc93wS4MxstsqcXpLz1nYiIGpHOAejDDz+ERCLBuHHjUFlZCQAwNTXF5MmTsXTp0jrXkcvl6NatG6KjozF8+HAAVZOgo6OjMXXq1FqfU1JSAqm05p37MpkMALS3ud/vHI1GU+fe6G8ajYAdx6suf40O1e2WdHO5Cbp52qGbp532WGmFGheziu5cPivEX9cLcCGzCEWlVX+XrBUmGNGtjf7eABERUS10DkByuRwrV65EVFQULl++DADw8fGBubm5zi8+c+ZMREZGIiQkBKGhoVixYgVUKhUmTJgAABg3bhzc3Ny0u8xHRERg+fLl6NKli/YS2IIFCxAREaENQhEREVi8eDE8PDwQEBCAU6dOYfny5Zg4caLO/VHV5OerN0pgZWaCp4JcGlxPYSpDkLstgtxttcfKKzVIyinC+cwiBLhaw9KsXqszEBER1Vm9f9OYm5sjMDCwQS8+cuRI5ObmYuHChcjKykJwcDAOHDignRidlpZWYzRn/vz5kEgkmD9/PjIyMuDg4KANPNVWrVqFBQsW4NVXX0VOTg5cXV3x73//GwsXLmxQr8Zq+53Rn+Fd3GAub5xgIjeRIsDVhhueEhFRk6nzOkB1HUHZuHFjgxpqDrgOUJW84jL0iIpGhVrAD68/et/5P0RERM1Bo6wDtHnzZnh6eqJLly7cVsJI/F/8NVSoBQQ9YPIzERGRIapzAJo8eTK+/vprXLlyBRMmTMALL7wAOzu7hz+RDNLdk5/H6Dj5mYiIqLmr82aoq1evRmZmJmbPno19+/bB3d0dzz33HA4ePMgRoRbo2J3Jz5Z6mvxMRETUnOi0G7yZmRlGjRqFQ4cO4dy5cwgICMCrr74KLy8vFBcXN1aPJIK/Jz+7NtrkZyIiIrHoFIBqPFEqhUQigSAIUKvV+uyJRJZXXHPlZyIiopZGpwBUVlaGr7/+Gk888QQ6dOiAM2fO4NNPP0VaWhosLS0bq0dqYrvumvzMW9OJiKglqvO1jVdffRU7duyAu7s7Jk6ciK+//hr29vYPfyIZFEEQ8LV25Wfuh0ZERC1TnQPQ2rVr4eHhAW9vb8TExCAmJqbW83bv3q235qjpxd49+bmzq9jtEBERNYo6B6Bx48ZBIuEGlS3d9riq0Z9hwa6w4JYURETUQum0ECK1bDfumvw8OoyTn4mIqOWq911g1PLsSrgz+bkN9+UiIqKWjQGIAFRPfk4HwFvfiYio5WMAIgBVk5+v5KlgaWaCiCBOfiYiopaNAYgAQDv6w8nPRERkDBiAqGry81mu/ExERMaDAYiwK+EaytUadG5jg05unPxMREQtHwOQkbt78vNojv4QEZGRYAAycsdS8nElTwULuYyTn4mIyGgwABm57Xf2/RrWxY2Tn4mIyGgwABmxuyc/8/IXEREZEwYgI7Y7IYOTn4mIyCgxABmpqsnPVZe/eOs7EREZGwYgI3UsJR8pnPxMRERGigHISH191+RnS05+JiIiI8MAZITyVeU4wMnPRERkxBiAjNCu+KqVnwPdOPmZiIiMEwOQkbl78vPoMI7+EBGRcWIAMjJxVzj5mYiIiAHIyFSP/gwN5uRnIiIyXgxARiRfVY4fz3DyMxEREQOQEdmd8Pfk58A2nPxMRETGiwHISAiCoN34lCs/ExGRsWMAMhLHr+QjJbdq8vPQYE5+JiIi48YAZCS2c/IzERGRFgOQEbjJyc9EREQ1MAAZgV13Jj93crPm5GciIiIwALV4nPxMRER0LwagFq568rO5XIZhwW5it0NERNQsMAC1cNUrPw8LduXkZyIiojsYgFqwm6py/HC2avIzL38RERH9jQGoBduVcA3llRoEuFoj0I2Tn4mIiKoxALVQgiBoL3+NDvOARCIRuSMiIqLmgwGohTpx9SYu35n8PDSIKz8TERHdjQGohdoelwoAGBrkCiuFqcjdEBERNS8MQC3Q3ZOfR4dx8jMREdE/MQC1QLtPZXDyMxER0QMwALUwd09+HhXKyc9ERES1YQBqYU5cvYnknOI7Kz9z8jMREVFtGIBamOrRH05+JiIiuj8GoBbkVkk5vj+TCYArPxMRET0IA1ALsjuhavKzv4s1Orfh5GciIqL7YQBqQfYmZgAAng915+RnIiKiB2AAaiFSb6jw57UCSCXAkEAXsdshIiJq1hiAWoj9f1bN/enVzh72lmYid0NERNS8MQC1ENUB6KnOHP0hIiJ6GAagFiA5pxjnMwthIpVgYICz2O0QERE1ewxALcD+P68DAB5tbw9bc7nI3RARETV/DEAGThCEuy5/ceVnIiKiumAAMnAXs4uQnFMMuUyKJwKcxG6HiIjIIDAAGbj9p6tGf/r4OsCaW18QERHVCQOQAau6/FU1/4d3fxEREdUdA5AB++t6Ia7eKIHCVIrwjrz8RUREVFeiB6DVq1fDy8sLCoUCYWFhOH78+APPX7FiBXx9faFUKuHu7o4ZM2agtLS0xjkZGRl44YUX0Lp1ayiVSgQGBuLkyZON+TZEse/O6M/jfo6wMDMRuRsiIiLDIepvzZ07d2LmzJlYu3YtwsLCsGLFCgwcOBAXL16Eo6PjPedv374dc+bMwcaNG9GzZ09cunQJ48ePh0QiwfLlywEAN2/eRK9evdCvXz/8+OOPcHBwQFJSElq1atXUb69RCYKA73n3FxERUb2IGoCWL1+Ol19+GRMmTAAArF27Ft9//z02btyIOXPm3HP+0aNH0atXL4wePRoA4OXlhVGjRiEuLk57zrJly+Du7o5NmzZpj7Vt2/aBfZSVlaGsrEz7dWFhYYPeV1NITL+Fazdvw1wuQz/fe8MiERER3Z9ol8DKy8sRHx+P8PDwv5uRShEeHo7Y2Nhan9OzZ0/Ex8drL5OlpKTghx9+wJAhQ7TnfPfddwgJCcGzzz4LR0dHdOnSBevXr39gL1FRUbCxsdE+3N3d9fAOG1f12j/hHZ2glMtE7oaIiMiwiBaA8vLyoFar4eRUc/Kuk5MTsrKyan3O6NGj8c4776B3794wNTWFj48P+vbti3nz5mnPSUlJwZo1a9C+fXscPHgQkydPxuuvv44tW7bct5e5c+eioKBA+0hPT9fPm2wkGs3dl7949xcREZGuRJ8ErYvDhw9jyZIl+Oyzz5CQkIDdu3fj+++/x7vvvqs9R6PRoGvXrliyZAm6dOmCV155BS+//DLWrl1737pmZmawtrau8WjO4tNuIquwFFYKE/TxdRC7HSIiIoMj2hwge3t7yGQyZGdn1zienZ0NZ+faN/RcsGABxo4di5deegkAEBgYCJVKhVdeeQVvvfUWpFIpXFxc4O/vX+N5HTt2xK5duxrnjYhg/+mqu78G+DvDzISXv4iIiHQl2giQXC5Ht27dEB0drT2m0WgQHR2NHj161PqckpISSKU1W5bJqgKAIAgAgF69euHixYs1zrl06RI8PT312b5o1BoB35+pukT4VBAvfxEREdWHqHeBzZw5E5GRkQgJCUFoaChWrFgBlUqlvSts3LhxcHNzQ1RUFAAgIiICy5cvR5cuXRAWFobk5GQsWLAAERER2iA0Y8YM9OzZE0uWLMFzzz2H48ePY926dVi3bp1o71Of4lJuIK+4DLbmpujdzl7sdoiIiAySqAFo5MiRyM3NxcKFC5GVlYXg4GAcOHBAOzE6LS2txojP/PnzIZFIMH/+fGRkZMDBwQERERFYvHix9pzu3btjz549mDt3Lt555x20bdsWK1aswJgxY5r8/TWGfXcmPw8KcIapzKCmcBERETUbEqH62hFpFRYWwsbGBgUFBc1qQnSFWoPQxT/jZkkFtr0Yht7tOQJERERUTZff3xxCMCBHL9/AzZIKtLaQ4xFvO7HbISIiMlgMQAak+u6vwYHOMOHlLyIionrjb1EDUV6pwcG/7tz9xb2/iIiIGoQByED8npSLwtJKOFqZobsXL38RERE1BAOQgaje+2tIoAtkUonI3RARERk2BiADUFqhxqFzVStmR3DxQyIiogZjADIAhy/morisEq42CnRxbyV2O0RERAaPAcgA7P+z6u6vJzu7QMrLX0RERA3GANTMlZRXIvp8DgAgIoh3fxEREekDA1AzF30+B7cr1PCwM0egm43Y7RAREbUIDEDNXPXlr6c6u0Ai4eUvIiIifWAAasaKSivw68VcAFz8kIiISJ8YgJqxn89no7xSA28HC3R0sRK7HSIiohaDAagZ23+6avHDpzq78vIXERGRHjEANVMFJRX4Lanq8ldEZy5+SEREpE8MQM3UwXNZqFAL8HWyQnsnXv4iIiLSJwagZqp676+nOPpDRESkdwxAzVC+qhx/JOcBAJ7i4odERER6xwDUDB04mwW1RkCAqzXa2luI3Q4REVGLwwDUDP29+CFHf4iIiBoDA1Azk1tUhmMpNwBw/g8REVFjYQBqZn48mwmNAAS528LdzlzsdoiIiFokBqBmZt/pqstfXPuHiIio8TAANSOZBbdx4upNAMCTDEBERESNhgGoGfn+zto/3b1awcVGKXI3RERELRcDUDPy9+KHvPuLiIioMTEANRPp+SVITL8FqQQYHOgsdjtEREQtGgNQM/H9marRn7C2reFopRC5GyIiopaNAaiZ0C5+GMTJz0RERI2NAagZuJqnwtmMQsikEgzuxABERETU2BiAmoHq0Z+ePq1hZyEXuRsiIqKWjwGoGai++yuCd38RERE1CQYgkSXnFOFCVhFMZRIMDODdX0RERE2BAUhk+05Xjf482t4BNuamIndDRERkHBiARCQIwt93f3HrCyIioibDACSi85lFuJyrgtxEiif8ncRuh4iIyGgwAImoevSnn68DrBS8/EVERNRUGIBEUnX5i3t/ERERiYEBSCRnMgqQll8CpakM/Ts6it0OERGRUWEAEkn16M/jHR1hLjcRuRsiIiLjwgAkAkEQ8L128UPe/UVERNTUGIBEkJB2Cxm3bsNCLkNfX17+IiIiamoMQCKovvvrCX8nKExlIndDRERkfBiAmphGI+CHM7z7i4iISEwMQE3sxNV8ZBeWwUphgkc72IvdDhERkVFiAGpi1Xd/DQxwhpkJL38RERGJgQGoCVWqNfjxbPXlL979RUREJBYGoCYUdyUfecXlaGVuil7tePmLiIhILAxATSizoBTWChMM6uQMUxn/0xMREYmFSxA3oX91a4OhQa5QlVWK3QoREZFRYwBqYnITKeQmcrHbICIiMmq8DkNERERGhwGIiIiIjA4DEBERERkdBiAiIiIyOgxAREREZHQYgIiIiMjoMAARERGR0WEAIiIiIqPDAERERERGhwGIiIiIjE6zCECrV6+Gl5cXFAoFwsLCcPz48Qeev2LFCvj6+kKpVMLd3R0zZsxAaWlprecuXboUEokE06dPb4TOiYiIyBCJHoB27tyJmTNnYtGiRUhISEBQUBAGDhyInJycWs/fvn075syZg0WLFuH8+fPYsGEDdu7ciXnz5t1z7okTJ/D555+jc+fOjf02iIiIyICIHoCWL1+Ol19+GRMmTIC/vz/Wrl0Lc3NzbNy4sdbzjx49il69emH06NHw8vLCgAEDMGrUqHtGjYqLizFmzBisX78erVq1emAPZWVlKCwsrPEgIiKilkvU3eDLy8sRHx+PuXPnao9JpVKEh4cjNja21uf07NkT27Ztw/HjxxEaGoqUlBT88MMPGDt2bI3zpkyZgieffBLh4eF47733HthHVFQU/vvf/95znEGIiIjIcFT/3hYE4aHnihqA8vLyoFar4eTkVOO4k5MTLly4UOtzRo8ejby8PPTu3RuCIKCyshKTJk2qcQlsx44dSEhIwIkTJ+rUx9y5czFz5kzt1xkZGfD394e7u3s93hURERGJqaioCDY2Ng88R9QAVB+HDx/GkiVL8NlnnyEsLAzJycmYNm0a3n33XSxYsADp6emYNm0aDh06BIVCUaeaZmZmMDMz035taWmJ9PR0WFlZQSKR6LX/wsJCuLu7Iz09HdbW1gZTm/XFq8364tU29PqG3Luh1zfk3g25viAIKCoqgqur60PPFTUA2dvbQyaTITs7u8bx7OxsODs71/qcBQsWYOzYsXjppZcAAIGBgVCpVHjllVfw1ltvIT4+Hjk5Oejatav2OWq1Gr/99hs+/fRTlJWVQSaTPbAvqVSKNm3aNPDdPZi1tXWj/KVq7NqsL15t1hevtqHXN+TeDb2+IfduqPUfNvJTTdRJ0HK5HN26dUN0dLT2mEajQXR0NHr06FHrc0pKSiCV1my7OtAIgoD+/fvjzJkzSExM1D5CQkIwZswYJCYmPjT8EBERUcsn+iWwmTNnIjIyEiEhIQgNDcWKFSugUqkwYcIEAMC4cePg5uaGqKgoAEBERASWL1+OLl26aC+BLViwABEREZDJZLCyskKnTp1qvIaFhQVat259z3EiIiIyTqIHoJEjRyI3NxcLFy5EVlYWgoODceDAAe3E6LS0tBojPvPnz4dEIsH8+fORkZEBBwcHREREYPHixWK9BZ2YmZlh0aJFNeYcGUJt1hevNuuLV9vQ6xty74Ze35B7bwn160Ii1OVeMSIiIqIWRPSFEImIiIiaGgMQERERGR0GICIiIjI6DEBERERkdBiAmtDq1avh5eUFhUKBsLCwezZwbYjffvsNERERcHV1hUQiwd69e/VWOyoqCt27d4eVlRUcHR0xfPhwXLx4UW/116xZg86dO2sXxOrRowd+/PFHvdW/29KlSyGRSDB9+nS91Hv77bchkUhqPPz8/PRSu1pGRgZeeOEFtG7dGkqlEoGBgTh58qReant5ed3Tv0QiwZQpUxpcW61WY8GCBWjbti2USiV8fHzw7rvv1mmPnroqKirC9OnT4enpCaVSiZ49e9Z5C5x/ethnSBAELFy4EC4uLlAqlQgPD0dSUpJeau/evRsDBgxA69atIZFIkJiYqLfeKyoq8OabbyIwMBAWFhZwdXXFuHHjcP36db3UB6o+B35+frCwsECrVq0QHh6OuLg4vdW/26RJkyCRSLBixQq91B4/fvw9f/8HDRqk197Pnz+PoUOHwsbGBhYWFujevTvS0tL0Ur+2z69EIsEHH3ygl/rFxcWYOnUq2rRpA6VSqd20XB+1s7OzMX78eLi6usLc3ByDBg2q82dKHxiAmsjOnTsxc+ZMLFq0CAkJCQgKCsLAgQORk5Ojl/oqlQpBQUFYvXq1XurdLSYmBlOmTMGxY8dw6NAhVFRUYMCAAVCpVHqp36ZNGyxduhTx8fE4efIkHn/8cQwbNgx//fWXXupXO3HiBD7//HN07txZr3UDAgKQmZmpfRw5ckRvtW/evIlevXrB1NQUP/74I86dO4ePPvoIrVq10kv9EydO1Oj90KFDAIBnn322wbWXLVuGNWvW4NNPP8X58+exbNkyvP/++1i1alWDa1d76aWXcOjQIWzduhVnzpzBgAEDEB4ejoyMDJ1rPewz9P777+OTTz7B2rVrERcXBwsLCwwcOBClpaUNrq1SqdC7d28sW7ZM574fVr+kpAQJCQlYsGABEhISsHv3bly8eBFDhw7VS30A6NChAz799FOcOXMGR44cgZeXFwYMGIDc3Fy91K+2Z88eHDt2rE7bHOhSe9CgQTU+B19//bXe6l++fBm9e/eGn58fDh8+jD///BMLFiyo81ZND6t/d9+ZmZnYuHEjJBIJRowYoZf6M2fOxIEDB7Bt2zacP38e06dPx9SpU/Hdd981qLYgCBg+fDhSUlLw7bff4tSpU/D09ER4eLjefrc8lEBNIjQ0VJgyZYr2a7VaLbi6ugpRUVF6fy0Awp49e/Ret1pOTo4AQIiJiWm012jVqpXwxRdf6K1eUVGR0L59e+HQoUNCnz59hGnTpuml7qJFi4SgoCC91KrNm2++KfTu3bvR6v/TtGnTBB8fH0Gj0TS41pNPPilMnDixxrFnnnlGGDNmTINrC4IglJSUCDKZTNi/f3+N4127dhXeeuutBtX+52dIo9EIzs7OwgcffKA9duvWLcHMzEz4+uuvG1T7bleuXBEACKdOnapH1w+vX+348eMCACE1NbVR6hcUFAgAhJ9//llv9a9duya4ubkJZ8+eFTw9PYWPP/5YL7UjIyOFYcOG6VyrrvVHjhwpvPDCC41W/5+GDRsmPP7443qrHxAQILzzzjs1jtXnM/bP2hcvXhQACGfPntUeU6vVgoODg7B+/Xqde68PjgA1gfLycsTHxyM8PFx7TCqVIjw8HLGxsSJ2Vj8FBQUAADs7O73XVqvV2LFjB1Qq1X23Q6mPKVOm4Mknn6zx/4G+JCUlwdXVFd7e3hgzZkydh7br4rvvvkNISAieffZZODo6okuXLli/fr3e6t+tvLwc27Ztw8SJE/WyCXDPnj0RHR2NS5cuAQBOnz6NI0eOYPDgwQ2uDQCVlZVQq9X3/EtaqVTqdRQOAK5cuYKsrKwaf39sbGwQFhZmsJ9hiUQCW1tbvdcuLy/HunXrYGNjg6CgIL3U1Gg0GDt2LGbNmoWAgAC91Lzb4cOH4ejoCF9fX0yePBk3btzQS12NRoPvv/8eHTp0wMCBA+Ho6IiwsDC9TlG4W3Z2Nr7//nu8+OKLeqvZs2dPfPfdd8jIyIAgCPj1119x6dIlDBgwoEF1y8rKAKDG51cqlcLMzEzvn9/7YQBqAnl5eVCr1drVras5OTkhKytLpK7qR6PRYPr06ejVq5detxY5c+YMLC0tYWZmhkmTJmHPnj3w9/fXS+0dO3YgISFBu52KPoWFhWHz5s04cOAA1qxZgytXruDRRx9FUVGRXuqnpKRgzZo1aN++PQ4ePIjJkyfj9ddfx5YtW/RS/2579+7FrVu3MH78eL3UmzNnDp5//nn4+fnB1NQUXbp0wfTp0zFmzBi91LeyskKPHj3w7rvv4vr161Cr1di2bRtiY2ORmZmpl9eoVv05bQmf4dLSUrz55psYNWqUXjeh3L9/PywtLaFQKPDxxx/j0KFDsLe310vtZcuWwcTEBK+//rpe6t1t0KBB+PLLLxEdHY1ly5YhJiYGgwcPhlqtbnDtnJwcFBcXY+nSpRg0aBB++uknPP3003jmmWcQExOjh+5r2rJlC6ysrPDMM8/oreaqVavg7++PNm3aQC6XY9CgQVi9ejUee+yxBtX18/ODh4cH5s6di5s3b6K8vBzLli3DtWvX9P75vR/Rt8IgwzJlyhScPXtW7wnd19cXiYmJKCgowP/93/8hMjISMTExDQ5B6enpmDZtGg4dOlTna+66uHs0o3PnzggLC4Onpye++eYbvfwrTKPRICQkBEuWLAEAdOnSBWfPnsXatWsRGRnZ4Pp327BhAwYPHqzT/IoH+eabb/DVV19h+/btCAgIQGJiIqZPnw5XV1e99b5161ZMnDgRbm5ukMlk6Nq1K0aNGoX4+Hi91G9pKioq8Nxzz0EQBKxZs0avtfv164fExETk5eVh/fr1eO655xAXFwdHR8cG1Y2Pj8fKlSuRkJCgl5HJf3r++ee1fw4MDETnzp3h4+ODw4cPo3///g2qrdFoAADDhg3DjBkzAADBwcE4evQo1q5diz59+jSo/j9t3LgRY8aM0evPulWrVuHYsWP47rvv4Onpid9++w1TpkyBq6trg0bUTU1NsXv3brz44ouws7ODTCZDeHg4Bg8erNcbJR6EI0BNwN7eHjKZDNnZ2TWOZ2dnw9nZWaSudDd16lTs378fv/76K9q0aaPX2nK5HO3atUO3bt0QFRWFoKAgrFy5ssF14+PjkZOTg65du8LExAQmJiaIiYnBJ598AhMTE738K+9utra26NChA5KTk/VSz8XF5Z4Q2LFjR71eZgOA1NRU/Pzzz3jppZf0VnPWrFnaUaDAwECMHTsWM2bM0OtInI+PD2JiYlBcXIz09HQcP34cFRUV8Pb21ttrANB+Tg35M1wdflJTU3Ho0CG9jv4AVZtOt2vXDo888gg2bNgAExMTbNiwocF1f//9d+Tk5MDDw0P7GU5NTcUbb7wBLy+vhjf+D97e3rC3t9fLZ9je3h4mJiZN8hn+/fffcfHiRb1+hm/fvo158+Zh+fLliIiIQOfOnTF16lSMHDkSH374YYPrd+vWDYmJibh16xYyMzNx4MAB3LhxQ++f3/thAGoCcrkc3bp1Q3R0tPaYRqNBdHS0Xue5NBZBEDB16lTs2bMHv/zyC9q2bdvor6nRaLTXiBuif//+OHPmDBITE7WPkJAQjBkzBomJiZDJZHro9m/FxcW4fPkyXFxc9FKvV69e9yw5cOnSJXh6euqlfrVNmzbB0dERTz75pN5qlpSU1NjIGABkMpn2X8X6ZGFhARcXF9y8eRMHDx7EsGHD9Fq/bdu2cHZ2rvEZLiwsRFxcnEF8hqvDT1JSEn7++We0bt260V9TX5/hsWPH4s8//6zxGXZ1dcWsWbNw8OBBPXRa07Vr13Djxg29fIblcjm6d+/eJJ/hDRs2oFu3bnqbdwVU/b2pqKho9M+xjY0NHBwckJSUhJMnT+r983s/vATWRGbOnInIyEiEhIQgNDQUK1asgEqlwoQJE/RSv7i4uMa/WK5cuYLExETY2dnBw8OjQbWnTJmC7du349tvv4WVlZV2zoONjQ2USmWDagPA3LlzMXjwYHh4eKCoqAjbt2/H4cOH9fLDzcrK6p65ShYWFmjdurVe5jD95z//QUREBDw9PXH9+nUsWrQIMpkMo0aNanBtAJgxYwZ69uyJJUuW4LnnnsPx48exbt06rFu3Ti/1gapfVJs2bUJkZCRMTPT3IyEiIgKLFy+Gh4cHAgICcOrUKSxfvhwTJ07U22scPHgQgiDA19cXycnJmDVrFvz8/Or1uXrYZ2j69Ol477330L59e7Rt2xYLFiyAq6srhg8f3uDa+fn5SEtL067NU/0L09nZuU4jTA+q7+Lign/9619ISEjA/v37oVartZ9hOzs7yOXyBtVv3bo1Fi9ejKFDh8LFxQV5eXlYvXo1MjIy6rycwsP++/wzsJmamsLZ2Rm+vr4Nqm1nZ4f//ve/GDFiBJydnXH58mXMnj0b7dq1w8CBA/XS+6xZszBy5Eg89thj6NevHw4cOIB9+/bh8OHDeqkPVIXx//3vf/joo4/qVFOX+n369MGsWbOgVCrh6emJmJgYfPnll1i+fHmDa//vf/+Dg4MDPDw8cObMGUybNg3Dhw9v8ATrOmuSe81IEARBWLVqleDh4SHI5XIhNDRUOHbsmN5q//rrrwKAex6RkZENrl1bXQDCpk2bGlxbEARh4sSJgqenpyCXywUHBwehf//+wk8//aSX2rXR523wI0eOFFxcXAS5XC64ubkJI0eOFJKTk/VSu9q+ffuETp06CWZmZoKfn5+wbt06vdY/ePCgAEC4ePGiXusWFhYK06ZNEzw8PASFQiF4e3sLb731llBWVqa319i5c6fg7e0tyOVywdnZWZgyZYpw69atetV62GdIo9EICxYsEJycnAQzMzOhf//+df5v9rDamzZtqvX7ixYtanD96lvra3v8+uuvDa5/+/Zt4emnnxZcXV0FuVwuuLi4CEOHDhWOHz9ep9p1+e/zT7rcBv+g2iUlJcKAAQMEBwcHwdTUVPD09BRefvllISsrS6+9b9iwQWjXrp2gUCiEoKAgYe/evXqt//nnnwtKpbJef/cfVj8zM1MYP3684OrqKigUCsHX11f46KOP6rRUxsNqr1y5UmjTpo1gamoqeHh4CPPnz9frz4eHkQhCE802IiIiImomOAeIiIiIjA4DEBERERkdBiAiIiIyOgxAREREZHQYgIiIiMjoMAARERGR0WEAIiIiIqPDAERERERGhwGIiKgB+vbti+nTp4vdBhHpiAGIiJq98ePHQyKRQCKRwNTUFG3btsXs2bNRWloqdmtEZKC4GSoRGYRBgwZh06ZNqKioQHx8PCIjIyGRSLBs2TKxWyMiA8QRICIyCGZmZnB2doa7uzuGDx+O8PBwHDp0CABQVlaG119/HY6OjlAoFOjduzdOnDihfe7mzZtha2tbo97evXshkUi0X7/99tsIDg7G1q1b4eXlBRsbGzz//PMoKirSnqNSqTBu3DhYWlrCxcWlXrtvE1HzwABERAbn7NmzOHr0KORyOQBg9uzZ2LVrF7Zs2YKEhAS0a9cOAwcORH5+vk51L1++jL1792L//v3Yv38/YmJisHTpUu33Z82ahZiYGHz77bf46aefcPjwYSQkJOj1vRFR02AAIiKDsH//flhaWkKhUCAwMBA5OTmYNWsWVCoV1qxZgw8++ACDBw+Gv78/1q9fD6VSiQ0bNuj0GhqNBps3b0anTp3w6KOPYuzYsYiOjgYAFBcXY8OGDfjwww/Rv39/BAYGYsuWLaisrGyMt0tEjYxzgIjIIPTr1w9r1qyBSqXCxx9/DBMTE4wYMQJ//vknKioq0KtXL+25pqamCA0Nxfnz53V6DS8vL1hZWWm/dnFxQU5ODoCq0aHy8nKEhYVpv29nZwdfX98GvjMiEgMDEBEZBAsLC7Rr1w4AsHHjRgQFBWHDhg3o3r37Q58rlUohCEKNYxUVFfecZ2pqWuNriUQCjUbTgK6JqLniJTAiMjhSqRTz5s3D/Pnz4ePjA7lcjj/++EP7/YqKCpw4cQL+/v4AAAcHBxQVFUGlUmnPSUxM1Ok1fXx8YGpqiri4OO2xmzdv4tKlSw17M0QkCgYgIjJIzz77LGQyGdasWYPJkydj1qxZOHDgAM6dO4eXX34ZJSUlePHFFwEAYWFhMDc3x7x583D58mVs374dmzdv1un1LC0t8eKLL2LWrFn45ZdfcPbsWYwfPx5SKX+MEhkiXgIjIoNkYmKCqVOn4v3338eVK1eg0WgwduxYFBUVISQkBAcPHkSrVq0AVM3V2bZtG2bNmoX169ejf//+ePvtt/HKK6/o9JoffPABiouLERERASsrK7zxxhsoKChojLdHRI1MIvzzwjgRERFRC8exWyIiIjI6DEBERERkdBiAiIiIyOgwABEREZHRYQAiIiIio8MAREREREaHAYiIiIiMDgMQERERGR0GICIiIjI6DEBERERkdBiAiIiIyOj8P3tOtcBB4BPTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(model_accuracy_arr)\n",
    "plt.xticks(range(num_rounds))\n",
    "plt.xlabel(\"Round\")\n",
    "plt.ylabel(\"Model Accuracy\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n",
    "\n",
    "[1] Griffin, G., Holub, A., & Perona, P. (2022). Caltech 256 (1.0). https://doi.org/10.22002/D1.20087\n",
    "\n",
    "[2] Goh, H. W., & Mueller, J. ActiveLab: Active Learning with Re-Labeling by Multiple Annotators. https://arxiv.org/abs/2301.11856"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "vscode": {
   "interpreter": {
    "hash": "00885e89789f58e60dbba52a405dc834aaf92411914fde0d391f9b48289a0610"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}