2.1-traffic-signs-recognition.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 交通標誌圖像辨識 (Traffic Sign Recognition)\n",
    "\n",
    "在這篇文章裡, 我們將通過解決一個現實世界的問題來練手深度學習。\n",
    "我們要使用的圖像是[德國交通標誌識別基準（GTSRB）](http://benchmark.ini.rub.de/?section=gtsrb&subsection=news)資料集。問題是要識別圖像中不同種類的交通標誌。解決這個問題對於自駕車在道路上行駛至關重要。\n",
    "\n",
    "![GTSRB](https://chsasank.github.io/assets/images/traffic/classes.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "這個數據集裡面的圖像具有不同大小，光照條件，遮擋情況下的43種不同交通標誌符號，圖像的成像情況與真實你實際在不同時間點下去\n",
    "路邊開車走路時看到的交通標誌的情形非常相似。訓練集包括大約39,000個圖像，而測試集大約有12,000個圖像。圖像不能保證是固定\n",
    "的尺寸，標誌不一定在每個圖像中都是居中。每個圖像包含實際交通標誌周圍10％左右的邊界。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在進行後面的相關動作之前, 你需要從[GTSRB網站](http://benchmark.ini.rub.de/?section=gtsrb&subsection=dataset)下載\"Images and annotations”(圖像資料與標籤資訊)\"以便進行模型的訓練和測試，並將其解壓縮到一個文件夾中。\n",
    "同時再另外下載'Extended annotations including class ids'資料集來作為模型的測試集。\n",
    "我們在這個範列中會將以上的資料組織鳥戮如下標示的目錄結構：\n",
    "\n",
    "    GTSRB\n",
    "    ├── GT-final_test.csv\n",
    "    ├── Final_Test\n",
    "    │   └── Images\n",
    "    └── Final_Training\n",
    "        └── Images\n",
    "            ├── 00000\n",
    "            ├── 00001\n",
    "            ├── ...\n",
    "            ├── 00041\n",
    "            └── 00042"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 資料預處理 (Data Preprocessing)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore') # 把一些惱人的警示訊息暫時忽略\n",
    "\n",
    "import numpy as np\n",
    "from skimage import io, color, exposure, transform\n",
    "from sklearn.model_selection import train_test_split\n",
    "import os\n",
    "import glob\n",
    "import h5py\n",
    "\n",
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "from keras.models import Sequential, model_from_json\n",
    "from keras.layers.core import Dense, Dropout, Activation, Flatten\n",
    "from keras.layers.convolutional import Conv2D\n",
    "from keras.layers.pooling import MaxPooling2D\n",
    "\n",
    "from keras.optimizers import SGD\n",
    "import keras.utils as np_utils\n",
    "from keras.callbacks import LearningRateScheduler, ModelCheckpoint\n",
    "\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "NUM_CLASSES = 43 # 共有43種要辨識的交通標誌\n",
    "IMG_SIZE = 48 # 每張圖像最後都要整理成 48x48的大小"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 圖像亮度直方圖均衡化 (Histogram equalization)\n",
    "\n",
    "從上面的代表性圖像可以看出，圖像在亮度上(illumination)有很大的差異不同。他們也有不同的大小。所以，我們來編寫一個函數，在HSV顏色空間中進行直方圖均衡，並將圖像調整為成一樣的標準大小。\n",
    "\n",
    "\"[直方圖均衡化](https://zh.wikipedia.org/wiki/%E7%9B%B4%E6%96%B9%E5%9B%BE%E5%9D%87%E8%A1%A1%E5%8C%96)\"是圖像處理領域中利用圖像`直方圖`對`對比度`進行調整的方法。\n",
    "\n",
    "這種方法通常用來增加許多圖像的全局對比度，尤其是當圖像的有用數據的對比度相當接近的時候。通過這種方法，亮度可以更好地在直方圖上分布。這樣就可以用於增強局部的對比度而不影響整體的對比度，直方圖均衡化通過有效地擴展常用的亮度來實現這種功能。\n",
    "\n",
    "這種方法對於背景和前景都太亮或者太暗的圖像非常有用, 而且它是一個相當直觀的技術並且是可逆操作，如果已知均衡化函數，那麼就可以恢復原始的直方圖，並且計算量也不大。 但是這種方法的一個缺點是它它可能會增加背景雜訊的對比度並且降低有用訊號的對比度。\n",
    "\n",
    "詳細說明請見: [Wikipedia-直方圖均衡化](https://zh.wikipedia.org/wiki/%E7%9B%B4%E6%96%B9%E5%9B%BE%E5%9D%87%E8%A1%A1%E5%8C%96)\n",
    "\n",
    "### 處理前:\n",
    "![Input image to preprocess_img(scaled 4x)](https://chsasank.github.io/assets/images/traffic/input.png \"Input image to preprocess_img(scaled 4x)\")\n",
    "\n",
    "### 處理後:\n",
    "![Processed image (scaled 4x)](https://chsasank.github.io/assets/images/traffic/output.png \"Processed image (scaled 4x)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 圖像大小的修改 (Image Resizing)\n",
    "\n",
    "由於圖像具有不同的形狀和大小，因此我們必須對圖像進行歸一化(normalize)處理以便可以決定模型的輸入。\n",
    "在這個範例我們選擇使用48x48的圖像大小。你也可以嘗試使用比較小的圖像比如32x32或24x24看看是不是也可得到類似的準確率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 用來對圖像進行處理的函式\n",
    "import os\n",
    "from pathlib import PurePath # 處理不同作業系統file path的解析問題 (*nix vs windows)\n",
    "\n",
    "# 圖像標高度均衡、置中及大小調整\n",
    "def preprocess_img(img):\n",
    "    # 進行\"直方圖均衡化\"處理\n",
    "    hsv = color.rgb2hsv(img) # 對彩色分量rgb分別做均衡化，會產生奇異的點，圖像不和諧。一般採用的是用hsv空間進行亮度的均衡\n",
    "    hsv[:,:,2] = exposure.equalize_hist(hsv[:,:,2])\n",
    "    img = color.hsv2rgb(hsv) # 再把圖像從hsv轉回rgb\n",
    "    \n",
    "    # 進行圖像置中\n",
    "    min_side = min(img.shape[:-1])\n",
    "    centre = img.shape[0]//2, img.shape[1]//2\n",
    "    img = img[centre[0]-min_side//2:centre[0]+min_side//2,\n",
    "              centre[1]-min_side//2:centre[1]+min_side//2,\n",
    "              :]\n",
    "    \n",
    "    # 改變大小\n",
    "    img = transform.resize(img, (IMG_SIZE, IMG_SIZE))    \n",
    "    return img\n",
    "\n",
    "# 取得圖像檔的分類標籤\n",
    "def get_class(img_path):\n",
    "    # 圖像檔所在的檔案夾名稱就是該圖像檔的分類標籤\n",
    "    return int(PurePath(img_path).parts[-2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 圖像處理並轉換成numpy ndarray\n",
    "\n",
    "讓我們預處理所有的訓練圖像並以numpy數組來存儲到檔案系統中。過程中我們還會從圖像檔的檔案路徑中獲取圖片的標籤。\n",
    "我們會將標籤(label)資料進行one-hot編碼："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded images from X.h5\n"
     ]
    }
   ],
   "source": [
    "# 從檔案系統中把保留的資料載入(如果沒有就進行第一次的資料轉換處理)\n",
    "try:\n",
    "    with h5py.File('X.h5') as hf:\n",
    "        X, Y = hf['imgs'][:], hf['labels'][:]\n",
    "    \n",
    "    print(\"Loaded images from X.h5\")\n",
    "    \n",
    "except(IOError, OSError, KeyError):\n",
    "    print(\"Error in reading X.h5. Processing all images...\")\n",
    "    root_dir = 'GTSRB/Final_Training/Images/'\n",
    "    imgs = []\n",
    "    labels = []\n",
    "    \n",
    "    all_img_paths = glob.glob(os.path.join(root_dir, '*/*.ppm')) # 我們有 Test與Traing兩個檔案夾的資料要處理\n",
    "    np.random.shuffle(all_img_paths) # 進行打散\n",
    "    for img_path in all_img_paths:\n",
    "        try:\n",
    "            img = preprocess_img(io.imread(img_path))\n",
    "            label = get_class(img_path)\n",
    "            imgs.append(img) # 保留圖像資料\n",
    "            labels.append(label) # 保留圖像標籤\n",
    "            \n",
    "            if len(imgs)%1000 == 0:\n",
    "                print(\"Processed {}/{}\".format(len(imgs), len(all_img_paths))) # 每1000筆秀一下進度\n",
    "        except(IOError, OSError):\n",
    "            print('missed', img_path)\n",
    "            pass\n",
    "        \n",
    "    X = np.array(imgs, dtype='float32') # 將資料轉換成numpy的ndarray, 資料型別為float32\n",
    "    Y = np.eye(NUM_CLASSES, dtype='uint8')[labels] # 對labels的資料進行one-hot (使用numpy.eye的函式)\n",
    "    \n",
    "    # 將處理過圖像資料與標籤保持在檔案系統, 下次可以加速載入與處理\n",
    "    with h5py.File('X.h5', 'w') as hf:\n",
    "        hf.create_dataset('imgs', data=X)\n",
    "        hf.create_dataset('labels', data=Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 網絡模型 (Model)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "現在我們來定義我們的模型架構。我們將使用具有6個卷積層的前饋網絡，然後是完全連接的隱藏層。\n",
    "我們也將在兩者之間使用Dropout層來防止網絡\"過擬合(overfitting)\"。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_1 (Conv2D)            (None, 48, 48, 32)        896       \n",
      "_________________________________________________________________\n",
      "conv2d_2 (Conv2D)            (None, 46, 46, 32)        9248      \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 23, 23, 32)        0         \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 23, 23, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_3 (Conv2D)            (None, 23, 23, 64)        18496     \n",
      "_________________________________________________________________\n",
      "conv2d_4 (Conv2D)            (None, 23, 23, 64)        36928     \n",
      "_________________________________________________________________\n",
      "max_pooling2d_2 (MaxPooling2 (None, 11, 11, 64)        0         \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 11, 11, 64)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_5 (Conv2D)            (None, 11, 11, 128)       73856     \n",
      "_________________________________________________________________\n",
      "conv2d_6 (Conv2D)            (None, 11, 11, 128)       147584    \n",
      "_________________________________________________________________\n",
      "max_pooling2d_3 (MaxPooling2 (None, 5, 5, 128)         0         \n",
      "_________________________________________________________________\n",
      "dropout_3 (Dropout)          (None, 5, 5, 128)         0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 3200)              0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 512)               1638912   \n",
      "_________________________________________________________________\n",
      "dropout_4 (Dropout)          (None, 512)               0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 43)                22059     \n",
      "=================================================================\n",
      "Total params: 1,947,979\n",
      "Trainable params: 1,947,979\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# 產生一個Keras序貫模型\n",
    "def cnn_model():\n",
    "    model = Sequential()\n",
    "    \n",
    "    model.add(Conv2D(32, (3, 3), padding='same', activation='relu', input_shape=(IMG_SIZE, IMG_SIZE, 3)))    \n",
    "    model.add(Conv2D(32, (3, 3), activation='relu'))\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "    model.add(Dropout(0.2))\n",
    "    \n",
    "    model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))\n",
    "    model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "    model.add(Dropout(0.2))\n",
    "    \n",
    "    model.add(Conv2D(128, (3, 3), padding='same', activation='relu'))\n",
    "    model.add(Conv2D(128, (3, 3), padding='same', activation='relu'))\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "    model.add(Dropout(0.2))\n",
    "    \n",
    "    model.add(Flatten())\n",
    "    model.add(Dense(512, activation='relu'))\n",
    "    model.add(Dropout(0.5))\n",
    "    model.add(Dense(NUM_CLASSES, activation='softmax'))\n",
    "    \n",
    "    return model;   \n",
    "\n",
    "model = cnn_model() # 初始化一個模型\n",
    "model.summary() # 秀出模型架構\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在訓練模型之前，我們需要將模型配置為學習算法並進行編譯。我們需要指定:\n",
    "* `loss`: 損失函數，我們要優化。我們不能使用`MSE`，因為它是不連續的數值。因此，我們使用：`categorical_crossentropy`\n",
    "* `optimizer`: 我們使用標準隨機梯度下降(Stochastic gradient descent)與涅斯捷羅夫動量(Nesterov momentum)\n",
    "* `metric`: 由於我們正在處理一個分類問題，我們用度量是`accuracy`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 讓我們先配置一個常用的組合來作為後續優化的基準點\n",
    "lr = 0.01\n",
    "sgd = SGD(lr=lr, decay=1e-6, momentum=0.9, nesterov=True)\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "             optimizer=sgd,\n",
    "             metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 訓練 (Training)\n",
    "\n",
    "現在，我們的模型已經準備好了。在訓練期間，我們的模型將進行迭代批量訓練，每個次的訓練資料的大小為batch_size。對於每批次，模型將會計算出梯度(gradient)，並自動更新網絡的權重。對所有訓練集的一次迭代被稱為一次的循環(epoch)。訓練通常會一直進行到損失收斂於一個常數。\n",
    "\n",
    "\n",
    "我們將增加一些功能到我們的訓練設定：\n",
    "* `Learning rate scheduler`: 隨著訓練循環的次數逐漸增加的過程中對權重調整的學習率進行衰減通常有助於讓模型學習更好\n",
    "* `Model checkpoint`: 我們將比對每個訓練循環的驗證準確度並只保存模型表現最好的模型。這對深度學度來說是很有用的設定，因為我們的網絡可能在一定數量的訓練循環後開始過擬合(overfitting)，但是我們需要在整過訓練過程中表現最好的模型留下來。\n",
    "\n",
    "這些設定不是必須的，但它們的確可以提高模型的準確性。這些功能是通過Keras的`callback`功能來實現的。`callback`是一組函式，將在訓練過程的特定階段被應用，比如將訓練結束。 Keras提供內置的學習速率調度([learning rate scheduling](https://keras.io/callbacks/#learningratescheduler) )和模型檢查點功能([model checkpointing](https://keras.io/callbacks/#modelcheckpoint))。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 31367 samples, validate on 7842 samples\n",
      "Epoch 1/30\n",
      "31367/31367 [==============================] - 11s 339us/step - loss: 1.7234 - acc: 0.5070 - val_loss: 0.2577 - val_acc: 0.9204\n",
      "Epoch 2/30\n",
      "31367/31367 [==============================] - 10s 326us/step - loss: 0.2787 - acc: 0.9119 - val_loss: 0.1255 - val_acc: 0.9661\n",
      "Epoch 3/30\n",
      "31367/31367 [==============================] - 11s 335us/step - loss: 0.1571 - acc: 0.9509 - val_loss: 0.0638 - val_acc: 0.9806\n",
      "Epoch 4/30\n",
      "31367/31367 [==============================] - 11s 340us/step - loss: 0.1060 - acc: 0.9671 - val_loss: 0.0602 - val_acc: 0.9800\n",
      "Epoch 5/30\n",
      "31367/31367 [==============================] - 11s 344us/step - loss: 0.0875 - acc: 0.9729 - val_loss: 0.0489 - val_acc: 0.9856\n",
      "Epoch 6/30\n",
      "31367/31367 [==============================] - 10s 325us/step - loss: 0.0686 - acc: 0.9786 - val_loss: 0.0429 - val_acc: 0.9869\n",
      "Epoch 7/30\n",
      "31367/31367 [==============================] - 10s 328us/step - loss: 0.0616 - acc: 0.9811 - val_loss: 0.0213 - val_acc: 0.9934\n",
      "Epoch 8/30\n",
      "31367/31367 [==============================] - 11s 335us/step - loss: 0.0487 - acc: 0.9843 - val_loss: 0.0319 - val_acc: 0.9913\n",
      "Epoch 9/30\n",
      "31367/31367 [==============================] - 11s 341us/step - loss: 0.0480 - acc: 0.9851 - val_loss: 0.0227 - val_acc: 0.9938\n",
      "Epoch 10/30\n",
      "31367/31367 [==============================] - 10s 327us/step - loss: 0.0418 - acc: 0.9871 - val_loss: 0.0222 - val_acc: 0.9930\n",
      "Epoch 11/30\n",
      "31367/31367 [==============================] - 11s 342us/step - loss: 0.0197 - acc: 0.9939 - val_loss: 0.0131 - val_acc: 0.9958\n",
      "Epoch 12/30\n",
      "31367/31367 [==============================] - 11s 346us/step - loss: 0.0133 - acc: 0.9957 - val_loss: 0.0114 - val_acc: 0.9963\n",
      "Epoch 13/30\n",
      "31367/31367 [==============================] - 11s 336us/step - loss: 0.0101 - acc: 0.9968 - val_loss: 0.0111 - val_acc: 0.9962\n",
      "Epoch 14/30\n",
      "31367/31367 [==============================] - 11s 349us/step - loss: 0.0100 - acc: 0.9969 - val_loss: 0.0115 - val_acc: 0.9966\n",
      "Epoch 15/30\n",
      "31367/31367 [==============================] - 11s 350us/step - loss: 0.0082 - acc: 0.9977 - val_loss: 0.0113 - val_acc: 0.9966\n",
      "Epoch 16/30\n",
      "31367/31367 [==============================] - 11s 339us/step - loss: 0.0077 - acc: 0.9974 - val_loss: 0.0109 - val_acc: 0.9964\n",
      "Epoch 17/30\n",
      "31367/31367 [==============================] - 10s 329us/step - loss: 0.0071 - acc: 0.9974 - val_loss: 0.0114 - val_acc: 0.9966\n",
      "Epoch 18/30\n",
      "31367/31367 [==============================] - 10s 333us/step - loss: 0.0062 - acc: 0.9977 - val_loss: 0.0116 - val_acc: 0.9964\n",
      "Epoch 19/30\n",
      "31367/31367 [==============================] - 10s 326us/step - loss: 0.0066 - acc: 0.9976 - val_loss: 0.0105 - val_acc: 0.9968\n",
      "Epoch 20/30\n",
      "31367/31367 [==============================] - 11s 346us/step - loss: 0.0056 - acc: 0.9981 - val_loss: 0.0105 - val_acc: 0.9964\n",
      "Epoch 21/30\n",
      "31367/31367 [==============================] - 10s 335us/step - loss: 0.0052 - acc: 0.9982 - val_loss: 0.0105 - val_acc: 0.9964\n",
      "Epoch 22/30\n",
      "31367/31367 [==============================] - 11s 349us/step - loss: 0.0058 - acc: 0.9979 - val_loss: 0.0105 - val_acc: 0.9963\n",
      "Epoch 23/30\n",
      "31367/31367 [==============================] - 10s 328us/step - loss: 0.0058 - acc: 0.9982 - val_loss: 0.0106 - val_acc: 0.9963\n",
      "Epoch 24/30\n",
      "31367/31367 [==============================] - 10s 318us/step - loss: 0.0054 - acc: 0.9983 - val_loss: 0.0105 - val_acc: 0.9966\n",
      "Epoch 25/30\n",
      "31367/31367 [==============================] - 10s 329us/step - loss: 0.0048 - acc: 0.9984 - val_loss: 0.0104 - val_acc: 0.9967\n",
      "Epoch 26/30\n",
      "31367/31367 [==============================] - 11s 358us/step - loss: 0.0055 - acc: 0.9981 - val_loss: 0.0105 - val_acc: 0.9967\n",
      "Epoch 27/30\n",
      "31367/31367 [==============================] - 10s 316us/step - loss: 0.0060 - acc: 0.9979 - val_loss: 0.0106 - val_acc: 0.9966\n",
      "Epoch 28/30\n",
      "31367/31367 [==============================] - 11s 337us/step - loss: 0.0050 - acc: 0.9984 - val_loss: 0.0105 - val_acc: 0.9967\n",
      "Epoch 29/30\n",
      "31367/31367 [==============================] - 10s 319us/step - loss: 0.0054 - acc: 0.9983 - val_loss: 0.0105 - val_acc: 0.9968\n",
      "Epoch 30/30\n",
      "31367/31367 [==============================] - 10s 330us/step - loss: 0.0043 - acc: 0.9986 - val_loss: 0.0104 - val_acc: 0.9967\n"
     ]
    }
   ],
   "source": [
    "def lr_schedule(epoch):\n",
    "    return lr*(0.1**int(epoch/10))\n",
    "\n",
    "batch_size = 32\n",
    "nb_epoch = 30\n",
    "\n",
    "history = model.fit(X, Y,\n",
    "         batch_size=batch_size,\n",
    "         epochs=nb_epoch,\n",
    "         validation_split=0.2,\n",
    "         shuffle=True,\n",
    "         callbacks=[LearningRateScheduler(lr_schedule),\n",
    "             ModelCheckpoint('model.h5', save_best_only=True)\n",
    "         ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAAEKCAYAAAAsOPKBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xl8lOXV//HPmZkkk5CwBhWCCLaW\nKhjZRPvgAmpVrCJVSrH1eaqt0urPurTaYmsRqVZafaz111pFi11+iqWogArihlJ3QDAsirhgSSK7\nIQlkklmu3x8z2WeSQDLMBL7v1ysvct/3dd9zQu1wcuZc12XOOUREREREpGN5Uh2AiIiIiMjBSIm2\niIiIiEgSKNEWEREREUkCJdoiIiIiIkmgRFtEREREJAmUaIuIiIiIJIESbRERERGRJFCiLSIiIiKS\nBEq0RURERESSwJfqADpSfn6+GzBgQKrDEBHZZytXrtzhnOud6jgOJL1ni0hn1db37IMq0R4wYAAr\nVqxIdRgiIvvMzD5LdQwHmt6zRaSzaut7tlpHRERERESSQIm2iIiIiEgSJK11xMxmA+cD25xzQ+Jc\nvwn4boM4jgV6O+d2mdkmoAIIAyHn3MhkxSkiIiIikgzJ7NH+K/BH4O/xLjrn7gLuAjCzC4AbnHO7\nGgwZ65zbkcT4RKQNgsEgxcXFBAKBVIdyUPD7/fTr14+MjIxUhyIiIkmWtETbObfMzAa0cfglwJxk\nxSIi+6+4uJi8vDwGDBiAmaU6nE7NOcfOnTspLi5m4MCBqQ6nzdrwCaUBfwDOA/YClznn3j2wUYqI\npJ+UrzpiZjnAucA1DU474Hkzc8CDzrlZLdw/BZgC0L9//za/7vxVJdy1ZAOlZVX07Z7NTecMYsKw\ngv36GUQOZoFAQEl2BzEzevXqxfbt21Mdyr76Ky18QgmMA46JfZ0E/Dn2p4i0oC25SEfnKx35mgc6\n/nSNqyUpT7SBC4DXm7SNjHbOlZrZYcALZvaBc25ZvJtjSfgsgJEjR7q2vOD8VSXc/OQaqoJhAErK\nqrj5yTUASrZF4lCS3XE6499lGz6hvBD4u3POAW+ZWXcz6+Oc+/yABCjJVzQXXpoBu4uhWz84cxoU\nTmo2bPnCBzny3bs4zG1nm/Vm8/CbOHH8D/d5zKHwrPmrSnjtqfv5J4/TN2sHpXvzufepycDVdblI\nW8ak6jUPdPypiqu90iHRnkyTthHnXGnsz21m9hQwCoibaO+Pu5ZsqEuya1UFw9y1ZIMSbRGRfVcA\nbG5wXBw7p0R7X7UxoT2gzyqaS2jBj/GFY/M0dm+OHkOj5y1f+CBDVt5CttWAwRFsp9vKW1gOdQlT\nW8a0Nm7kBVOoCUdY+fSDDFt9a7Mxr9aEOPacH+A1w+fxsG7JQ3HHve0iDD8/+prvPvMghe9Oazbm\n9XCY486+grBzRCKO95//C6PWTG82bml1kGPO+j5mxsYXZ3NSnDEvVtXQ99TvEYpECIYdyxc+wAyb\nRY7VANDPdjDDzeL2hR78GVcTcfD2gsRjcrOuwes1vGbsfvtRzvzojui42Gt2XXkLC3bvpduo7xJx\njnAE3knwmr9e4AF+hMPxzoIHE47ZW3MlEedwwLrnHk44zvFDPGYJ479joYfeeddhBh4zdrzxD87Y\n2Dz+J7/YS/bwyQQjLuGzauOKPgtWP/tQ3HEzFnjYuef7hCMRPn75kbhjfvesjwnDbtv3/4+0wKIF\niOSIVUCeidfTF7veDfgUONI5tyd2rgvgcc5VxL5/AZjhnHuutdcbOXKka8vmBwOnPku8n9qAT2d+\no9X7RQ4l77//Pscee2zKXr+srIzHHnuMq6++ep/uO++883jsscfo3r17kiLbf/H+Ts1sZTqvsNTS\n+7mZPQvc6Zx7LXb8EvAz59zKOGMbtvuN+OyzQ26fnsSK5sLT10Kwqv5cRjZccF/jBLktCXRbn9Xa\n85xj728HkRPY2izcmoyuZJ7+E8KhINWBKiJv/ZlcqpqNq3TZrDliAjVhx/DtC8izeGP8vOofi9eF\n8boQo2teJ8eqm43b6zJ5NTIUB4zxrK5LlJqOeTEyAocRwTjbs4IucZ+VxQuREXiIcKbn3XY9q8pl\n8npkMD4inOxZj9+CzcYEXAZvRo4jhJcwXk7zvBf3Nfe4LBaFo51X53nfjvt6e1wWC8L/BRgO40Lv\n6+Ra8wnrlc7Pv8Kn1x1/y/tq3HF7XBaLIyfhIcK5nuVx/+5rf0YDPET4Wgs/5zuRrwIwyvNB3DFV\nLoM3IkNwgMPDKZ410V9M4ox7PTIED46vedaRHff1fKx2x8T+Jhwn2MdkWajZuGrnY7X7MhHnYZjn\nQ/xxxhRH8uk34+Nm5+Np63t2Mpf3mwOMAfLNrBi4FcgAcM49EBv2TeD52iQ75nDgqdjHqz7gsbYk\n2fuib/dsSsqa/x+9b/fsjnwZkUNSR/e9lZWVcf/99zdLtMPhMF6vN+F9ixYt2u/XlH1WDBzZ4Lgf\nUBpv4P60+x0yXprRODGG6PGLt8KQi8HjbZ5A794cPYbomD07oKIUnrs5/rMW3QQ1leDzgy8LilfC\n8ochXF3/vKd+BK/dCy4MZZvJCe4hnsxgObw4HS+QAzhHtGLVRBeqOGHLk3jMkUXzZCo6JsApwTeJ\nmJeIecmmeaIHkE0NJ+buwMzI3hP/WdnUMCavFHDgHDl7Ez2rmtNzi3FA9t7Ezzo9rwQjmtLm7In/\nLD81DO9RjfN4yfqieTIIkEWQYfkRPC6Ix4XJ3h3/NXOsmgndo8merzL+6+VYNd/qug5cBJzDVxV/\nVaguBPjvnLeiBwbe6vjjcqyaC7t/AubBV57gZ7QaTu8TAvMAhm9rgp/Tgozqmwk4shKM8VuQk3sH\nAYdzDv+u+H8XfoKMPjyEmZG1PdHrhRjerzvOov8rZRZviDsu00KM6N8z+ovC5uZJNkBfz86459sj\nmauOXNKGMX8lOsmm4blPgBOSE1XUTecMatSjDZCd4eWmcwYl82VFDnrJmP8wdepUPv74Y4YOHUpG\nRga5ubn06dOH1atXs379eiZMmMDmzZsJBAJcd911TJkyBajf3ruyspJx48Zxyimn8MYbb1BQUMCC\nBQvIztYv1h1oIXCNmT1OdBLk7kOqP7utFeZ4YyJhKF0FH70UTXLjKS+FGb0gpxcEyiDSJEkIVkWT\n46d+FE2OWxIog2duaHmMC+N2fMiWI8awMe94Cncuors1T7ZLIz15ZPg8crtk06NLNmc9fxZ9ab4q\n7xbrTZ/pH0W/n/5ljqD5ZOCt1psjpn1Uf09L436+qvUxPytq47PWtOFZa9r2rJ+82fqY616rO977\n26+SU9X8/yZV2X3IuXF962N+9kHb4vpF/d9ri8/76brWx1zdxvivernVMV2ufb1t8V/zWuuvd2V9\nPbbFcT9Y3OKYQPYR5DQ72z7p0KN9wNX+g/+bRe+zraKaHjkZ3HrBYPVni7TitqfXsb60POH1Vf8p\noyYcaXSuKhjmZ/OKmPPOf+Lec1zfrtx6weCEz5w5cyZr165l9erVvPLKK3zjG99g7dq1dcvjzZ49\nm549e1JVVcWJJ57IxRdfTK9evRo9Y+PGjcyZM4eHHnqISZMm8cQTT3DppZe29cc+5LXhE8pFRJf2\n+4jo8n6XpybSFGipwlybbMcbs+BqeOvPsOtjCOwmWm7MgHCcqp2/O5z0Q9izHVbMjh+HC8OpN0Le\nEZDXB569ASq3NR/XtQCueAlCAQhV4+4/GYvTTOnCIb72yeV09fs42x3ODGY1anPY6zKZlfk/TJ8w\nou7c8h0/o0dtX3VMlcukeMRN9Ikdbx5+E93ijNk84iaOaPD6bRnX2Z+VM25G4953IOT1kzNuxj6N\nSdVrHuj4UxFXRzgkE22IJttf+1IvTvrNS/z0bC3tJ9IRmibZrZ3fH6NGjWq0BvV9993HU089BcDm\nzZvZuHFjs0R74MCBDB06FIARI0awadOmDovnUNDaJ5Sx1Ub+zwEKp2N01ETBRO0ei26E7R9AdSWs\n+kfzMeEgfP4eDP0OfOkMOHoMfPRi3H/8fefdVRfb3nWL41bi9mb3IefMX9UdL99YzJCVv2qUuOx1\nmSzs9n3KVu1l154adlRW81PXiwKLU4WmF/N+9DWGHtmdZ4qGMO0puN49Tl/bSanrxb1M5pRvTGl0\nz4njf8hyiK0csYNtls/mEY1XjmjLmEPlWRROiiZhDf479DX977AtY1L0mgc8/hTE1RGSOhnyQGvr\nZMhae2tCHDdtCT8/96tcNeZLSYxMpPPal8mQo2e+HHf+Q0H3bF6fesZ+vf6mTZs4//zzWbt2La+8\n8gp33303zzzzDACvvPIKt9xyC88//zw5OTmMGTOG6dOnM2bMmEatI7X3A9x9991UVlYyffr0/Yqn\nI3TGyZDJsK/v2R2mvZMOqyuh+B34z1vw6m8Tv455ISsv2q4RfwBMr79Wu+TY9TROaEdPuIr/+nI+\nn+3cy1N/+z2/cg80qy7/Inwlb2SfQXUoQnUoTCAYYbznNX7mm1v3rN+FJrEwcgoAWT4P+blZjCh/\ngZkZDzd73s3BK/jDb+5sFJv2nhCpl/LJkJ1BdoYXr8eorI7fYC8i+yYZ8x/y8vKoqKiIe2337t30\n6NGDnJwcPvjgA9566639fh05hCSqQj//KygYEZ0kuGEJPP9LCDVo95h/Fbz0aygvibZqmAc8GRCJ\n829I135ww1owS9gPujfWDxoKR9heWc0dz77P9pr/Yh7/1WjcvLnvNTg6mT2eUJwEejSXHHsYWT4v\nWT4PDy77hIWRU1hYc0qjZxmw9rZzyMn0YmaMnglTy2n2vJVdv97ovgnDCpRYi+yHQzrRNjNys3xU\nBOLPPhWRfVP7D3FHVr569erF6NGjGTJkCNnZ2Rx++OF1184991weeOABCgsLGTRoECeffHK7fwY5\nBOwujn++cgv83+GJ74uEoHIrnPoT6P816HcifPhc/Or4WbdCbHOi3wW/zc/c/c2qxr+suIjX73iR\nHZXVRFr5cHnGhYPp3zOHnz9RxMLy5gl0Qfds7ryosO74maLPE66u1SWr/p/+6C/HNY2el53h5U4t\nDiDSIQ7pRBsgz69EW6QjJaPy9dhjj8U9n5WVxeLFi+Neq+3Dzs/Pr2sbAbjxxhs7NDbpRCLhxJMJ\nIbqqxzm/gVB1/YTGpsI1cMYt9ceFk1i+6YvGvaXH38SXvjSBd9ZuYfmmXfy1chS7PDVxq9CTBvXm\niK5+Du/m53+f/5BdcZasK+iezf98bQAAN487tk2fGrX106Vk/HIsIvWUaPszqAiodURE5KBW8m50\nWbvPV0PvY+GLT6Mrb9TKyIZzZ9b3aC+7K/5ye936NTqcv6qEm5cfRVXwD3Xn7A1wb7wAQKbPQ6bX\nw8Jw/Cr07ybWr2bbJdPXanLc1sR4XxJotYWIJI8SbbWOiIgcPJpOYDz1Rti6Bpb/BXIPh4mzYfBF\nsOZfLa86cua0+CuAnDkN5xybd1Xx1qc7mb5wXaPEGMABXf0+Zl92Isf368biNVs6tLrc1sRYCbRI\n6inR9vv4fHf8nZJERKQTibde9TPXRb8/6Ucw9hfg7xY9LpzU4lJe88OjeS14RaMVQP439G0+e62A\n0mdeZkt5y/9uVARCjBzQE1B1WeRQpkTb7+PDbWodERHp9OKtJgLRSva4Fpbhi+OuJRsoibMCiGdz\nGecd34eTBvbkpKN7cdkj71Ba1jzp7tu98c6jSqBFDk2HfKKd6/dRqdYREZHOL+FqInF2SGxFaZwV\nOwCcgz9+p35lkp+d89UOX9JSRA4eh3yiHZ0MGcI5h8WWYhIRkU6oW782TWBsiXOO//fWZwmvx6tU\ng1btEJH4PKkOINXy/D5CEUcg2HFbRItI6uTm5gJQWlrKxIkT444ZM2YMre1IeO+997J379664/PO\nO4+yskQ7/ElaOHNa3drVdTKyo+fboLSsiv+Z/Q6/WrCOrxyRh9/X+J/IRJXqCcMKeH3qGXw68xu8\nPvUMJdkiUkcVbX8GABXVQbIzvSmORuQgkGjb6gOsb9++zJs3b7/vv/fee7n00kvJyckBYNGiRR0V\nmiRLZpdob0d2D6gqa/G/v8ZbivsZM6g3C9/7nFDYcfuEIXz3pP4sWF2qSrWItIsS7dgOWRWBEIfl\npTgYkc4u3qoPtRt/7Gey/fOf/5yjjjqKq6++GoDp06djZixbtowvvviCYDDI7bffzoUXXtjovk2b\nNnH++eezdu1aqqqquPzyy1m/fj3HHnssVVX1/bdXXXUVy5cvp6qqiokTJ3Lbbbdx3333UVpaytix\nY8nPz2fp0qUMGDCAFStWkJ+fzz333MPs2dGNT6644gquv/56Nm3axLhx4zjllFN44403KCgoYMGC\nBWRnN241kCSJRODlO6DXl+Hqt8Gb+J+3+atKGvVVl5QFePTtzQzslcNfvz+Ko3p1ATSBUUTaT4m2\nvz7RFpFWLJ4KW9Ykvl68HMLVjc8Fq2DBNbDyb/HvOeJ4GDcz4SMnT57M9ddfX5doz507l+eee44b\nbriBrl27smPHDk4++WTGjx+fcJ7Fn//8Z3JycigqKqKoqIjhw+sns91xxx307NmTcDjMmWeeSVFR\nEddeey333HMPS5cuJT8/v9GzVq5cySOPPMLbb7+Nc46TTjqJ008/nR49erBx40bmzJnDQw89xKRJ\nk3jiiSe49NJLE/99ScdZ9yRsWwcX/6XFJBui/dRN174GqA5H6pJsEZGOoB7t2tYR7Q4p0n5Nk+zW\nzrfBsGHD2LZtG6Wlpbz33nv06NGDPn368Itf/ILCwkLOOussSkpK2Lp1a8JnLFu2rC7hLSwspLCw\nsO7a3LlzGT58OMOGDWPdunWsX7++xXhee+01vvnNb9KlSxdyc3O56KKL+Pe//w3AwIEDGTp0KAAj\nRoyo2wZekiwcgqW/gcOHRDejaUWiFUU+j7NMn4hIexzyFe3cWOuIlvgTaYMWKs8A/H5IglUfjoTL\nn93vl504cSLz5s1jy5YtTJ48mUcffZTt27ezcuVKMjIyGDBgAIFAy0lSvGr3p59+yt13383y5cvp\n0aMHl112WavPcc4lvJaVlVX3vdfrbdSiIkn03mOw62OYPAc8rdeP+nT3t2ntaxGR9lJFW60jIh3n\nzGnRVR4a2odVHxKZPHkyjz/+OPPmzWPixIns3r2bww47jIyMDJYuXcpnnyVejg3gtNNO49FHHwVg\n7dq1FBUVAVBeXk6XLl3o1q0bW7duZfHixXX35OXlUVFREfdZ8+fPZ+/evezZs4ennnqKU089tV0/\nn7RDqBpe+S0UjIBB49p0y+gv5zc7p7WvRSQZkpZom9lsM9tmZmsTXB9jZrvNbHXsa1qDa+ea2QYz\n+8jMpiYrRoCusdaRcrWOiLRf4SS44L5oBRuL/nnBfe1edWTw4MFUVFRQUFBAnz59+O53v8uKFSsY\nOXIkjz76KF/96ldbvP+qq66isrKSwsJCfve73zFq1CgATjjhBIYNG8bgwYP5/ve/z+jRo+vumTJl\nCuPGjWPs2LGNnjV8+HAuu+wyRo0axUknncQVV1zBsGHD2vXzSTuseATKi+GMXzVf2i+OHZXVPL9u\nK1/q3YWC7n4MKOiezZ0XHa+JjyLS4aylj0Hb9WCz04BK4O/OuSFxro8BbnTOnd/kvBf4EPg6UAws\nBy5xzrXcOAmMHDnStbY2blPhiONLv1jE9Wcdw/VnfWWf7hU5FLz//vsce+yxqQ7joBLv79TMVjrn\nRqYopJTYn/fsRmr2wB9OgN5fhe893aZE+8Z/vcf8VSUsvu5UjjlcS02JyP5p63t20irazrllwK79\nuHUU8JFz7hPnXA3wOHBhK/fsN6/HyMn0qnVERKSzeftB2LO9zdXs5Zt2MW9lMVecerSSbBE5IFLd\no/01M3vPzBab2eDYuQKg4Wyq4ti5pMnz+7TqiIhIZ1JVBq//AY45G/qf1OrwYDjCLU+tpW83P9ee\n+eUDEKCISGpXHXkXOMo5V2lm5wHzgWOAeGWJhP0tZjYFmALQv3///Qokz59BZbUq2iKJOOcSrlEt\n+yZZ7XqHnDf/BIEyOOOWNg3/2xub2LC1ggf/ewQ5mYf8glsicoCkrKLtnCt3zlXGvl8EZJhZPtEK\n9pENhvYDSlt4zizn3Ejn3MjevXvvVyy5WT61jogk4Pf72blzpxLEDuCcY+fOnfj9/lSH0rnt2QFv\n3Q/HTYA+J7Q6fMvuAL9/4UPGDurN2ccdfgACFBGJStmv9WZ2BLDVOefMbBTRpH8nUAYcY2YDgRJg\nMvCdZMaS5/dRrkRbJK5+/fpRXFzM9u3bUx3KQcHv99OvX79Uh9G5vfZ7CO6Fsb9s0/BfP7ueUMRx\n2/gh+mRGRA6opCXaZjYHGAPkm1kxcCuQAeCcewCYCFxlZiGgCpjsoiWzkJldAywBvMBs59y6ZMUJ\n0SX+ShLsFCZyqMvIyGDgwIGpDkMEiubCC7dCRSlk5MDnq6F3y6tFLftwO88Wfc5Pvv4V+vfKOUCB\niohEJS3Rds5d0sr1PwJ/THBtEbAoGXHFk5vl086QIiLprGguPH0tBGNFkeDe6DEkXKe9OhTm1oXr\nGNArhymnHX2AAhURqZfqVUfSQnTVESXaIiJp66UZ9Ul2rWBV9HwCs179hE937GHGhUPwZ3iTHKCI\nSHOaek101ZGqYJhgOEKGV797iIiknd3FbTo/f1UJdy3ZQGlZFQ4Y2q8bp31l/ybKi4i0l7JKohVt\ngD1a4k9EJD11SzCBtMH5+atKuPnJNZTEkmyAD7ZUMH9VSfLjExGJQ4k2kBtLtNU+IiKSps6cBhnZ\njc9lZEfPx9y1ZANVwXCjIYFQhLuWbDgQEYqINKNEG+gaS7TLtTukiEh6KpwEF9wH3Y4ELPrnBfc1\nmghZmmD1qETnRUSSTT3aRHu0Aa08IiKSgJmdC/yB6LKrDzvnZja5fhQwG+gN7AIudc4laKzeT4WT\nEq4wAtC3e3bcpVr7ds+OM1pEJPlU0Sa6vB+odUREJB4z8wJ/AsYBxwGXmNlxTYbdDfzdOVcIzADu\nPLBRwk3nDCK7yeoi2Rlebjpn0IEORUQEUKIN1E+GrKhW64iISByjgI+cc58452qAx4ELm4w5Dngp\n9v3SONeTbsKwAu686HgyY6tHFXTP5s6LjmfCsIIDHYqICKBEG6hvHVFFW0QkrgJgc4Pj4ti5ht4D\nLo59/00gz8x6HYDYGpkwrICB+V04+7jDeX3qGUqyRSSllGjToKKtRFtEJB6Lc841Ob4RON3MVgGn\nAyVAszdVM5tiZivMbMX27ds7PlKgIhCsK6CIiKSSEm0gy+chw2tKtEVE4isGjmxw3A8obTjAOVfq\nnLvIOTcM+GXs3O6mD3LOzXLOjXTOjezdOzkbyVQEQnUFFBGRVFKiDZgZef4MKrS8n4hIPMuBY8xs\noJllApOBhQ0HmFm+mdX+m3Iz0RVIDrhIxFFZE6pbtlVEJJWUaMfk+X1UamdIEZFmnHMh4BpgCfA+\nMNc5t87MZpjZ+NiwMcAGM/sQOBy4IxWxVtaEcA61johIWtCv/DG5WT61joiIJOCcWwQsanJuWoPv\n5wHzDnRcTdW+j6t1RETSgSraMXl+n1pHREQ6udr3cVW0RSQdKNGOifZoq6ItItKZqaItIulEiXZM\nnlpHREQ6vfqKthJtEUk9Jdoxah0REen86ivaah0RkdRToh2T58+gsjqEc033YBARkc6iPJZoa3k/\nEUkHSrRj8vw+Ig721oRTHYqIiOwnTYYUkXSStETbzGab2TYzW5vg+nfNrCj29YaZndDg2iYzW2Nm\nq81sRbJibChX27CLiHR6FYEQPo/hz1AdSURSL5nvRH8Fzm3h+qfA6c65QuDXwKwm18c654Y650Ym\nKb5Gaqsf6tMWEem8KgJB8vw+zCzVoYiIJG/DGufcMjMb0ML1NxocvgX0S1YsbVE7Q71Cu0OKiHRa\nFYGQ2kZEJG2ky2drPwAWNzh2wPNmttLMphyIAPKy1DoiItLZRRNtTYQUkfSQ8ncjMxtLNNE+pcHp\n0c65UjM7DHjBzD5wzi1LcP8UYApA//799zsOtY6IiHR+ta0jIiLpIKUVbTMrBB4GLnTO7aw975wr\njf25DXgKGJXoGc65Wc65kc65kb17997vWPI0GVJEpNNT64iIpJOUJdpm1h94Evhv59yHDc53MbO8\n2u+Bs4G4K5d0pNpEu1KJtohIp6XWERFJJ0l7NzKzOcAYIN/MioFbgQwA59wDwDSgF3B/bHZ4KLbC\nyOHAU7FzPuAx59xzyYqzVpdMH2ZqHRER6czKA0G6qqItImkimauOXNLK9SuAK+Kc/wQ4ofkdyeXx\nGLmZvrpdxUREpHOJRByV1apoi0j6SJdVR9JCnt9HpZb3ExHplPbUhHAOJdoikjaUaDeQ6/epdURE\npJOqncyuyZAiki6UaDeQ58/QqiMiIp1UfaKtiraIpAcl2g2odUREpPOq/URSFW0RSRdKtBtQRVtE\npPNSRVtE0o0S7QZys9SjLSLSWZXH3r+7KtEWkTShRLuBrn4t7yci0llpMqSIpBsl2g3k+X3UhCJU\nh8KpDkVERPaRWkdEJN0o0W4gN0vbsIuIdFYVgSBej5Gd4U11KCIigBLtRmo/btSESBGRzqciEN0V\n0sxSHYqICKBEu5Hajxu1xJ+ISOdTEQiqbURE0ooS7QZyY2/Q5Vp5RESk06kIhMjL0kRIEUkfSrQb\n6KrWERGRTqu2dUREJF0o0W6grnVEibaISKdTHghqaT8RSStKtBuonwyp1hERkc6mIhDSZjUiklaU\naDdQu7yfWkdERDofTYYUkXSjRLuBTJ+HLJ+HCq06IiLSiJmda2YbzOwjM5sa53p/M1tqZqvMrMjM\nzjuQ8TnnqKwOqXVERNKKEu0m8vwZqmiLiDRgZl7gT8A44DjgEjM7rsmwW4C5zrlhwGTg/gMZ456a\nMBGnXSFFJL0o0W4iz+9Tj7aISGOjgI+cc58452qAx4ELm4xxQNfY992A0gMYX937tiraIpJO9Kt/\nE9FEWxVtEZEGCoDNDY6LgZOajJkOPG9mPwa6AGcdmNCiat+3VdEWkXSS1Iq2mc02s21mtjbBdTOz\n+2I9f0VmNrzBte+Z2cbY1/fVgs6uAAAgAElEQVSSGWdDeX6fdoYUEWks3p7mrsnxJcBfnXP9gPOA\nf5hZs39jzGyKma0wsxXbt2/vsADrK9pKtEUkfSS7deSvwLktXB8HHBP7mgL8GcDMegK3Eq2YjAJu\nNbMeSY00Ji8rQ60jIiKNFQNHNjjuR/PWkB8AcwGcc28CfiC/6YOcc7OccyOdcyN79+7dYQGW11W0\n1ToiIukjqYm2c24ZsKuFIRcCf3dRbwHdzawPcA7wgnNul3PuC+AFWk7YO0yuWkdERJpaDhxjZgPN\nLJPoZMeFTcb8BzgTwMyOJZpod1zJuhW179taR1tE0kmqJ0PG6/sraOF8Mx39MWSe36edIUVEGnDO\nhYBrgCXA+0RXF1lnZjPMbHxs2E+BK83sPWAOcJlzrml7SdJoMqSIpKNU/+qfqO+vLf2A0ZPOzQJm\nAYwcObLdb+p5/gwqa0JEIg6PJ14YIiKHHufcImBRk3PTGny/Hhh9oOOqpcmQIpKOUl3RTtT315Z+\nwKTIy/LhHFTWqKotItJZVASCeD1GTqY31aGIiNRJdaK9EPif2OojJwO7nXOfE/148mwz6xGbBHl2\n7FzS1VZD1KctItJ5VARC5Gb5MNMnkSKSPpL6GZuZzQHGAPlmVkx0JZEMAOfcA0Q/hjwP+AjYC1we\nu7bLzH5NdAIOwAznXEuTKjtMbX+f+rRFRDqPikBIbSMiknaS+q7knLuklesO+D8Jrs0GZicjrpbk\n1lW0tcSfiEhnUREIaiKkiKSdVLeOpB21joiIdD7lqmiLSBpSot1E7RqsFdodUkSk06gIhLSGtoik\nHSXaTdR+9KjWERGRzqO8Sq0jIpJ+2pRom9l1ZtY1tjrIX8zsXTM7O9nBpUJullpHREQ6m2iPtira\nIpJe2lrR/r5zrpzoMnu9ia4OMjNpUaVQTqYXr8e06oiIHJTM7Jtm1q3BcXczm5DKmNrLOUdltXq0\nRST9tDXRrl2Y9DzgEefce8TfvbHTMzNys3xqHRGRg9WtzrndtQfOuTKiS692WntqwkSctl8XkfTT\n1kR7pZk9TzTRXmJmeUAkeWGlVjTRVkVbRA5K8d73O3UpuLYwooq2iKSbtr4r/QAYCnzinNtrZj2J\nbS5zMMrz+yhXoi0iB6cVZnYP8CfAAT8GVqY2pPapLYyooi0i6aatFe2vARucc2VmdilwC7C7lXs6\nra7+DCqr1ToiIgelHwM1wD+BuUAVCTYO6yxU0RaRdNXWd6U/AyeY2QnAz4C/AH8HTk9WYKmU5/ex\npTyQ6jBERDqcc24PMDXVcXSk2k8gtY62iKSbtla0Q7Ht0i8E/uCc+wOQl7ywUivXrx5tETk4mdkL\nZta9wXEPM1uSypjaS60jIpKu2vrrf4WZ3Qz8N3CqmXmBg/YdLc/vo1I7Q4rIwSk/ttIIAM65L8zs\nsFQG1F5qHRGRdNXWiva3gWqi62lvAQqAu5IWVYrl+TOoCASJFvFFRA4qETPrX3tgZgOITorstFTR\nFpF01aZf/51zW8zsUeBEMzsfeMc59/fkhpY6uVk+gmFHdSiCP8Ob6nBERDrSL4HXzOzV2PFpwJQU\nxtNuFYEgHoMumXq/FpH00tYt2CcB7wDfAiYBb5vZxGQGlkq1E2rUpy0iBxvn3HPASGAD0ZVHfkp0\n5ZFOqyIQIjfLh9lBuY+aiHRibW1o+yVwonNuG4CZ9QZeBOYlK7BUqv34sSIQpHdeVoqjERHpOGZ2\nBXAd0A9YDZwMvAmckcq42qMiEFLbiIikpbb2aHtqk+yYnftwb6eTm6WKtogctK4DTgQ+c86NBYYB\n21MbUvtUBIKaCCkiaamt70zPxZZ/mhM7/jawKDkhpV6eWkdE5OAVcM4FzAwzy3LOfWBmg1IdVHuU\nB0J0VUVbRNJQWydD3mRmFwOjAQNmOeeeSmpkKVT7EaR2hxSRg1BxbB3t+cALZvYFUJrimNqlIhCi\nbzd/qsMQEWmmzZ+1OeeeAJ5IYixpo7aiXa6KtogcZJxz34x9O93MlgLdgOdSGFK7VQSC5B2em+ow\nRESaaTHRNrMK4q+vaoBzznVt5f5zgT8AXuBh59zMJtd/D4yNHeYAhznnuseuhYE1sWv/cc6Nb+Vn\n6TBqHRGRQ4Fz7tXWR6U/TYYUkXTVYqLtnNvvbdZju0f+Cfg6UAwsN7OFzrn1DZ5/Q4PxPyY6KadW\nlXNu6P6+fnvUToasVKItIpLWnHNUVoc0GVJE0lIyVw4ZBXzknPvEOVcDPA5c2ML4S6ifbJlSPq+H\nnExv3ba+IiKSnvbWhAlHnCraIpKWkploFwCbGxwXx841Y2ZHAQOBlxuc9pvZCjN7y8wmJHoRM5sS\nG7di+/aOW6EqN8un1hERkTRXv/26Ktoikn6SmWjH26IrXr83wGRgnnMu3OBcf+fcSOA7wL1m9qV4\nNzrnZjnnRjrnRvbu3bt9ETeQ5/dRWa1EW0QkndV+8qhEW0TSUTIT7WLgyAbH/Ui8hNRkmrSNOOdK\nY39+ArxC4/7tpMvzZ1Cu1hERkbRWuzqU1tEWkXSUzER7OXCMmQ00s0yiyfTCpoNiGyX0ILoFcO25\nHmaWFfs+n+j63eub3ptMeX61joiIpDtVtEUknSXtnck5FzKza4AlRJf3m+2cW2dmM4AVzrnapPsS\n4HHnXMO2kmOBB80sQvSXgZkNVys5EPL8PkrLqg7kS4qIyD6q79FWRVtE0k9SSwDOuUU02ardOTet\nyfH0OPe9ARyfzNhak5eVoR5tEZGY9uyLkEyaDCki6UzvTAmodUREJKoD9kVIGrWOiEg6S2aPdqeW\n6/extyZMKBxJdSgiIqmWtvsiVARCmEGXTCXaIpJ+lGgnUNvvt6c63MpIEZGDXnv3RWh4vUP3PqgI\nBMnN8uHxxFtRVkQktZRoJ1D7MaSW+BMRafe+CPU3dfDeBxWBkJb2E5G0pUQ7gbysaKKtPm0Rkfbt\ni5BM5YGQ+rNFJG0p0U6gtnVEK4+IiOz/vgjJVhEIqqItImlLiXYCtRWSCrWOiMghzjkXAmr3RXgf\nmFu7L4KZjW8wNN6+CElVoYq2iKQxvTslUJ9oq6ItIrK/+yIkW0V1kK/4cw/0y4qItIkq2gnkqqIt\nIpL2ohVttY6ISHpSop1Abc9fhXq0RUTSknNOrSMiktaUaCeQ5fOQ4TW1joiIpKmqYJhwxKmiLSJp\nS4l2AmZGbpZPrSMiImmqthCiiraIpCsl2i3I82dQqYq2iEhaqi2EKNEWkXSlRLsFeX6fWkdERNJU\neez9Wetoi0i6UqLdAiXaIiLpS60jIpLulGi3IDcrg3L1aIuIpKX61hFVtEUkPSnRbkFXv09bsIuI\npClVtEUk3SnRboFaR0RE0pcmQ4pIulOi3YLcWEXbOZfqUEREpImKQAgz6JKpRFtE0lNSE20zO9fM\nNpjZR2Y2Nc71y8xsu5mtjn1d0eDa98xsY+zre8mMM5E8fwbhiKMqGE7Fy4uISAsqAiFys3x4PJbq\nUERE4kpaGcDMvMCfgK8DxcByM1vonFvfZOg/nXPXNLm3J3ArMBJwwMrYvV8kK954aj+OrAiEyFHF\nREQkrZQHglraT0TSWjIr2qOAj5xznzjnaoDHgQvbeO85wAvOuV2x5PoF4NwkxZlQ7Ux27Q4pIpJ+\nKgIh9WeLSFpLZqJdAGxucFwcO9fUxWZWZGbzzOzIfbw3qfKy6ivaIiKSXioCQSXaIpLWkplox2ua\nazqr8GlggHOuEHgR+Ns+3BsdaDbFzFaY2Yrt27fvd7DxNGwdERGR9BKtaKt1RETSVzIT7WLgyAbH\n/YDShgOcczudc9Wxw4eAEW29t8EzZjnnRjrnRvbu3btDAq9V3zqiRFtEJN2odURE0l0yE+3lwDFm\nNtDMMoHJwMKGA8ysT4PD8cD7se+XAGebWQ8z6wGcHTt3QOXWVbTVoy0ikm7UOiIi6S5p71DOuZCZ\nXUM0QfYCs51z68xsBrDCObcQuNbMxgMhYBdwWezeXWb2a6LJOsAM59yuZMWaSO0buHaHFBFJL845\ntY6ISNpLainAObcIWNTk3LQG398M3Jzg3tnA7GTG15rc2JJ+5WodERFJK4FghFDEqaItImlNO0O2\nwOMxcrN8ah0REUkz9duvq6ItIulLiXYr8vw+KlXRFhFJK7WfNHZVRVtE0pgS7Vbk+X1adUREJM3U\nV7SVaItI+lKi3Yo8fwYV1WodERFJJ7UFELWOiEg6U6LditwstY6IiKSb+kRbFW0RSV9KtFuh1hER\nkfSjyZAi0hko0W5Fnj9Dy/uJiKQZVbRFpDNQot2KaEVbPdoiIumkIhDErH6/AxGRdKREuxV5WT6q\nQxFqQpFUhyIiIjHlgRC5mT48Hkt1KCIiCSnRboW2YRcRST/R7ddVzRaR9KZEuxW1E23UPiIikj4q\nAkFNhBSRtKdEuxW5sYqJVh4RkUOZmZ1rZhvM7CMzm5pgzCQzW29m68zssWTGo4q2iHQGepdqRZ4S\nbRE5xJmZF/gT8HWgGFhuZgudc+sbjDkGuBkY7Zz7wswOS2ZMFdVBeudmJfMlRETaTRXtVnRV64iI\nyCjgI+fcJ865GuBx4MImY64E/uSc+wLAObctmQFFK9pqHRGR9KZEuxW5WZoMKSKHvAJgc4Pj4ti5\nhr4CfMXMXjezt8zs3GQGpNYREekM9C7VCrWOiIgQbw091+TYBxwDjAH6Af82syHOubJGDzKbAkwB\n6N+//34F45zTZEgR6RRU0W5F/WRItY6IyCGrGDiywXE/oDTOmAXOuaBz7lNgA9HEuxHn3Czn3Ejn\n3MjevXvvVzDVoQjBsFNFW0TSnhLtVixeswWAu5//kNEzX2b+qpIURyQicsAtB44xs4FmlglMBhY2\nGTMfGAtgZvlEW0k+SUYw5bHCR1cl2iKS5pRot2D+qhJufnJN3XFJWRU3P7lGybaIHFKccyHgGmAJ\n8D4w1zm3zsxmmNn42LAlwE4zWw8sBW5yzu1MRjy1rXxqHRGRdKdyQAvuWrKBqmC40bmqYJi7lmxg\nwrCm84BERA5ezrlFwKIm56Y1+N4BP4l9JVV9oq1/wkQkvSW1ot3aBgdm9pPY5gZFZvaSmR3V4FrY\nzFbHvpp+RNl+RXPh90Ngevfon0Vzmw0pLauKe2ui8yIikny1c2ZU0RaRdJe0ckBbNjgAVgEjnXN7\nzewq4HfAt2PXqpxzQ5MSXNFcePpaCMYS5t2bo8cAhZPqhvXtnk1JnKS6b/fspIQlIiKtU0VbRDqL\nZFa0W93gwDm31Dm3N3b4FtGZ7Mn30oz6JLtWsCp6voGbzhlEdoa30blMr3HTOYOSHaGIiCRQX9FW\noi0i6S2ZiXZbNjho6AfA4gbHfjNbEdv4YEKHRra7uE3nJwwr4M6LjqegezYGZHiNDK+HU47J79Bw\nRESk7TQZUkQ6i2SWA9qywUF0oNmlwEjg9Aan+zvnSs3saOBlM1vjnPs4zr37vvlBt37RdpF455uY\nMKygbuLjh1srOP//vsYvnlzDg/89ArN4P6KIiCRTeSzRrt25V0TqBYNBiouLCQQCqQ7loOD3++nX\nrx8ZGfv3i30y36XassEBZnYW8EvgdOdcde1551xp7M9PzOwVYBjQLNF2zs0CZgGMHDkybiLfzJnT\nGvdo1zrlhhZv+8rhedx49lf4zaIPePLdEi4ecWA6XUREpF5FIEhulg+vR8UOkaaKi4vJy8tjwIAB\nKgi2k3OOnTt3UlxczMCBA/frGclsHWl1gwMzGwY8CIx3zm1rcL6HmWXFvs8HRgMNJ1G2T+EkuOA+\n6HYkYJB7BOCBze+0eusPTjmaEwf0YPrT67T6iIhIClQEQurPFkkgEAjQq1cvJdkdwMzo1atXuz4d\nSFqi3cYNDu4CcoF/NVnG71hghZm9R3Tjg5lNVitpv8JJcMNamF4GN26A034KRY/Dx0tbvM3rMe7+\n1gmEI46fP1FEdOlYERE5UCoCQSXaIi1Qkt1x2vt3mdR1tJ1zi5xzX3HOfck5d0fs3DTn3MLY92c5\n5w53zg2NfY2PnX/DOXe8c+6E2J9/SWacAJx6I/Q8Gp65oXlLSRNH9erCL847ln9v3MH/e/s/SQ9N\nRETqRSvamggpko7Kysq4//779/m+8847j7KysiRElFragr1Whh/O/z188Sksu7vV4d89qT+nHpPP\nb559n0079hyAAEVEBNQ6ItKR5q8qYfTMlxk49VlGz3yZ+atK2vW8RIl2OByOM7reokWL6N69e7te\nOx0p0W7o6DFwwiXw+r2w7f0Wh5oZv5tYiM9r3Piv9whH1EIiInIgRFtHVNEWaa/5q0q4+ck1lJRV\n4YCSsipufnJNu5LtqVOn8vHHHzN06FBOPPFExo4dy3e+8x2OP/54ACZMmMCIESMYPHgws2bNqrtv\nwIAB7Nixg02bNnHsscdy5ZVXMnjwYM4++2yqqjrvnDiVBJo6+w74cAk8fR1c/hx4Ev8u0qdbNjMu\nHMwN/3yPYTOepyIQom/3bG46Z1DdkoAiItKxVNEWaZvbnl7H+tLyhNdX/aeMmnCk0bmqYJifzSti\nzjvxW2OP69uVWy8YnPCZM2fOZO3ataxevZpXXnmFb3zjG6xdu7Zu1Y7Zs2fTs2dPqqqqOPHEE7n4\n4ovp1atXo2ds3LiROXPm8NBDDzFp0iSeeOIJLr300rb+2GlFFe2muvSCc+6AzW/Du39tfbwDj0XX\nde2o3wZFRCQxJdoiHaNpkt3a+f0xatSoRkvj3XfffZxwwgmcfPLJbN68mY0bNza7Z+DAgQwdOhSA\nESNGsGnTpg6L50DTO1U8J1wCqx+DF6bDoG9A3uEJh979/Ic07RqpCoa5a8kGVbVFRDpYIBimJhyh\nq1pHRFrVUuUZYPTMlymJs1RxQfds/vnDr3VIDF26dKn7/pVXXuHFF1/kzTffJCcnhzFjxsRdOi8r\nK6vue6/X26lbR1TRjscMzr8XQgF4bmqLQxOtpa01tkVEOl799uuqE4m0103nDCI7w9voXHaGl5vO\nGbTfz8zLy6OioiLutd27d9OjRw9ycnL44IMPeOutt/b7dToLvVMlkv9lOO1GWHoHfLoM9u6MbtF+\n5rToGtwxfbtnx/1tMMvnoaSsioLu2QcyahGRg1pFIAgo0RbpCLWfvN+1ZAOlZVUdMs+sV69ejB49\nmiFDhpCdnc3hh9d3BZx77rk88MADFBYWMmjQIE4++eR2/wzpTu9ULenWDzDYuyN6vHtzdOt2qEu2\nbzpnEDc/uYaqYP2yNT6PEYpEOOt/X+WGrx/D5aMHkuHVhwciIu1VV9HOUuuISEeYMKygw1tdH3vs\nsbjns7KyWLx4cdxrtX3Y+fn5rF27tu78jTfe2KGxHWhKtFuy9DdAkwbsYBW8NKMu0U702+CIo3pw\n29Pr+M2iD3hiZQl3fHMIxV9UdehvjSIihxq1johIZ6J3qpbsLm7T+US/DT78vRN5ft0Wpi9cx8QH\n3sTrsbr1tmtXJ6m9X0REWlffOqKKtoikP/UztKRbv/jnc3q2+RFnDz6CF35yOrlZvmab2tSuTtJU\nR+/SJCJysFBFW0Q6EyXaLTlzGmQ0mcxoFp0Y+exPIdh8SZp4umT52FMdinutpKyK255ex9PvlVJa\nVpWUXZpERA4W5bGKtpb3E5HOQCWBltSuLvLSjGi7SLd+MPYXsHUdvPlH+M/b8K2/RlcoaUWi1Uky\nvR7mvPMfHnl9ExDd/EbrcouIxFdb0c5VRVtEOgG9U7WmcFKj5fzqDDgV5v8IZp0OhZNh45L6ZLzJ\nEoAQXZ3ktafu53oep6/toNTlcy+TOeWbV/ONwj68/3k5q/5Txq0L18UNo6SsihfWb2V4/+70ys1i\n/qqSNk2sbOs4EZHOoCIQokumF6/HUh2KiEir1DqyvwadCz96DXIPhxUPR5f+w9UvAVg0t9HwCd7X\nmZnxMP08O/AY9PPsYGbGw0zwvk6G10Nhv+58778GtLju9pV/X8GI21/kxNtf4Kdz32vUXjL1yaJm\n7SVqQxGRg01FIKiJkCIHkdzcXABKS0uZOHFi3DFjxoxhxYoVLT7n3nvvZe/evXXH5513HmVlZR0X\n6H5SRbs9uvWDcHXz88EqeOYG2L4BMrtEv165E1+4cU+3LxxotFQgJK58nzT+RwzI78LKz77g3hc+\n5Bv2b36WMbduzO9Ck7jhnxH+94UN9OySRX6XTN78ZGej9b0hfhuKquMi0llUBEKaCCnSkYrmNm6R\njfOp/IHQt29f5s2bt9/333vvvVx66aXk5OQAsGjRoo4KrV30btVeuxNUh2sq4d//S7N1uJvdvxnm\n/g/0GAg9BzLhi8+4wPcQ3kg0ge9nO5jpfRhf5gkwYBInDujJ+0se5s6Mh8mxmvoxGQ9DEDz9J7Fz\nTw2f7w6wtyYc9yVLyqq48u8r+FLvXHZX1fDkuyVUhyJ11+ItO1hbHa9N3Fsap2RcRJKlojqoRFuk\noxTNjX4KH4zNIYuzMd+++vnPf85RRx3F1VdfDcD06dMxM5YtW8YXX3xBMBjk9ttv58ILL2x036ZN\nmzj//PNZu3YtVVVVXH755axfv55jjz2Wqqr6OW5XXXUVy5cvp6qqiokTJ3Lbbbdx3333UVpaytix\nY8nPz2fp0qUMGDCAFStWkJ+fzz333MPs2bMBuOKKK7j++uvZtGkT48aN45RTTuGNN96goKCABQsW\nkJ3dsTt6692qvbr1i7WNND1/JFy/BkIBqNkLD4yGis+bj/P5Ycta+GARRKKz6b1Nh4QDsPBa+PRV\nyO7JrzP/Rg41jcbkWA2/yPwXR0y+s+7c6Jkvx52A6c/wsGnHHl7ZsI1g2DHe8xo/y2xcHf/J3DC3\nP7seM8NrxvaKAOE4kzR/+dQaPt5eSfecTD7ZUcG/lpdQE245aRcR2V8VgRA9cjJTHYZI57B4KmxZ\nk/h68fLmn8wHq2DBNbDyb/HvOeJ4GDcz4SMnT57M9ddfX5doz507l+eee44bbriBrl27smPHDk4+\n+WTGjx+PWfy5Fn/+85/JycmhqKiIoqIihg8fXnftjjvuoGfPnoTDYc4880yKioq49tprueeee1i6\ndCn5+fmNnrVy5UoeeeQR3n77bZxznHTSSZx++un06NGDjRs3MmfOHB566CEmTZrEE088waWXXpr4\n72s/KNFurzOnNf5tEKJLAp45LboUYEZ29OvrM+KPu+C+6G+NkXD0Y5s/nEDcKnioCj56Cfbuoitx\n2lWAw9kO2z6A3oPArMUJmBOGFRAKR/jpr36ZsDqeN/g7RBxEIo5/rtgcTch9jRPyhTWn8MelH+Fi\nIcdL2mc842HsoMPollPfV6nKt4jsj4pAiP49c1IdhsjBIV77a0vn22DYsGFs27aN0tJStm/fTo8e\nPejTpw833HADy5Ytw+PxUFJSwtatWzniiCPiPmPZsmVce220sl5YWEhhYWHdtblz5zJr1ixCoRCf\nf/4569evb3S9qddee41vfvObdOnSBYCLLrqIf//734wfP56BAwcydOhQAEaMGFG3DXxHUqLdXvGW\nAIzX39TaOI8XehzVcoX8hrXR7+8ZDOXNd600gPtPik7QHHgaEzJyuMD3eKM2lN96HsK7zeDt/vjK\nPuO3mQ/jj1Md/23mX8ju4oWu/aBrX778/jNcGnqSbAvWPWtmxsP0zMhk2i+mUx4Icuvtt8ZN2qdW\nwbBfBzm+oBujvxz9TXPrG//gnzxO36wdlO7N596nJgNXN0q2ly98kCPfvYvD3Ha2WW82D7+JE8f/\nsNnP3ZZxHfksoG09bW3te9Oz0vtZHf2a0i6aDCmyD1qoPAPw+yGJc47Ln93vl504cSLz5s1jy5Yt\nTJ48mUcffZTt27ezcuVKMjIyGDBgAIFAy3uRxKt2f/rpp9x9990sX76cHj16cNlll7X6HOcSt/Bm\nZWXVfe/1ehu1qHSUpCbaZnYu8Aei3RAPO+dmNrmeBfwdGAHsBL7tnNsUu3Yz8AMgDFzrnFuSzFjb\nJdESgPszrqUKea2zbo0/5oxfQWYufLoMPnkV9mxr1obijVTDm/83euDzk9Ukya7lpxreeSja+gJc\nCbFMvl6O1XBr6A/YXX+ju3m4J3MX3ibV+Byr4TeZj3DRQD8rvsjmzX/7Oc59xK8z5jRKyGe4Wfx6\ngYfD8q6jR5dMtr3+D0atmU621YDBEWyn28pbWA6NEt/lCx9kyMpbWhzXljH7Mo6iuYQW/Lh+cuvu\nzdFjqP/fty1j9Kz0f1ZHv6a0y/xVJeyorGHOO/9h2Yfb9UmYSHu1JefYD5MnT+bKK69kx44dvPrq\nq8ydO5fDDjuMjIwMli5dymeffdbi/aeddhqPPvooY8eOZe3atRQVFQFQXl5Oly5d6NatG1u3bmXx\n4sWMGTMGgLy8PCoqKpq1jpx22mlcdtllTJ06FeccTz31FP/4xz/a9fPti6Ql2mbmBf4EfB0oBpab\n2ULn3PoGw34AfOGc+7KZTQZ+C3zbzI4DJgODgb7Ai2b2Fedc/Nl9B5O2VMhbGzPie+Ac3NaD+JMx\nDX66AXIPw+49Pu5vs1bbY753F5SXwIOnxX2W4eD4b4GL4Fn+cNwfqQtVjPn0HsYAJChE5VgN0939\nrP778+xyXk7yfIg/Vj2vlW01DF75K974YDEejwevx8Pg3cuiiXGcce//5yUMGLLjlbhjhrx7C5tK\nX8LM8JgxpPTl+M9691dsLXsLzINhdP/0GTIjzVeQqZl/LXvWPIfh6PLxIjLijAnO/zGhNQswDHD4\nPn4RX7xxC64l/OHLOPPiW/8kGXFWrAkuuB63eSVmHszArfxb/HELryfj8/cACC5/pNUxLY5bcD2+\nkncBCK34e+JnxcZgRnDFAYhr4fX4SlbigPCKf7QcV6xK0mJcpavrX3PFX/c7/r2Lp5GjRLtDzF9V\nwtQni+qONQdEpAO09VP5fTR48GAqKiooKCigT58+fPe73+WCCy5g5MiRDB06lK9+9ast3n/VVVdx\n+eWXU1hYyNChQxk1asw9w4YAAA1+SURBVBQAJ5xwAsOGDWPw4MEcffTRjB49uu6eKVOmMG7cOPr0\n6cPSpUvrzg8fPpzLLrus7hlXXHEFw4YNS0qbSDzWUkm9XQ82+xow3Tl3Tuz4ZgDn3J0NxiyJjXnT\nzHzAFqA3MLXh2IbjWnrNkSNHutbWWTyktPSRUG0bStMZx9C4d3xfntXSmB8uiybs5aW4xyY1LY4D\n0TS+/PCTCYdq6LHjXeLNkXAOtnj74HDgHH0iWxOO+8wdTgRjoG1JOOYT16fu+Gj7POG4EvIxHB4c\nR7CrxXERZxxp2xOO+cgV1P3KcoyVJBz3OT3xEuEwyhKO2YO/7rgLgYTj9sbG5fz/9u4/SOr6vuP4\n88VxyKEE4nFV9EBQKSD1FyCamLGJITmBCK1WRavRtinR0YptRTGZEuLo9EfaJBPjpGMMM+ZXMaMx\npS3aONYmjWkMalEhZ61Vpl4ARRkabryacPfuH/slWe529/Zw93Y/u6/HzM5+v59973ff9+HuzWe/\n38/3+y0RU+629pM7I3sifSViJuS+hAFHlYh7S1leUfzz+nQEBAQqmX8uLzGRt4rG9Ga5i9wXwFL9\nFdlvaam4viz/tiL5DyDGrC/vOq6Sno6IhWUF10AZRyivAT4DHLwU0xcjovC378xIanaxk7uPn9zG\nE2vPL2sbZs2gu7ubuXPn1jqNhlKoT8ut2dWcOnI8kD/q6gHOLhYTEQck/S/QnrX/aNB7C+6ykLQK\nWAUwffr0iiTeMMo5JFTut9lytlUqZsLRucexp9LXNpUJfUOvwNLXNpVJ1+VmCO1efzLHsmdIzGvq\nYOq6F365XipuxqdfLBmzWx286+Zn+UX/AAf6g913ncpU3igQN4WdV29hIIKBCPrvO5tODY37aUyh\ne+UPGYhgzP3nFo155sJH6B+AgQgmbD6/aNy/LMntbf9AiZiHFz9KfwT9A8GKx7uKxn3tnH8kgI/+\n6CNFY77+nn9ijGCMxMonlhaNe/C8RwC4+PsXFI25/32biYAguPyJZUXjvrJwExHwsaeWF425d8Em\nWsaIMYKrn7ywaNwD5z3MGImLvlc8r43nbqY/goGB4KoSfbHhrH9AYtjPvPv0hzjQH6x+/qKCMTsH\n2ukc0pqeMo9QAtwfETdUI4edBQbZpdrNzOpBNe8MWWynZTkx5bw31xhxT0QsjIiFHR0dI0yxwZ12\naW7P9KRpgHLPg/dUH4z7422wfl/uudAho3K2VebnTVhyOwdaxh/SdqBlPBOW3P7L9Vfnr6EvDr2E\nV1+M49X5aw5pKyeuWEzP/DV0TDyC4ya3Mb19Aj3zbykSdwuLZh7NOSe2896TpnDvuCt5a1DcWzGO\ne8ddyYdOOYaueceWjLnsrOlccfZ0rjznhJJxH33PDK4aJuYPzzuRa3/zJK7/wMkl425bOpdPLJ1b\nMmbtkjnccsEcbu6aXTJu9eJZrF48q2TMn354Njd3zWZN15yScZ+6cB7rl88rGbN++Tz+7COn8Mll\np5SMu2nxr3PjB0vndXPXbG69YA63DdMX6y48pazP/POLTuMzl5xeMqZBLAJeioiXI+LnwEZgxTDv\nqajjitw1t1i7mVk9qOZAuweYlrfeCewsFpNNHZkE7C3zvVaOcgbRldxWmTFjV9x1yIB87Iq7Dok9\na/nH2bbgDnbTwUCI3XSwbcEdQ64AUk5cJbcFcMayVayLVfQM5KaJ9AxMYV2s4oxlq0YU423V/7Yq\n/ZkJK3SEstBRxoslPSfpAUnTCrx+2NZ0zaat9dDTu9taW1jTNbuSH2NmVlHVnKM9FngR+CC5OXtb\ngCsiYntezPXAqRFxbXYy5EURcamkecA3ye1FOQ54DJg13MmQnqNto6Wc64BX8tb23lbttlXpzyym\nnudoS7oE6IqIj2XrVwGLIuKP8mLagd6IeFvStcClETFk8vSg6X4Lhrv6QD5ff99seN3d3cyZM6fo\nzWBsZCKCF1544bDnaFdtoJ0lsRT4PLmTZzZExJ2SbgeeiohNksYDXwPOJLcne2VEvJy995PA7wMH\ngJsi4uHhPs8DbTNLVZ0PtIc9uX1QfAuwNyImldqua7ZZ5b3yyitMnDiR9vZ2D7bfoYjgzTffZP/+\n/cycOfOQ1+rhZEgiYjOweVDburzl/wMuKfLeO4E7q5mfmZmVZQswS9JMckcoVwJX5AdImhoRB89y\nXg50j26KZgbQ2dlJT08Pe/YMvQiAjdz48ePp7Dz809p9Z0gzMyspuyrUDcA/86sjlNvzj1ACN0pa\nTu4o5F7gmpolbNbEWltbh+x9tdrxQNvMzIZVxhHK24DbRjsvM7N6Vs2rjpiZmZmZNS0PtM3MzMzM\nqqCqVx0ZbZL2AOVfKypnChS4HWA6nH9tpZx/yrlD4+V/QkQ01V23DrNmQ9r/9innDs6/llLOHRov\n/7JqdkMNtA+HpKfq9ZJa5XD+tZVy/innDs6/maXcdynnDs6/llLOHZo3f08dMTMzMzOrAg+0zczM\nzMyqwANtuKfWCbxDzr+2Us4/5dzB+TezlPsu5dzB+ddSyrlDk+bf9HO0zczMzMyqwXu0zczMzMyq\noKkH2pIukPSfkl6StLbW+YyUpB2Snpe0VdJTtc5nOJI2SHpd0ra8tqMlPSrpv7Lnd9cyx2KK5L5e\n0k+z/t8qaWktcyxF0jRJj0vqlrRd0uqsve77v0TuSfS/pPGSfizp2Sz/T2ftMyU9mfX9/ZLG1TrX\neueaPbpSrtmQdt1OuWZD2nW70jW7aaeOSGoBXgQ+BPQAW4DLI+InNU1sBCTtABZGRBLXpZR0HtAL\nfDUifiNr+ytgb0T8RfYf57sj4tZa5llIkdzXA70R8de1zK0ckqYCUyPiGUkTgaeB3wKuoc77v0Tu\nl5JA/0sScGRE9EpqBX4ArAb+BPh2RGyU9LfAsxHxpVrmWs9cs0dfyjUb0q7bKddsSLtuV7pmN/Me\n7UXASxHxckT8HNgIrKhxTg0tIr4P7B3UvAK4L1u+j9wfYt0pknsyImJXRDyTLe8HuoHjSaD/S+Se\nhMjpzVZbs0cA5wMPZO112fd1xjV7lKVcsyHtup1yzYa063ala3YzD7SPB17NW+8hkV+CPAF8V9LT\nklbVOpnDdExE7ILcHybwazXOZ6RukPRcdoiyLg/hDSZpBnAm8CSJ9f+g3CGR/pfUImkr8DrwKPDf\nwL6IOJCFpFh/Rptrdn1IqmYUkUTdOCjlmg1p1u1K1uxmHmirQFtq82jOjYj5wBLg+uwwmY2eLwEn\nAWcAu4C/qW06w5N0FPAgcFNE/KzW+YxEgdyT6f+I6I+IM4BOcntm5xYKG92skuOabZWQTN2AtGs2\npFu3K1mzm3mg3QNMy1vvBHbWKJfDEhE7s+fXgYfI/TKk5rVsLtfBOV2v1zifskXEa9kf4wDwZeq8\n/7O5Zg8C34iIb2fNSfR/odxT63+AiNgH/CtwDjBZ0tjspeTqTw24ZteHJGpGMSnVjZRrNjRG3a5E\nzW7mgfYWYFZ2Fuk4YCWwqcY5lU3SkdkJBkg6EvgwsK30u+rSJuDqbPlq4O9rmMuIHCx2md+mjvs/\nO7njK0B3RHw276W67/9iuafS/5I6JE3OltuAxeTmKz4O/E4WVpd9X2dcs+tD3deMUhKqG8nWbEi7\nble6ZjftVUcAssvKfB5oATZExJ01Tqlskk4kt0cEYCzwzXrPX9LfAe8HpgCvAZ8CvgN8C5gO/A9w\nSUTU3ckrRXJ/P7nDXwHsAD5+cO5cvZH0PuDfgOeBgaz5E+TmzNV1/5fI/XIS6H9Jp5E7caaF3M6N\nb0XE7dnf8EbgaOA/gCsj4u3aZVr/XLNHV8o1G9Ku2ynXbEi7ble6Zjf1QNvMzMzMrFqaeeqImZmZ\nmVnVeKBtZmZmZlYFHmibmZmZmVWBB9pmZmZmZlXggbaZmZmZWRV4oG1NQVK/pK15j7UV3PYMSXV3\nLVAzs1S5ZlujGDt8iFlD6Mtup2pmZvXPNdsagvdoW1OTtEPSX0r6cfY4OWs/QdJjkp7Lnqdn7cdI\nekjSs9njvdmmWiR9WdJ2Sd/N7iaFpBsl/STbzsYa/ZhmZg3BNdtS44G2NYu2QYchL8t77WcRsQj4\nIrm7zpEtfzUiTgO+AXwha/8C8L2IOB2YD2zP2mcBd0fEPGAfcHHWvhY4M9vOtdX64czMGoxrtjUE\n3xnSmoKk3og4qkD7DuD8iHhZUiuwOyLaJb0BTI2IX2TtuyJiiqQ9QGf+bVclzQAejYhZ2fqtQGtE\n3CHpEaCX3G2LvxMRvVX+Uc3MkueabY3Ce7TNIIosF4sp5O285X5+df7DMuBuYAHwtCSfF2Fm9s64\nZlsyPNA2g8vynv89W/4hsDJb/l3gB9nyY8B1AJJaJL2r2EYljQGmRcTjwC3AZGDIHhozMxsR12xL\nhr+pWbNok7Q1b/2RiDh4uagjJD1J7ovn5VnbjcAGSWuAPcDvZe2rgXsk/QG5vSDXAbuKfGYL8HVJ\nkwABn4uIfRX7iczMGpdrtjUEz9G2ppbN91sYEW/UOhczMyvNNdtS46kjZmZmZmZV4D3aZmZmZmZV\n4D3aZmZmZmZV4IG2mZmZmVkVeKBtZmZmZlYFHmibmZmZmVWBB9pmZmZmZlXggbaZmZmZWRX8P76H\nAwRsVLHYAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1fa7f0fb438>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 透過趨勢圖來觀察訓練與驗證的走向 (特別去觀察是否有\"過擬合(overfitting)\"的現象)\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def plot_train_history(history, train_metrics, val_metrics):\n",
    "    plt.plot(history.history.get(train_metrics),'-o')\n",
    "    plt.plot(history.history.get(val_metrics),'-o')\n",
    "    plt.ylabel(train_metrics)\n",
    "    plt.xlabel('Epochs')\n",
    "    plt.legend(['train', 'validation'])\n",
    "    \n",
    "    \n",
    "plt.figure(figsize=(12,4))\n",
    "plt.subplot(1,2,1)\n",
    "plot_train_history(history, 'loss','val_loss')\n",
    "\n",
    "plt.subplot(1,2,2)\n",
    "plot_train_history(history, 'acc','val_acc')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 驗證評估 (Evaluation)\n",
    "\n",
    "讓我們加載測試數據並評估我們的模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_test.shape:  (12630, 48, 48, 3)\n",
      "y_test.shape:  (12630,)\n"
     ]
    }
   ],
   "source": [
    "# 載入測試資料\n",
    "import pandas as pd\n",
    "test = pd.read_csv('GTSRB/GT-final_test.csv', sep=';') # 裡面包含了六個欄位 Filename;Width;Height;Roi.X1;Roi.Y1;Roi.X2;Roi.Y2;ClassId\n",
    "\n",
    "X_test = []\n",
    "y_test = []\n",
    "\n",
    "# 迭代處理每一筆要測試的圖像檔\n",
    "i=0\n",
    "for file_name, class_id in zip(list(test['Filename']),list(test['ClassId'])):\n",
    "    img_path = os.path.join('GTSRB/Final_Test/Images/',file_name)\n",
    "    X_test.append(preprocess_img(io.imread(img_path)))\n",
    "    y_test.append(class_id)\n",
    "    \n",
    "# 轉換成numpy ndarray\n",
    "X_test = np.array(X_test)\n",
    "y_test = np.array(y_test)\n",
    "\n",
    "print(\"X_test.shape: \", X_test.shape)\n",
    "print(\"y_test.shape: \", y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12630/12630 [==============================] - 1s 116us/step\n",
      "Test accuracy = 0.9707046714172605\n"
     ]
    }
   ],
   "source": [
    "# 預測與比對\n",
    "y_pred = model.predict_classes(X_test)\n",
    "acc = np.sum(y_pred==y_test)/np.size(y_pred)\n",
    "print(\"Test accuracy = {}\".format(acc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 哇! 我們在這樣的架構下就可以訓練出一個準確率達到**97%**的模型。與人類的平均表現相差不遠了（98.84%）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 圖像增強 (Data Augmentation)\n",
    "\n",
    "你可能會認為40,000圖像是很多的圖像。再想一想,我們的模型裡有1,358,155個參數。這是訓練圖像的數量的40多倍。\n",
    "如果我們可以從現有的圖像生成新的訓練圖像，這將是一個很好的方式來增加訓練數據集的大小。\n",
    "\n",
    "讓我們直接使用keras的內置功能來完成圖像增強 (Data Augmentation)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_val, Y_train, Y_val = train_test_split(X, Y, test_size=0.2, random_state=42)\n",
    "\n",
    "datagen = ImageDataGenerator(featurewise_center=False, \n",
    "                            featurewise_std_normalization=False, \n",
    "                            rotation_range=10.,\n",
    "                            width_shift_range=0.1,\n",
    "                            height_shift_range=0.1,                             \n",
    "                            shear_range=0.1,\n",
    "                            zoom_range=0.2,\n",
    "                            )\n",
    "\n",
    "datagen.fit(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 2.5697 - acc: 0.2650 - val_loss: 1.1405 - val_acc: 0.6341\n",
      "Epoch 2/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.9100 - acc: 0.7102 - val_loss: 0.1976 - val_acc: 0.9338\n",
      "Epoch 3/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.4247 - acc: 0.8645 - val_loss: 0.0777 - val_acc: 0.9748\n",
      "Epoch 4/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.2750 - acc: 0.9142 - val_loss: 0.0568 - val_acc: 0.9811\n",
      "Epoch 5/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.2022 - acc: 0.9380 - val_loss: 0.0382 - val_acc: 0.9878\n",
      "Epoch 6/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.1674 - acc: 0.9508 - val_loss: 0.0206 - val_acc: 0.9936\n",
      "Epoch 7/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.1443 - acc: 0.9556 - val_loss: 0.0244 - val_acc: 0.9929\n",
      "Epoch 8/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.1338 - acc: 0.9591 - val_loss: 0.0170 - val_acc: 0.9944\n",
      "Epoch 9/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.1177 - acc: 0.9646 - val_loss: 0.0148 - val_acc: 0.9959\n",
      "Epoch 10/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0995 - acc: 0.9700 - val_loss: 0.0108 - val_acc: 0.9960\n",
      "Epoch 11/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0568 - acc: 0.9825 - val_loss: 0.0065 - val_acc: 0.9978\n",
      "Epoch 12/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0431 - acc: 0.9877 - val_loss: 0.0062 - val_acc: 0.9972\n",
      "Epoch 13/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0413 - acc: 0.9870 - val_loss: 0.0054 - val_acc: 0.9982\n",
      "Epoch 14/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0366 - acc: 0.9887 - val_loss: 0.0064 - val_acc: 0.9981\n",
      "Epoch 15/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0349 - acc: 0.9895 - val_loss: 0.0054 - val_acc: 0.9980\n",
      "Epoch 16/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0368 - acc: 0.9894 - val_loss: 0.0056 - val_acc: 0.9983\n",
      "Epoch 17/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0302 - acc: 0.9906 - val_loss: 0.0057 - val_acc: 0.9983\n",
      "Epoch 18/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0291 - acc: 0.9913 - val_loss: 0.0047 - val_acc: 0.9982\n",
      "Epoch 19/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0288 - acc: 0.9914 - val_loss: 0.0052 - val_acc: 0.9983\n",
      "Epoch 20/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0309 - acc: 0.9905 - val_loss: 0.0051 - val_acc: 0.9985\n",
      "Epoch 21/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0256 - acc: 0.9912 - val_loss: 0.0052 - val_acc: 0.9986\n",
      "Epoch 22/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0276 - acc: 0.9912 - val_loss: 0.0048 - val_acc: 0.9987\n",
      "Epoch 23/30\n",
      "981/980 [==============================] - 13s 14ms/step - loss: 0.0267 - acc: 0.9915 - val_loss: 0.0046 - val_acc: 0.9986\n",
      "Epoch 24/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0231 - acc: 0.9928 - val_loss: 0.0045 - val_acc: 0.9986\n",
      "Epoch 25/30\n",
      "981/980 [==============================] - 13s 14ms/step - loss: 0.0246 - acc: 0.9918 - val_loss: 0.0046 - val_acc: 0.9986\n",
      "Epoch 26/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0263 - acc: 0.9922 - val_loss: 0.0048 - val_acc: 0.9986\n",
      "Epoch 27/30\n",
      "981/980 [==============================] - 13s 14ms/step - loss: 0.0231 - acc: 0.9930 - val_loss: 0.0046 - val_acc: 0.9986\n",
      "Epoch 28/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0253 - acc: 0.9924 - val_loss: 0.0045 - val_acc: 0.9985\n",
      "Epoch 29/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0232 - acc: 0.9931 - val_loss: 0.0044 - val_acc: 0.9987\n",
      "Epoch 30/30\n",
      "981/980 [==============================] - 14s 14ms/step - loss: 0.0247 - acc: 0.9924 - val_loss: 0.0045 - val_acc: 0.9986\n"
     ]
    }
   ],
   "source": [
    "# 重新重置一個新的模型\n",
    "model = cnn_model()\n",
    "\n",
    "# 讓我們用相同的組合\n",
    "lr = 0.01\n",
    "sgd = SGD(lr=lr, decay=1e-6, momentum=0.9, nesterov=True)\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "             optimizer=sgd,\n",
    "             metrics=['accuracy'])\n",
    "\n",
    "def lr_schedule(epoch):\n",
    "    return lr*(0.1**int(epoch/10))\n",
    "\n",
    "batch_size = 32\n",
    "nb_epoch = 30\n",
    "\n",
    "# 透過data generator來產生訓練資料, 由於資料是可持續產生, 我們可以透過設定'steps_per_epoch'的數量來讓模型可以有更多的訓練批次\n",
    "history = model.fit_generator(datagen.flow(X_train, Y_train, batch_size=batch_size),\n",
    "                            steps_per_epoch=X_train.shape[0]/batch_size,\n",
    "                            epochs=nb_epoch,\n",
    "                            validation_data=(X_val, Y_val),\n",
    "                            callbacks=[LearningRateScheduler(lr_schedule),\n",
    "                                       ModelCheckpoint('model2.h5',save_best_only=True)]\n",
    "                           )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtQAAAEKCAYAAAAy8cIyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xl8lOW5//HPnclkmbBMElAggEHr\nQVYJRMViKxarghVxKWJrWz2n2mqt1bZY6GndzumRVo9b6/LD1rb251J+oIAVpWq17lYQZJWCBSQJ\nexYgmSSTmfv3x5OELM8kA5mZTJjv+/XKi8wz9zzPBbWTK/dcz3UZay0iIiIiInJ00ro7ABERERGR\nnkwJtYiIiIhIFyihFhERERHpAiXUIiIiIiJdoIRaRERERKQLlFCLiIiIiHSBEmoRERERkS5QQi0i\nIiIi0gVKqEVEREREuiC9uwM4Uv369bOFhYXdHYaIyFFZuXLlPmtt/+6OI1H0ni0iPVm079k9LqEu\nLCxkxYoV3R2GiMhRMcZs7+4YEknv2SLSk0X7nq2SDxERERGRLlBCLSIiIiLSBUqoRURERES6oMfV\nUItI7AWDQUpKSqitre3uUI4ZWVlZDB48GK/X292hiIhInCmhFhFKSkro3bs3hYWFGGO6O5wez1rL\n/v37KSkpYdiwYd0dTtSMMU8AXwH2WGtHuzxvgAeBaUANcLW19qPERikiknyO+ZKPxatKmTTvbwyb\n8yKT5v2NxatKuzskkaRTW1tLfn6+kukYMcaQn5/fE3f8/wBc0MHzU4GTG7+uAx5NQEySSGsWwP2j\n4Q6/8+eaBfE9V7TX68nn6unxp8K5YuCY3qFevKqUuc+tJRAMAVBaGWDuc2sBmFFU0J2hiSQdJdOx\n1RP/Pa21bxpjCjtYcjHwpLXWAu8bY/zGmIHW2p0JCTDVrFkAr90FVSXQdzBMuQ3Gzjy6ddGueeEm\nCAacx1U7nMfQem2sznUk1+up5+rp8afCuWLkmE6o71m+qTmZbhIIhrhn+SYl1CIiR64A2NHicUnj\nMSXUTWKVBCcq+Rh9GdQfgrpD8NefH17TJBiAv/4MjhsBaV7Y8gr87b+hofbwuZZ+Hyo/g2FnO8dD\ndfDyXPdzvfSTxgcGXprjvublOZCe6awxpoNzzXG+t+EO1tzq/P2avHZX5HXBgHO9V26LvKahFqwF\nLLx6h/u6ZbMhUOmssRbeuDvCuh/DgTJn3dv3u6958UdQvvXwud5/pON1xsC7v4l8vaoW//d958HO\n10WzJhbnOlDmxI6Bt+7t+Fw2DO881MG/w79o/m8n0r/Fa3fFPKE2zkZDz1FcXGyjHRIwbM6LuP3t\nDLB13oUxjUukJ9u4cSMjRozotutXVlby9NNPc8MNNxzR66ZNm8bTTz+N3++PU2Rd4/bvaoxZaa0t\n7qaQOtW4Q/2XCDXULwJ3W2vfbnz8GnCrtXZlm3XX4ZSEMHTo0Anbtx8Ds2yOZlcWID0bps6DMTPB\nkwFpHlj7/9zXffku+NwUqK+G/3sZVO9pH0dmHyi+BuprIFgD656DhkD7dZ4MGDgOsLDzYwjVu/yl\njPO8SIqxGMwdlVGtjfY9+5jeoR7kz6a0sv0bzSB/djdEI3LsWLyqlHuWb6KsMsAgfzazzx/epU99\nKisreeSRR9ol1KFQCI/HE/F1y5YtO+prylEpAYa0eDwYKGu7yFo7H5gPziZIYkKLo0g7vPWHoP8p\nsPcT2PMJrPyDszvbUkMAXviB8wVETGIbAvDS7M5jqTsA7z8GGT7w5rgn0+Ak0Bk+53quyTROHJPn\nQkYvyOwFr94Fgf3tl/n6wVfuh3AQu/DfcStmsoD5+kJnZ9mTSeCpr5Fdt6/dukDWcWR/+yXAEnh8\nKtl1e9uvyexP9jWLadqVDfzhEvdzZfYn69qXMGkeAvPPJ7u2/S8g1ZnHUXnVX6lvCFHXECb3qfM5\nnvJ263aTx7uTnyUUCvOFt67keCrar7G5LBr/B8LWSci+uupqBpj259pFHq99cSHedA8ZHg8TX7mY\nAbT/d91JPm+ft4yghXNemcZAlzVl9GPRWS9icXZvL397KoNo/29RZvvxpzOWYm2Yb/5jBoOM25p8\nfjr4j9QGQwSCYR4p/zYFpv01S20+P+j/BBnphnt3XsMglzVlNp+7hv2JUNgSsvDfn33TdV2pzedb\nOY/RELI8VXt9xOt9PePXeD3g9Rh+e+jGiOf6WuYj1IVhUfAG13OV2H5ckv4o3jTnXE8HvuO6bjf9\nGNDuaNfELaE2xgwBngQGAGFgvrX2wTZrJgNLgK2Nh56z1t4Vqxhmnz+8VQ01QLbXw+zzh8fqEiIp\nJx73JsyZM4dPP/2UcePG4fV66dWrFwMHDmT16tVs2LCBGTNmsGPHDmpra/nBD37AddddBxwea33o\n0CGmTp3KWWedxbvvvktBQQFLliwhO1u/PMfYUuBGY8yzwBlA1TFRP93Z7vOrd7p/bPyXWw4/9ua0\nT6ZbOvcOCAWdrzd/FXndJfOdRPgvt0B1+2STvoPhlvWHH98/uvVH783rhsA3l3S+ZvKcFn8HHw1L\nvk966PDNtA2eLNIvuBtGTqchFGYvtzLQJaHbST9e3HkS6R5DuieNDXVf4+f2MXzmcDJfYzO4I3AF\n2e/UUl0fwgau4L/M/HZrfnpoJu/+bhfBUJiGkOWc4EzmeX/bbt2cg19l6b3/BGB62uWua+YevJyl\nD29oPjY9bZbrul8EZ7H05X2Na66MsOZKXv5HAGMgzRg2h9zP9T/BWSz9664W17zCdd3dwStYuvTT\nDtfMC85k6aufNh/bkub+bzEvOJOX3/2MNANloUhrruBAMI0sr5d+Pg+/3O1+zV8Gr8Dn81EXDDEv\nGCmuK/isKownzZCWZiKu+2XwCkYN7Y/Xk8YvV0Vec/qYwTSELA1hy7y1kdedNnogXo/hlyvc1/wq\nOJMvjxtIQ+N/O7/8ONK//VdplZDGQDx3qBuAH1lrPzLG9AZWGmNesdZuaLPuLWvtV+IRQNMP99uW\nrONAbQOD+mZx6wWnqH5apAN3vrCeDWUHIj6/6rNK6kPhVscCwRC3LlzDM//4zPU1Iwf14faLRkU8\n57x581i3bh2rV6/mjTfe4MILL2TdunXNLeeeeOIJ8vLyCAQCnHbaaVx22WXk5+e3OsfmzZt55pln\nePzxx5k5cyaLFi3iqquuivavLYAx5hlgMtDPGFMC3A54Aay1jwHLcFrmbcFpm3dN90QaQ267z0tu\nhH++DCYNdq2FAyWRX//1hdB/OPQZDA+OjZy4ntUi+f74mcjrTr3C+T4YaF8W4s2GKbe3fs2U2yKs\nu+3I1gCLQ5N4O/htbuZZBpn9lNl87m24gs1vDCDwyhvsKK9hqo2c0C1dtrHF2SZSndbArekLms/1\nq4aZLA2fSd/VZeRkeCir/zzBtLDLmknMOuU4vJ40vJ40nnjnLAjisu4sfjDlZCzw0GtEXPOry8eS\n5fWQlZ7G3OcymBNov+6DXlP44MazSE8zfOXXWcw52H7Nyj5f5p9zvtT8N5w0L4M5B9qvW9H7XD6+\n+YvUNYSobwhz6SOZzKl2v+a7N0zC60lj+m/cr7miz7lsmX1O803OX/hVlus1W8Y2aV5mxDXv3DCp\nRfwHI6/799Mb1wQir/nBF1qcqy7yullFzppP90dec/mph8+1vSLyuq866yb9c1/kNZeMOXyureUR\n18Va3BLqxl2LnY3fHzTGbMS5eaVtQh1XM4oKOFTXwM8Wr+O5GyYxoG9WIi8vcsxpm0x3dvxonH76\n6a36Nz/00EM8//zzAOzYsYPNmze3S6iHDRvGuHHjAJgwYQLbtm2LWTypwlp7ZSfPW+B7CQqn6zra\neQ4FYf8W9xvjQnWwbpGTJA8Y49wwVefyS2bfIXByix/MUSauUa1rirOzuu1o1kWxpqK6njuWrqey\n/vMs5POtLpG+6xDnjjie80cN4JkP0plT555Err3lizSELMFwmIt+/TZLD5zF0vqzWp2rwJ/NO82J\n399YWum+Zt5lY5sfL1+/K+K6W778bwAsWlkScc3M4sNVSjX1IeY+F2q1Ltvr4e6pIzi+j5Mf/OSC\nU5j7XLD9mjafbjufgte3X3fBKfTN9tL4uyg/nTaCuc81uF6zqQQ18jVPId1zuMPxred3HlvEuKKN\nP8XOFSsJqaFuvMmlCPjA5ekzjTEf49Th/dhau95lTZfk5WQAUFFTr4RapBMd7SSD80PQ7d6EAn82\nf/7OmTGJIScnp/n7N954g1dffZX33nsPn8/H5MmTXfs7Z2ZmNn/v8XgIBCLUlkpqcNt5XnwDfPgE\nBA/B3k0d1BYDGPjhevdzgXuiHMskuGldNJ0Ioli3ODSJe+oeoqw2wKCsbGaHhnNefQOvbNjN0tVl\n/P2fe2kIu5e7h8KWx74xAYDhx/dm7nNh1ySyd9bhqaBzp47otOQy2rLMaNZFe66mT6g7ugckmjWx\nXqdzdd+5YiXuCbUxphewCLjZWtv2V/yPgBOstYeMMdOAxTgDA9qeo+Ud40ccg9/n/J+8oqajN08R\niUY87k3o3bs3Bw8edH2uqqqK3NxcfD4fn3zyCe+///5RX0dSiFt7tHAQSj6Ak77kfB0/2mkJd2h3\n+9f3HXz4+2gT4Ka1MUqCo9XZTcJu9z38aMHHpBkIhi0D+2bxH2cN4/lVpew52L4OvOWN/MmaFB1J\n4jSjqKDThCqaNbFep3N137liIa4JtTHGi5NMP2Wtfa7t8y0TbGvtMmPMI8aYftbafW3WdemO8Vyf\ns0NdWRM80peKSBvx+I0/Pz+fSZMmMXr0aLKzszn++OObn7vgggt47LHHGDt2LMOHD2fixIld/jtI\nCqiKUPdsLVy1qPWxaHefY9y3NhY6ukn4nOHH8em+Q9zxwvp2MxlC1pLp9fB/rz6N0wrzSEszjBjY\nJ+pd3mRMihKVOIm4iWeXDwP8Dthorb0vwpoBwG5rrTXGnI4zCt2lZ0/XNCXU2qEWiY14/OB6+umn\nXY9nZmby0ksvuT7XVCfdr18/1q1b13z8xz/+cUxjkx4op797H+eWO89wZLvPCRZNe8pfLf/EdYDZ\nDxesJkIFx+F19SHOOPHwvQiJ/Hhc5FgTzx3qScA3gLXGmNWNx34KDIXmO8YvB643xjQAAWCWjcOk\nmaaSD+1Qi4ikgP2fOsNR2vZ8dtt5hm7ZfT6aMo2fLFrDB1v3k+vL4NO9h/jX3mrKKtvfTwAQtjB3\n6imc1L8XP31+baelHE20yytydOLZ5eNtcO3/3nLNb4DfxCuGJlleD9leDxXV2qEWETmmVe+Hpy53\nBoycMxc++D8J3XmOZlc5UrK8YWcVJ/XvxZ4DdTz290/b7TzXNYR55h87SE8znJDv48T+vdhZVcuh\nuoZ2cRT4s/nO2ScBcKiuQTMZROLsmJ6U2FKuz0uFdqhFRI5dwVp49mtQVQrfegGGngGf/37CLu9e\nz7yGQ3UNjBrUh8/Ka/hsfw2PRkiW57+51e20rRhg439dgLexlVrba0L7ZFmlHCLxlzIJtd+XQaVq\nqEVEjk3hMCy5AXa8D5f/3kmmE+ye5Ztc6pnD/GzxugivaM0Ab956Dv17ZzLlf//u2p5ykD+7OZmG\nI+uUoQRaJH5SJqHOzfHqpkQRkWPV6//tDGM59w4YfWlCL32wNshLa3e5JsBNHv9mMSfk+xiS6+Pc\n+yIny0PyfMCRtadUsizS/VImofb7MthZGXmcsoiI9FAr/whv/S+M/xZMujkul2hbG/3DL5+M35fB\n86tKeWXDbuoawnjSDCGX1hoF/my+PPJwK8hokmWVaYj0LCmTUDs11NqhFjkW9OrVi0OHDlFWVsZN\nN93EwoUL262ZPHky9957L8XFxRHP88ADD3Ddddfh8zm7gtOmTePpp5/G7/fHLXaJkZZjxbFw3Ei4\n8H/BdHgv/FFxHYzy/9YAzs+WK04bwiVFBWzbV81Pn18Xk2l9TeuUQIv0DCmTUOf5MqgMBAmFLZ60\n2L/hiqSUlslMN/btHTRokGsyHa0HHniAq666qjmhXrZsWaxCk3hyGwVevhXWPx+X/w7daqMB8nIy\neH/uFDLSnZrmoqG5GGNiNq1PRHqOtM6XHBv8vgyshQMBdfoQ6ZKmZKZqB2CdP1+4yTl+lH7yk5/w\nyCOPND++4447uPPOO5kyZQrjx49nzJgxLFmypN3rtm3bxujRowEIBALMmjWLsWPHcsUVVxAIHE62\nrr/+eoqLixk1ahS33347AA899BBlZWWcc845nHPOOQAUFhayb58zqPW+++5j9OjRjB49mgceeKD5\neiNGjODaa69l1KhRnHfeea2uIwniNla8IeAcj4NItdEV1fXNyXSTGUUFvDPnS2yddyHvzPmSkmaR\nFJEyO9S5Oc5wl4qaenJzMro5GpEk9tIc2LU28vMlH0KozZCIYACW3OjUsroZMAamzot4ylmzZnHz\nzTdzww03ALBgwQJefvllbrnlFvr06cO+ffuYOHEi06dPx0T4SP/RRx/F5/OxZs0a1qxZw/jx45uf\n+8UvfkFeXh6hUIgpU6awZs0abrrpJu677z5ef/11+vXr1+pcK1eu5Pe//z0ffPAB1lrOOOMMzj77\nbHJzc9m8eTPPPPMMjz/+ODNnzmTRokVcddVVkf+9JPYijRWPdPwoVVTXc/vS9RGfdxuMIiKpKaV2\nqAH1ohbpqrbJdGfHo1BUVMSePXsoKyvj448/Jjc3l4EDB/LTn/6UsWPHcu6551JaWsru3bsjnuPN\nN99sTmzHjh3L2LFjm59bsGAB48ePp6ioiPXr17Nhw4YO43n77be55JJLyMnJoVevXlx66aW89dZb\nAAwbNoxx48YBMGHChObx55JAbceHd3b8KLyyYTdfvv9NXlq3k2mjB5Dlbf3jUoNRRKSl1Nmhbkyo\n1YtapBMd7CQDcP/oxnKPNvoOgWtePOrLXn755SxcuJBdu3Yxa9YsnnrqKfbu3cvKlSvxer0UFhZS\nW+s+ZrmJ2+711q1buffee/nwww/Jzc3l6quv7vQ81rbv1NAkMzOz+XuPx6OSj+4w5TbnE5GWv8RF\nGisehZYdPAb0zaLAn8WK7ZWMGNiHJ//9dEYO6hPVBEQRSV0ps0Od62sq+dAOtUiXTLnNSV5a6kIy\n02TWrFk8++yzLFy4kMsvv5yqqiqOO+44vF4vr7/+Otu3b+/w9V/84hd56qmnAFi3bh1r1jhdGA4c\nOEBOTg59+/Zl9+7dvPTSS82v6d27NwcPHnQ91+LFi6mpqaG6uprnn3+eL3zhC136+0kMjZ3Z4uZD\n4/wyd9FDR3VDYlMHj9LKABbYWVXLiu2VnD/yeJZ8bxIjB/UBVBstIh1LmR1qv3aoRWKjKWmJcZeP\nUaNGcfDgQQoKChg4cCBf//rXueiiiyguLmbcuHGccsopHb7++uuv55prrmHs2LGMGzeO008/HYBT\nTz2VoqIiRo0axYknnsikSZOaX3PdddcxdepUBg4cyOuvv958fPz48Vx99dXN5/j2t79NUVGRyjuS\nSe8BYNLgZ3vA4z3q00Tq4LGu7EC7Gw5FRCIxHX20mYyKi4vtihUrjvh11lo+958v8d2zT2T2+R3/\nYBZJNRs3bmTEiBHdHcYxx+3f1Riz0lobuTl2NzLGXAA8CHiA31pr57V5/gTgCaA/UA5cZa3t8E7A\no33P7tTz34Wtb8IPO66H78ywOS/i9lPQAFvnXdilc4tIzxfte3bK/PptjMGf7VXJh4iIC2OMB3gY\nmAqMBK40xoxss+xe4Elr7VjgLuDuxEbZQtOnI110XJ9M1+Pq4CEiRyJlEmoAv8+rkg8REXenA1us\ntf+y1tYDzwIXt1kzEnit8fvXXZ5PnKodTu10FwRDYbJcyjrUwUNEjlRKJdS5vgwqqrVDLeKmp5V/\nJbse+O9ZALRs31LSeKylj4HLGr+/BOhtjMlPQGythcNQVdrlHepf/20L28sDfOvMEyjwZ2OAAn82\nd186RjcdisgRSZmbEsG5MbGkoqa7wxBJOllZWezfv5/8/PyIg1MketZa9u/fT1ZWVneHciTc/odv\n+1vBj4HfGGOuBt4ESoGGdicy5jrgOoChQ4fGNkqAQ7shHOxSQv3htnJ+87fNXDZ+MHdePJo7u2+v\nXUSOASmVUOfleFlbqpIPkbYGDx5MSUkJe/fu7e5QjhlZWVkMHhy7QSMJUAK0rKEYDJS1XGCtLQMu\nBTDG9AIus9ZWtT2RtXY+MB+cmxJjHmnTRMSjLPk4UBvk5mdXMzjXx50Xj4phYCKSqlIqoc71ZVBR\nE8Raq104kRa8Xi/Dhg3r7jCke30InGyMGYaz8zwL+FrLBcaYfkC5tTYMzMXp+JF4TYOF/EeXUN+2\neB27DtTy/757Jr0yU+rHoIjESUrVUPt9GdQ3hF17joqIpDJrbQNwI7Ac2AgssNauN8bcZYyZ3rhs\nMrDJGPNP4HjgF90SbPMO9ZF/ArB4VSmLV5fxgyknM35obowDE5FUlVK/mreclujLSKm/uohIp6y1\ny4BlbY7d1uL7hcDCRMfVTtUOyOwDWX2P6GU7ymv4+eJ1FJ+Qyw2TT4pTcCKSilJuhxqgolp11CIi\nPdZR9KBuCIW5+c+rAbj/inGke1Lqx5+IxFlKbdM27VBXariLiEjPFWUP6sWrSrln+SbKKgP0ykrn\nYG0DD84ax5A8XwKCFJFUklK/oufmNO5Qa7iLiEjPFcUO9eJVpcx9bi2llQEscLC2AY8x9Lz24CLS\nE6RUQu1v3qFWQi0i0iPVHYJARacJ9T3LN7W7AT1kLfcs3xTP6EQkRaVWQp3dtEOtkg8RkR4pyh7U\nZZWBIzouItIVKZVQZ6Sn0SszXSUfIiI9VVNC3UkP6kH+7CM6LiLSFSmVUINT9qGbEkVEeqimoS6d\nlHzMPn842d7WP+KyvR5mnz88XpGJSApLuYQ615dBudrmiYj0TFU7wHig14AOl80oKuDHLZLnAn82\nd186hhlFBfGOUERSUNza5hljhgBPAgOAMDDfWvtgmzUGeBCYBtQAV1trP4pXTOB0+tBNiSIiPVRV\nCfQZBJ7Of3zlNXZ2WnbTFxg5qE+8IxORFBbPHeoG4EfW2hHAROB7xpiRbdZMBU5u/LoOeDSO8QBO\nL2rdlCgi0kNVlUTVgxpgxbYKememM3xA7zgHJSKpLm4JtbV2Z9Nus7X2ILARaPtZ28XAk9bxPuA3\nxgyMV0zglHzopkQRkR6qakfUUxJXbq9g3FA/njQT56BEJNUlpIbaGFMIFAEftHmqANjR4nEJ7ZPu\nmPL7vBysbaAhFI7nZUREJNbCIThQFlVCfaA2yKbdB5lwQm4CAhORVBf3hNoY0wtYBNxsrT3Q9mmX\nl7SbY2WMuc4Ys8IYs2Lv3r1diifX59TUVQZU9iEi0qMc3AXhhqgS6lWfVWItFJ+Ql4DARCTVxTWh\nNsZ4cZLpp6y1z7ksKQFaFsMNBsraLrLWzrfWFltri/v379+lmDQtUUSkh2ruQT2006Urt5WTZmDc\nUH+cgxIRiWNC3djB43fARmvtfRGWLQW+aRwTgSpr7c54xQSHd6h1Y6KISA8TZQ9qgBXbKxgxsA+9\nMuPWzEpEpFk832kmAd8A1hpjVjce+ykwFMBa+xiwDKdl3hactnnXxDEeoEVCrV7UIiI9S1NC3afj\nW20aQmFW76jkqxOiu3lRRKSr4pZQW2vfxr1GuuUaC3wvXjG4OVzyoR1qEZEepaoEsvpCVsc9pTfu\nPEhNfYgJhaqfFpHESL1JiTlNJR/aoRYR6VGqSqBv5/XTK7aXA1CsDh8ikiApl1DnZHjwegzlSqhF\nRFoxxlxgjNlkjNlijJnj8vxQY8zrxphVxpg1xphpCQ2wqiSq+umV2ysY2DeLQf7sBAQlIpKCCbUx\nhlxfBpXVKvkQEWlijPEAD+NMsB0JXOky3fZnwAJrbREwC3gkoUFWRjfUZeX2CvWfFpGESrmEGjQt\nUUTExenAFmvtv6y19cCzONNsW7JAUwFzX1zanMZNbRXUVXWaUJdWBthZVatyDxFJqJTsJ+T3eXVT\noohIa26Ta89os+YO4K/GmO8DOcC5iQkNqCp1/vQP6XDZim2N9dO6IVFEEkg71CIiAtFNrr0S+IO1\ndjBOy9M/GWPa/RyJ5XTbZk1DXfp2nFCv3F6BL8PDKQN6x+a6IiJRSM2EOserwS4iIq1FM7n2P4AF\nANba94AsoF/bE8Vyum2zqs+cPzsp+VixrYKioX7SPSn5401EuklKvuP4fRlU1tTjtMEWERHgQ+Bk\nY8wwY0wGzk2HS9us+QyYAmCMGYGTUMdoC7oTVSWQlg69jo+45FBdA5/sOsCEE1TuISKJlZIJda7P\nS0PYcqiuobtDERFJCtbaBuBGYDmwEaebx3pjzF3GmOmNy34EXGuM+Rh4BrjaJmpnoqrEmZCY5om4\nZPVnlYQt6vAhIgmXojclOsNdKmuC9M7ydnM0IiLJwVq7DFjW5thtLb7fAExKdFxAYw/qTm5I3F6O\nMVA01J+goEREHCm6Q61piSIiPUoUPahXbq9g+PG96aONEhFJsBRNqJ03W92YKCLSA4Qa4GBZhwl1\nKGxZ9VklxYUq9xCRxEvJhPpwyYd2qEVEkt7BnWDDHfag/mTXAQ7VNVCsGxJFpBukZELdtENdXq2E\nWkQk6TX3oI68Q/3R9gpANySKSPdIyYS6b7YXY1TyISLSI1Q1DnDs4KbEFdsrOK53JoNzsxMUlIjI\nYSmZUKd70uiT5VXJh4hIT9CcUEfeoV6xrYLiwlyMcRv4KCISXymZUINT9qEdahGRHqCqBLLzICPH\n9eldVbWUVgY00EVEuk3KJtRN0xJFRCTJVZV0vDu9vRyAYtVPi0g3SdmE2tmhVkItIpL0Knd0XD+9\nrYJsr4eRg/okMCgRkcNSOKHOoKJaJR8iIknNWqeGuqMOH59VcOqQvng9KfsjTUS6Wcq++6jkQ0Sk\nB6itgvpDEXtQ19Q3sL7sgPpPi0i3StmEOtfnpbo+RH1DuLtDERGRSDrpQb16RyWhsFX/aRHpVimb\nUPtzNC1RRCTpddKDeuU2Z6DL+KFKqEWk+6RsQt00LVGt80REklgnO9Qrtlfwb8f3om/je7qISHdI\n4YTa2aHW+HERkSRWtQM8GZCy0Wy/AAAgAElEQVRzXLunwmHLR59VqP+0iHS7lE2o/Y27GSr5EBFJ\nYlUl0KcA0tr/uNq85xAHaxvUf1pEul3KJtR5jTXUKvkQEUlile4t8xavKuWK//MeAL9a/gmLV5Um\nOjIRkWYpm1A3lXxouIuISBKrKml3Q+LiVaXMfW4tlQFnQ2T3gTrmPrdWSbWIdJuUTaizvB6yvGkq\n+RARaWSMucAYs8kYs8UYM8fl+fuNMasbv/5pjKmMa0ChIBzc2a4H9T3LNxEIhlodCwRD3LN8U1zD\nERGJJL27A+hOub4MlXyIiADGGA/wMPBloAT40Biz1Fq7oWmNtfaWFuu/DxTFNagDZYBtV/JRVhlw\nXR7puIhIvMVth9oY84QxZo8xZl2E5ycbY6pa7HbcFq9YItG0RBGRZqcDW6y1/7LW1gPPAhd3sP5K\n4Jm4RtTcg7p1Qj3In+26PNJxEZF4i2fJxx+ACzpZ85a1dlzj111xjMVVrs+rHWoREUcBsKPF45LG\nY+0YY04AhgF/i2tEzT2oW5d8zD5/OJnprX98ZXs9zD5/eFzDERGJJG4JtbX2TaA8XuePBafkQzvU\nIiKAcTlmI6ydBSy01obcnjTGXGeMWWGMWbF3796jjyjCDvWMogK+e/aJzY8L/NncfekYZhS55v8i\nInHX3TXUZxpjPgbKgB9ba9cn8uJ+n5dK7VCLiICzI91yK3gwznuzm1nA9yKdyFo7H5gPUFxcHCkp\n71xVCfj6gbd9KcfoAj8AL9x4FmMG9z3qS4iIxEJ3dvn4CDjBWnsq8GtgcaSFMdvtaCO3sYY6HD76\n93sRkWPEh8DJxphhxpgMnKR5adtFxpjhQC7wXtwjitCDGqCiccptbo5GjotI9+u2hNpae8Bae6jx\n+2WA1xjTL8La+dbaYmttcf/+/WMWg9/nJWzhYG1DzM4pItITWWsbgBuB5cBGYIG1dr0x5i5jzPQW\nS68EnrXWxn8noqokYkK9vzGhzs/JjHsYIiKd6baSD2PMAGC3tdYaY07HSe73JzKGpuEu5TX19PVp\nl0NEUlvj5sayNsdua/P4jgQF4yTUn5vi+nRFTT1Z3jSyMzwJCUdEpCNxS6iNMc8Ak4F+xpgS4HbA\nC2CtfQy4HLjeGNMABIBZCdnxaKHpo8KKmnqGkZPIS4uISEcCFRCsjrxDfaheu9MikjTillBba6/s\n5PnfAL+J1/Wj0bRDrV7UIiJJJkKHjyYVNfWqnxaRpJGyo8fhcEJdUa1OHyIiSaW5B7V7Ql1eXd/8\nHi4i0t2UUIN6UYuIJJvmhHqo69Pl1fXk5yihFpHkkNIJde+sdNIM6kUtIpJsqnaAJxNyXJs/UVFd\nT64SahFJEimdUKelGfyaligiknyaelCb9gMc6xvCHKxrIE8lHyKSJKJKqI0xPzDG9DGO3xljPjLG\nnBfv4BJB0xJFRJJQBz2omzZB8nopoRaR5BDtDvW/W2sPAOcB/YFrgHlxiyqBcrVDLSLHGGPMJcaY\nvi0e+40xM7ozpiNWVQL+Ia5PlTcOddEOtYgki2gT6qbP3KYBv7fWftziWI+W6/NSoR1qETm23G6t\nrWp6YK2txJkF0DM01MGhXdDXPaE+PHZcCbWIJIdoE+qVxpi/4iTUy40xvYFw/MJKHL8vQ32oReRY\n4/be3m2TcY/ImgXw4KnO9x885jxu4/DYcSXUIpIcon2D/Q9gHPAva22NMSYPp+yjx8v1eZs/PhQR\nOUasMMbcBzwMWOD7wMruDSkKaxbACzdBMOA8DlQ4jwHGzmxe1lSmpx1qEUkW0e5QnwlsstZWGmOu\nAn4GVHXymh7B78ugriFMoD7U3aGIiMTK94F64M/AAiAAfK9bI4rGa3cdTqabBAPO8RaaNkH82ZqU\nKCLJIdod6keBU40xpwK3Ar8DngTOjldgiZKXc3i4S3ZGdjdHIyLSddbaamBOd8dxxJqGuXRyvLy6\nHr/PS7onpTu/ikgSifbdqMFaa4GLgQettQ8CveMXVuLk+pwdDnX6EJFjhTHmFWOMv8XjXGPM8u6M\nKSoR2uS1PV5eXa8OHyKSVKJNqA8aY+YC3wBeNMZ4gGPiszZ/45uyelGLyDGkX2NnDwCstRXAcd0Y\nT3Sm3AbeNp8UerOd4y2Ua0qiiCSZaBPqK4A6nH7Uu4AC4J64RZVAub7DJR8iIseIsDFmaNMDY0wh\nzs2JyW3sTLjoocZ2ecb586KHWt2QCI071EqoRSSJRFVDba3dZYx5CjjNGPMV4B/W2ifjG1piHC75\n0A61iBwz/hN42xjz98bHXwSu68Z4ojd2ZrsEuq2KmnpOHezvcI2ISCJFO3p8JvAP4KvATOADY8zl\n8QwsUZpLPtQ6T0SOEdbal4FiYBNOp48f4XT66PGstSr5EJGkE22Xj/8ETrPW7gEwxvQHXgUWxiuw\nRMlITyMnw6MdahE5Zhhjvg38ABgMrAYmAu8BX+rkdRcADwIe4LfW2nkua2YCd+CUkHxsrf1aTIPv\nxKG6BoIhq6EuIpJUoq2hTmtKphvtP4LXJj1NSxSRY8wPgNOA7dbac4AiYG9HL2i82fxhYCowErjS\nGDOyzZqTgbnAJGvtKODmOMTeoYpqZ/NDO9Qikkyi3aF+ubHl0jONj68AlsUnpMTLzfHqpkQROZbU\nWmtrjTEYYzKttZ8YY4Z38prTgS3W2n8BGGOexWmVuqHFmmuBhxu7htBmoyUh9lfXARo7LiLJJdqb\nEmcbYy4DJgEGmG+tfT6ukcXKmgXOlK2qEqeX6ZTb2t3wkuvLoFwlHyJy7Chp7EO9GHjFGFMBlHXy\nmgJgR8tzAGe0WfNvAMaYd3DKQu5orNdOGI0dF5FkFO0ONdbaRcCiOMYSe2sWwAs3HR5lW7XDeQyt\nkmq/L4PPymu6IUARkdiz1l7S+O0dxpjXgb5AZ4mvcTtVm8fpwMnAZJz67LeMMaNb9rwGMMZcR2NX\nkaFDhxJL5Y0lHxrsIiLJpMM6aGPMQWPMAZevg8aYA4kK8qi9dtfhZLpJMOAcbyHX56VCXT5E5Bhk\nrf27tXaptbazN7kSYEiLx4Npv6tdAiyx1gattVtxuoic7HLN+dbaYmttcf/+/bsSfjvljSUfeb2U\nUItI8ugwobbW9rbW9nH56m2t7ZOoII9aVUlUx3N9GRyobaAhFE5AUCIiSelD4GRjzDBjTAYwC1ja\nZs1i4BwAY0w/nBKQfyUyyPLqIBkepzuTiEiyOGY6dbjqOziq403DXaoCqqMWkdRkrW0AbgSWAxuB\nBdba9caYu4wx0xuXLQf2G2M2AK8Ds621+xMZZ0V1Pbk5Xoxxq1AREekeUddQ90hTbmtdQw3gzXaO\nt9B0c0tFTZD8XpmJjFBEJGlYa5fRpoOTtfa2Ft9b4IeNX91if3U9eTl6nxaR5HJs71CPnQkXPQS+\nfs7jXsc7j9t0+WielqjWeSIiSa2ipp68HG93hyEi0sqxnVCDkzx/c4nz/dRftkum4XDJh6Yliogk\nt/LqenLV4UNEksyxn1AD5J7g/Fmxzf1pX1PJh3aoRUSSWXl1vYa6iEjSSY2EOrM3+PKhYrvr0/7G\nHWqVfIiIJK+GUJiqQFBDXUQk6aRGQg2QWxhxh7pXZjrpaUYlHyIiSazpPTpPCbWIJJm4JdTGmCeM\nMXuMMesiPG+MMQ8ZY7YYY9YYY8bHKxagw4TaGIPfl6EdahGRJNZUlqeEWkSSTTx3qP8AXNDB81Nx\nJmydjDOi9tE4xgL+E5zR4+GQ69O5Pi/lmpYoIpK0mt6jNXZcRJJN3BJqa+2bQHkHSy4GnrSO9wG/\nMWZgvOIhtxDCDXCg1P1pX4ZKPkREklhTQq0aahFJNt1ZQ10A7GjxuKTxWDvGmOuMMSuMMSv27t17\ndFdr7vQR+cZElXyIiCSvpoRaXT5EJNl0Z0LtNjfWui201s631hZba4v79+9/dFfLLXT+jFBHnZej\nHWoRkWRW0ZhQ+1XyISJJpjsT6hJgSIvHg4GyuF2tz2AwnogJddNNic5kXRERSTb7q+vpnZlORnrq\nNKgSkZ6hO9+VlgLfbOz2MRGostbujNvVPOnQdzBUupd85Pq8BEOW6nr3mxZFRKR7VdTUk9dLu9Mi\nknzS43ViY8wzwGSgnzGmBLgd8AJYax8DlgHTgC1ADXBNvGJp1kHrvK37qgEYc/tyBvmzmX3+cGYU\nuZZ0i4hIN9DYcRFJVnFLqK21V3byvAW+F6/ru8o9ATa91O7w4lWlPPdRiRMXUFoZYO5zawGUVIuI\nJIny6noG9Mnq7jBERNpJrUK03EKo3gv11a0O37N8E/Wh1rXTgWCIe5ZvSmBwIiLSkYrqerXME5Gk\nlHoJNbRrnVdWGXBdHum4iIgklrWW/dX1mpIoIkkptRJqf6HzZ5sbEwf5s12XRzouIiKJFQiGqGsI\nK6EWkaSUWgl1hF7Us88fTrbX0+pYttfD7POHJyYuERHpkMaOi0gyS62E2pcHGb3aJdQzigq4+9Ix\n5Pq8APTvncndl47RDYkiklKMMRcYYzYZY7YYY+a4PH+1MWavMWZ149e3ExWbxo6LSDJLrYTamMbW\nee17Uc8oKuDFm74AwHe+eKKSaRFJKcYYD/AwMBUYCVxpjBnpsvTP1tpxjV+/TVR8zTvUSqhFJAml\nVkINHfaiHuTPpjDfx3uf7k9oSCIiSeB0YIu19l/W2nrgWeDibo6pWUWNEmoRSV6pl1D7T3BuSoww\nYvzMk/rxj63lNITCCQ5MRKRbFQA7WjwuaTzW1mXGmDXGmIXGmCGJCQ32H1INtYgkr9RLqHMLIVjj\n9KN2ceZJ+Rysa2Bd2YHExiUi0r2My7G2Ow8vAIXW2rHAq8AfXU9kzHXGmBXGmBV797q/1x6pipp6\nPGmGPtlxm0cmInLUUjChPsH5M0LZx5kn5gOo7ENEUk0J0HLHeTBQ1nKBtXa/tbau8eHjwAS3E1lr\n51tri621xf37949JcOXVQXJ9GRjjlveLiHSvFEyoC50/XW5MBKfDx78d34t3P92XuJhERLrfh8DJ\nxphhxpgMYBawtOUCY8zAFg+nAxsTFVx5dR15Od5EXU5E5IikXkLtH+r8GWGHGpxd6hXbKqhvUB21\niKQGa20DcCOwHCdRXmCtXW+MucsYM71x2U3GmPXGmI+Bm4CrExVfRXVQNySKSNJKvYTamw29BnSc\nUJ/Uj0AwxMcllYmLS0Skm1lrl1lr/81ae5K19heNx26z1i5t/H6utXaUtfZUa+051tpPEhVbeY3G\njotI8kq9hBqcso9K95IPgIkn5mGM6qhFRJJFeXU9uerwISJJKkUT6hM63KH2+zIYObCP6qhFRJJA\nKGyprKknXzvUIpKkUjShLoQDpdBQH3HJmSfm89FnldQGQ4mLS0RE2qkKBAlbjR0XkeSVugm1DUPV\njohLPv+5fOobwny0vSJxcYmISDsaOy4iyS41E2p/x72oAU4rzMOTZnjvX6qjFhHpTho7LiLJLjUT\n6qZe1B3cmNg7y8uYgr68qxsTRUS6VdPYcd2UKCLJKjUT6t4DwZPR4Q41OGPIP95RSXVdQ2LiEhGR\ndpp2qPN7KaEWkeSUmgl1Wpoz4CXCtMQmnz8pn4aw5cNt5QkKTERE2mqqodYOtYgkq9RMqMEp++hk\nh7r4hDy8HtVRi4h0p/LqenwZHrK8nu4ORUTEVeom1P6Oe1EDZGd4KBqSqwEvIiLdqKJaUxJFJLml\nbkKdWwi1lRDoeLz4mSfls660iqpAMDFxiYhIKxo7LiLJLrUTauiw0wc4CXXYwj+2qo5aRKQ7aOy4\niCS7FE6oO+9FDVA01E9meprGkIuIdJPyao0dF5HklsIJdaHzZyedPjLTPRQXqo5aRKS7VFTXa+y4\niCS11E2os/pClr/THWqAz5/Uj092HWT/obr4xyUiIs1qgyGq60OqoRaRpJa6CTU4u9Sd1FADTDwx\nH4APVEctIpJQGjsuIj1BXBNqY8wFxphNxpgtxpg5Ls9fbYzZa4xZ3fj17XjG004UvagBxg7uS06G\nR3XUIiIJprHjItITxC2hNsZ4gIeBqcBI4EpjzEiXpX+21o5r/PptvOJxlXsCVH4G4XCHy7yeNE4b\nlqc6ahGRBNMOtYj0BPHcoT4d2GKt/Ze1th54Frg4jtc7crmFEKqHgzs7Xfr5k/L5dG81uw/Uxj8u\nEZFu0Nmnii3WXW6MscaY4njH1DR2XAm1iCSzeCbUBcCOFo9LGo+1dZkxZo0xZqExZkgc42nPH13r\nPIAzT+wHwPsaQy4ix6BoP1U0xvQGbgI+SERcSqhFpCeIZ0JtXI7ZNo9fAAqttWOBV4E/up7ImOuM\nMSuMMSv27t0buwijHO4CMHJQH/pkpfPuFiXUInJMivZTxf8CfgUk5OO6iup6jIG+2d5EXE5E5KjE\nM6EuAVruOA8GylousNbut9Y29aJ7HJjgdiJr7XxrbbG1trh///6xi7DvEDBpUe1Qe9IME0/M5z3t\nUIvIsanTTxWNMUXAEGvtXxIVVHmNMyXRk+a2RyMikhzimVB/CJxsjBlmjMkAZgFLWy4wxgxs8XA6\nsDGO8bSXngF9CqJKqMEZQ/5ZeQ0lFTXxjUtEJPE6/FTRGJMG3A/8qNMTxfBTRWfsuHanRSS5xS2h\nttY2ADcCy3ES5QXW2vXGmLuMMdMbl91kjFlvjPkYpybv6njFE1FuYafTEpvUBkMAnPXL15k0728s\nXlUax8BERBKqs08VewOjgTeMMduAicBStxsTY/mpojN2PLNL5xARibf0eJ7cWrsMWNbm2G0tvp8L\nzI1nDJ3ynwBbXu102eJVpTz02ubmx6WVAeY+txaAGUVu91qKiPQozZ8qAqU4nyp+relJa20V0K/p\nsTHmDeDH1toV8QyqojpIYT9fPC8hItJlqT0pEZwd6kO7IBjocNk9yzcRCLbuVx0Ihrhn+aY4Bici\nkhhRfqqYcPur69XhQ0SSXlx3qHuE5k4fn0H/4RGXlVW6J9yRjouI9DSdfarY5vjkBMRDRY0SahFJ\nftqhzo2uF/Ugf7br8YH+rBgHJCIiAAdqGwiFrcaOi0jSU0LdtEPdyY2Js88fTrbX0+745/r3ikNQ\nIiKioS4i0lMooc7pD15fpzvUM4oKuPvSMRT4szFAgT+bs0/ux5ub9/HsPz5LSKgiIqlECbWI9BSq\noTbG6fQRRS/qGUUFrTp6NITC/PsfV/DzJesY1i+HM07Mj2OgIiKpRQm1iLtgMEhJSQm1tQkZWJoS\nsrKyGDx4MF7v0fW9V0INTtlHFOPH20r3pPHrK4u45JF3uP6pj1jyvUkMyVN7JxGRWKhoTKhVQy3S\nWklJCb1796awsBBjNEW0q6y17N+/n5KSEoYNG3ZU51DJBzg3JlZsA2s7XdpW32wvv/1mMQ2hMNc+\nuYJDdQ2xj09EJAWV1zgJdX4vJdQiLdXW1pKfn69kOkaMMeTn53dpx18JNTg71PWHoKb8qF5+Yv9e\nPPz18Wzec4hb/ryacPjIE3MREWmtvLqezPQ01xvCRVKdkunY6uq/pxJqcGqoIao66ki+cHJ/fn7h\nCF7ZsJvv/GkFk+b9jWFzXtSIchGRo1TeONRFiYNI8qmsrOSRRx454tdNmzaNysrKOETUvZRQQ4vW\neVu7dJpvfb6QM0/M45WNeyitDGA5PKJcSbWIyJGp0JREkZhYvKo05ht9kRLqUCjU4euWLVuG3+/v\n8vWTjRJqODzc5ShuTGzJGMP2/TXtjmtEuYjIkdPYcZGuW7yqlLnPrY35Rt+cOXP49NNPGTduHKed\ndhrnnHMOX/va1xgzZgwAM2bMYMKECYwaNYr58+c3v66wsJB9+/axbds2RowYwbXXXsuoUaM477zz\nCAR67vRpdfkAyMhx+lF3oeSjyc4q94J2jSgXETkyFTX1DFXnJJEO3fnCejaUHYj4/KrPKqkPhVsd\nCwRD3LpwDc9EmKMxclAfbr9oVIfXnTdvHuvWrWP16tW88cYbXHjhhaxbt665S8YTTzxBXl4egUCA\n0047jcsuu4z8/NbthTdv3swzzzzD448/zsyZM1m0aBFXXXVVNH/tpKMd6ia5hTFJqCONKPdlejhQ\nG+zy+UVEUkW5dqhFuqxtMt3Z8aN1+umnt2o599BDD3HqqacyceJEduzYwebNm9u9ZtiwYYwbNw6A\nCRMmsG3btpjGlEjaoW7iPwFKPuzyaWafP5y5z60lEDxcQ+RJM1TXhfjSvW9w6wWncPn4waSl6SYb\nEZFI6hvCHKxtUEIt0onOdpInzfsbpS6fkhf4s/nzd86MWRw5OTnN37/xxhu8+uqrvPfee/h8PiZP\nnuzaki4zM7P5e4/Ho5KPY0KwxqmhvsMPfQfDlNtg7MwjPk3TJMV7lm+irDLAIH82s88fzueO68Xt\nS9dz68I1PP3BZ3zplP78+cOSVmtaTmEUEUlllY09qHOVUIt0idtGX7bXw+zzh3fpvL179+bgwYOu\nz1VVVZGbm4vP5+OTTz7h/fff79K1egIl1ABrFsCWVxofWKjaAS/c5Dw8yqTaLTle+N0zeX5VKbcv\nWcfqHYdbxjTdIND0WhGRVNc81EUJtUiXRNro62q+kZ+fz6RJkxg9ejTZ2dkcf/zxzc9dcMEFPPbY\nY4wdO5bhw4czceLELl2rJ1BCDfDaXRBqU98cDDjHjyKhjsQYw6XjB3PP8k0crGvdViYQDPE/yzZy\n8bhBrXquLl5VGvP/E4iIJLvyQxo7LhIrkTb6uurpp592PZ6ZmclLL73k+lxTnXS/fv1Yt25d8/Ef\n//jHMY8vkZRQA1SVHNnxLtoVoRPInoN1nP4/r/HFk/tz9vD+HAwE+e8XNzZ/TKOdbBFJFRo7LiI9\niRJqcGqmq3a4H4+DQf5s1xsE/D4vZwzL47VPdrPoI/dkvqmndcuEWrvYInKsKa/WDrWI9BxKqMG5\nAfGFm5wyj5bGfDUul4t0g8AdF41iRlEBobBlTUkllzzyruvrSysD3P3SRkYO7MOuqgAPvLqZQDDc\n/FykXWwl3iLSEWPMBcCDgAf4rbV2Xpvnvwt8DwgBh4DrrLUb4hFLU0Lt93njcXoRkZhSQg2H66Rf\nu8sp8+g90Hn8/iNwwiQ4+dyYXq6zGwQ8aYaiobkURNjJTk8zPPH2VoIh63r+QDDEnS+sp1+vTPJ7\nZZDfK4O3/7mX/1y8vtPyESXdIqnJGOMBHga+DJQAHxpjlrZJmJ+21j7WuH46cB9wQTziqaiup2+2\nF69H4xJEJPkpoW4ydmbrGxCr98OfLoZnr4Sv/hFOmRbTy0Vzg0Ckney7Lx3DtDED+XTvIaY++Jbr\naytqglz1uw86PH9T4j2sXw5D8nz8fdMefvr8OtVsi6Sm04Et1tp/ARhjngUuBpoTamtty3FsOYD7\nb/UxoLHjItKTKKGOJCcfvvUC/OlSWPANuOx3MGpGQkPobCd7xMA+EXexj+udyW++Np59h+rYf6iO\nny9Z73qNipogFz/8DgCG9j8dA8EQd7+0kemnDmo1jEY72SLHnAKg5c0kJcAZbRcZY74H/BDIAL4U\nr2AqaurJVbmHiPQQ+iytI9m58M0lUFAMC6+Bv/wQ7h/tDH+5f7TTvzrOZhQV8M6cL7F13oW8M+dL\n7ZLW2ecPJ9vraR2218NPp43g9GF5TBszkG+cWUhBhJHox/XOZP43JvDzr4yMuNW0+0AdI257mfPv\nf5Pv/GkF//GHD5m98GNKKwNYDu9kL15VGoO/sYh0E7fxre3eFqy1D1trTwJ+AvzM9UTGXGeMWWGM\nWbF3796jCqa8OkheTmbnC0WkR+jVqxcAZWVlXH755a5rJk+ezIoVKzo8zwMPPEBNTU3z42nTplFZ\nWdnBKxJDCXVnsvrAVYsg/2RY8bvGbiAthr8kIKnuyIyiAu6+dAwF/mwMzijRuy8dc0SJ93mjBvAf\nZw2LmHT7s71888wTGJLnY8ueQ7z2yZ529duBYIhfvfxJTP9uIpJQJcCQFo8HA2UdrH8WcP3Yzlo7\n31pbbK0t7t+//1EFU15dR16OdqhFYmLNgoRvCEYyaNAgFi5ceNSvb5tQL1u2DL/fH4vQukQJdTQy\ne0F9dfvjTcNf2krwf7id7WI3reks8Y6UdN8xfRT/eeFIfvutYl770WTXbSyAsqpafrTgY97YtIdg\nKMziVaVMmvc3hs15kUnz/qYdbJHk9iFwsjFmmDEmA5gFLG25wBhzcouHFwKb4xGItZaK6qDGjovE\nwpoFzgZgjDcEf/KTn/DII480P77jjju48847mTJlCuPHj2fMmDEsWbKk3eu2bdvG6NGjAQgEAsya\nNYuxY8dyxRVXEAgcLmG9/vrrKS4uZtSoUdx+++0APPTQQ5SVlXHOOedwzjnnAFBYWMi+ffsAuO++\n+xg9ejSjR4/mgQceaL7eiBEjuPbaaxk1ahTnnXdeq+vEimqoo3UgQjJYtQN+Pw36nwLHjYCDu+C9\nh6EhcPh5tzHmaxYc7irSd7DTui+GUxnddHYjZLTjSSP10fZlePjrhl0s+qgEnzeNupAlFHZ2smPR\nzi+adbGs7VaduKQSa22DMeZGYDlO27wnrLXrjTF3ASustUuBG40x5wJBoAL4Vjxiqa4PUR8Ka+y4\nSDRemgO71kZ+vuRDCNW1PhYMwJIbYeUf3V8zYAxMnef+XKNZs2Zx8803c8MNNwCwYMECXn75ZW65\n5Rb69OnDvn37mDhxItOnT281AbqlRx99FJ/Px5o1a1izZg3jx49vfu4Xv/gFeXl5hEIhpkyZwpo1\na7jpppu47777eP311+nXr1+rc61cuZLf//73fPDBB1hrOeOMMzj77LPJzc1l8+bNPPPMMzz++OPM\nnDmTRYsWcdVVV3X49ztSSqijFWn4izcHwg2wdiHUVbm/NhiAl+fCoCLILYT1z7fuex0p6Y5WDJPz\nrnQf+Z9LxjB1zADe/Oc+bnpmFaFwuNXrAsEQP1u8lvLqegb2zWKgP5uPd1Qw76VPOu2jvXhVaatr\nuq2LZk3L83WULMfyXH5cdBMAABI8SURBVNGu0bmOrfh7ImvtMmBZm2O3tfj+B4mIQ2PHRWKobTLd\n2fEoFRUVsWfPHsrKyti7dy+5ubkMHDiQW265hTfffJO0tDRKS0vZvXs3AwYMcD3Hm2++yU03OfnP\n2LFjGTt2bPNzCxYsYP78+TQ0NLBz5042bNjQ6vm23n77bS655BJycnIAuPTSS3nrrbeYPn06w4YN\nY9y4cQBMmDChefx5LCmhjpbb8BdvNlz0gJO8WgsHd8J9I3HtJFWzD35TDMYDxjhJeEvBALx2Z/tE\nuLNkuemjnGiS82gS7yjWzCgqoGDHXxjy0T0cZ/eyx/Rnx/jZnFbktKP98sjjqW2RbLd0qC7EXX/p\neA5EIBji1oVrePofn5GeZvCkGf6xtZy6BrcEfR3b99eQnZHGw69/2irJb1rzX3/ZwHG9M0lrPNc7\nW/bx6BufNp+vtDLArYvWsHnPQT5/Uj8awpb/+ssG13P9z7KNTDwxn15Z6fi8HpZ+XBazRD+WvzT0\n5HMdC/FL1zSNHVfbPJEodLKTzP2jI0yDHgLXvNilS19++eUsXLiQXbt2MWvWLJ566in27t3LypUr\n8Xq9FBYWUltb2+E53Havt27dyr333suHH35Ibm4uV199dafnsTZyF8/MzMM3OHs8np5X8hHF1K1M\n4ElgArAfuMJauy2eMR21tsNf2iabxkCfQZF3snsdB+feBfs3w1v/636NqhL4zemQdyLknwSBcli7\n6PBvkVU7YOn3oWIbDPsiNNQ6O99tJzwGA/DXn8GQM5ybKjP7wLpFnSfe0SbnaxZw2trbgQAYGMBe\nBqy9HQpzm9cN8mcz4cAr3Jq+gEFmH2W2H79qmMnKPufywve/wM6qADsra/n2kyuYnvZ2u3VLQ2eR\nZiAYChMIWuoawu7r6s7i/lf/2Rya65rqs/jab1v35J6e9ja3ZrRe9/DrYR5+/dOOz3XwLCbe/Vrz\n/+RYuMjlXLMXhvn9u9vwGFhXeoAL7Jvt1vxkkW01Yv4fW8s5P9x+3eyFYX739lYawpbNuw8yjbfa\nrbl1kWXpx2VkZ3jweT0sW7uTL4f+3m7dz/5/e/ceJUV55nH8+3TPIAjscFFZwoDgSlaDi8BOPJt1\nw6qsFzTejjdcNyd4omzcSzTJMSHmrKKGDZqsybpJTJTVjQZjSAwG1/smE4wbV7yhgOSiXBRFLnKb\nOXKZ6X72j6qZaWaqenqmGaur5/c5p89U1Tz91tPVzMPb1W/V+5Cx+p1dtOadxc+/FRlz3RJYvn47\nGYOMGQ++tDEy7vqfG7v2tDCgJsOtj/82Muam/84yoCZDLu/k3bnx4dWRcTcsNbY07SWXh1w+z/ef\nXhsTl+H9/TkG1GSY/8hrkTE3PpzBcVpzTmveWfBYdG7zlgZ51WQtNq+bHs6QzRh5D9q7OWafX39i\ngDrUB8lDL7/NzeGH7rkPruQrZ7Xq2IqUI+6E4Izr459TolmzZnHllVeybds2li1bxuLFizniiCOo\nra2lsbGRDRs2FH3+9OnTWbRoESeffDKrVq3i1VdfBWD37t0MHjyYuro6Nm/ezGOPPcZJJ50EwNCh\nQ2lqauoy5GP69OnMnj2buXPn4u4sWbKE++67r+zXWKo+61CXOOvWp4Ed7n60mc0CbgEu6aucytZ5\n8pcocf9wT5t/YMc1qtN9yFA4bCJsXwtrG4MOc2ete6FxfvAopnkz/HvhVyMRd5luG0O14n7I1sK6\np7vus2UPPPJ52PYHqB0INQPh6a/HdOK/AqOOg9qBfPfoF/jwyoUMsuBMU71t45bahaz6yHhGDJ7B\niMEDmPShOmYPWc4XWxZyaEHcgtqFjKgdwLw5N7Y3P++rN8TG/ct189jbkuOWW29mbmvXmKGZGs75\n5DXk8k7OnZ/ccxsLarvGWQtcduW1ZDPGkv/6Jtflu8YMthr+bOaVNO9roXlfjvWN90S2RQvsHHQ+\n7s4Z/nRszMZ9n8AJvuA4PR8f1zTkfGqyGSZufjQ2Zm3Tmby/P8ee/TlmtC6LjJvbAoue+2uyGSsa\n8+TqGbgHneAZLTFx++GGpcE3LedknomO2QP/sGh/+/sYG7cP/vXR1pLirlvSUjxmL3zuxy0ltfWF\nn3Tf1j//qPu2vrwb+vCWzP1G528Atjbv0zcAIuXq7oRgGSZNmkRTUxNjxoxh9OjRXHbZZZx99tk0\nNDQwZcoUjjnmmKLPv+qqq7j88suZPHkyU6ZM4YQTTgDg+OOPZ+rUqUyaNImjjjqKE088sf05c+bM\nYebMmYwePZrGxsb27dOmTWP27NntbVxxxRVMnTq1T4Z3RLFip8jLatjsY8A8dz89XP8ygLt/rSDm\niTDmWTOrAd4FDvciSTU0NHh39yhMXE+HaUA4fOT2jrh8Hm4aQfREZBbcyq9mIPx0NjRv6Rpy6Eg4\n9SbYuwv27oZlRb4Sqv9oMATlnZd78WJ7wdpuLmO452JufmvYgCFBrBm+dxcWcSwcw2oPBc/jrXti\nb6RrtYODtjIZ8nt3k4loK4eRPXRE8Jz3t8fvb2Bd+1rRtoaMAsvQ2vQuNeS7xLSSoaZuTNCbxsnt\n3kQ2Ii5HhuzQUcFy0+YiMR1j1HJN70bGte8TaN31dnxew4/sWN+xITbOhk8IXv2O9dTQdZhPK1ly\nw8Zj7mCQ2bkhJi5DZtiRYMFHv1yRfVI3LngfdsXFZMkPOxIzwzDYsS5yny1kYfiEoK0da6mNickP\nPwrDg39bO9ZFxr3L4fzxvNe7bI9iZi+6e0NJwVWgJzX7xAW/jLzgecywQfzvXH1gEWmzZs0ajj32\n2KTTqDpRx7XUmt2XQz5KmXWrPSa8wnwXMBLYVhhkZnOAOQDjxo3rq3wPnu7OZJfyaTGTiR8+UlcP\nR88Ilk+bH905P2PBge2tWBQ/huqK/wmWi42zumYl5PYH+/nux6Ap4va0hx4GZ30DWvbCQ5+Jf/0f\n/0J7J9Jihr8YDtM+GcR5Hlv+/fi4hsvBMthvbo+JIYjxfNDWc9+LjMvg8JHzwAx7fmH8/iZ3fIkS\nl1cGhw+fDp4n+3L0V05Z8jD+4+HYESOz4ocxbeVh4qnB8kv3xse0/ZsAMsX2OWE6uJN95f74mPqP\nhmtOdse62DgbE1zk4TveiInJUTPm+GDFDN+5Nr6tsQ20zS1SdJ/jgtx8ZVxMjpowryC36I5uDTls\n9HFhTPTd34KYSR35x7Q16sCSJb30TkRnuth2EZFK0Zcd6lJm3Sp1Zq47gTshONtRfmoVoJzhI4Xj\nnkr9KqeUtorFmEHNIcHj1BtjOvFfg0nnB+uN8+M756cUTK4WN/ylbmzQXpvfPRofd3o4/GX1ku5j\nAPvtI5FxVjcWPnFbsPL7J+LbOvPWjufE5GV1Y+GcoINva38VH3P+HR3r65YVaes/guU3GuNjzv12\nx3qxfZ4X3DfU1v86PuaCuzrW3/y/+LgL7w6W31oeH3PRPR3rxeIu6PgQY28+221c0bwK97nx+fi4\ni4NbRlnMh8nCmKCtF2Li6rtsk56LuyXnh2ImnRIRqRR9ObFLKbNutceEQz7qgO19mFO6TL44GAZS\nNxaw4GfhsJDCuM+tgnk7g59RHfVS2urJ/rqLm3F90MkuFHURxMGMU1vV0VY15C+9Eje51LWn/2lC\nGYmIlKYvz1C3z7oFvE0w69bfdopZSjAxwLPAhcAvi42f7pdKOZN9MNsqdX8HY1jLwY5TW9XRVjXk\nL71S6uRSIhLcJi5uwhTpuXK7n312USKAmZ0JfIuOWbfmF866ZWYDgfuAqQRnpme5e/Qgy1AqLkoU\nEYmhixJFpFzr1q1j6NChjBw5Up3qg8Ddee+992hqamLChAkH/K4SLkosZdatvcBFfZmDiIiISDWp\nr69n48aNbN26NelUqsbAgQOpr+/99TCaKVFEREQkRWpra7ucSZVk9eVFiSIiIiIiVU8dahERERGR\nMqhDLSIiIiJShj69y0dfMLOtwIZePPUwSPV0ZmnOP825g/JPWprzj8r9SHc/PIlkkqCanVppzj/N\nuYPyT1rn/Euq2anrUPeWmb2Q5ltVpTn/NOcOyj9pac4/zbknLe3HTvknJ825g/JPWm/z15APERER\nEZEyqEMtIiIiIlKG/tShvjPpBMqU5vzTnDso/6SlOf805560tB875Z+cNOcOyj9pvcq/34yhFhER\nERHpC/3pDLWIiIiIyEHXLzrUZnaGmf3OzF43s7lJ59MTZrbezFaa2QozeyHpfLpjZneb2RYzW1Ww\nbYSZPWVmfwh/Dk8yx2Ji8p9nZm+H78EKMzszyRzjmNlYM2s0szVmttrMrg63p+L4F8k/Lcd/oJkt\nN7NXwvxvDLdPMLPnwuP/YzMbkHSulS7NNRvSVbdVs5OV5rqtmt2pvWof8mFmWeD3wKnARuB54FJ3\nfy3RxEpkZuuBBndPxT0dzWw60Azc6+7HhdtuBba7+4LwP8fh7v6lJPOME5P/PKDZ3b+RZG7dMbPR\nwGh3f8nMhgIvAucBs0nB8S+S/8Wk4/gbMNjdm82sFngGuBr4PPAzd3/AzL4HvOLudySZayVLe82G\ndNVt1exkpbluq2YfqD+coT4BeN3d17r7fuAB4NyEc6pa7v40sL3T5nOBH4TLPyD4g6tIMfmngrtv\ncveXwuUmYA0whpQc/yL5p4IHmsPV2vDhwCnAT8PtFXv8K4hq9gdINTtZaa7bqtkH6g8d6jHAWwXr\nG0nRG07w5j5pZi+a2Zykk+mlUe6+CYI/QOCIhPPpjX8ys1fDrxcr7qu3zsxsPDAVeI4UHv9O+UNK\njr+ZZc1sBbAFeAp4A9jp7q1hSNrqTxLSXrMh/XU7dTUjQipqRqE0123V7P7RobaIbWka53Kiu08D\nZgL/GH69JR+sO4A/AaYAm4B/Szad4sxsCPAgcI277046n56KyD81x9/dc+4+BagnONN6bFTYB5tV\n6qS9ZoPqdtJSUzPapLluq2YH+kOHeiMwtmC9HngnoVx6zN3fCX9uAZYQvOFpszkca9U25mpLwvn0\niLtvDv/o8sBdVPB7EI4DexBY5O4/Czen5vhH5Z+m49/G3XcCvwL+AhhmZjXhr1JVfxKS6poNVVG3\nU1MzoqStZqS5bqtmd+gPHerngYnhVZsDgFnA0oRzKomZDQ4H+mNmg4HTgFXFn1WRlgKfCpc/Bfw8\nwVx6rK2ohc6nQt+D8AKL/wTWuPttBb9KxfGPyz9Fx/9wMxsWLg8C/oZgTGEjcGEYVrHHv4KktmZD\n1dTtVNSMOGmpGZDuuq2a3am9ar/LB0B4y5ZvAVngbnefn3BKJTGzowjObgDUAPdXeu5m9iPgJOAw\nYDNwA/AQsBgYB7wJXOTuFXkRSUz+JxF8deXAeuDv28a2VRIz+yvg18BKIB9uvo5gTFvFH/8i+V9K\nOo7/ZIILWLIEJysWu/tN4d/xA8AI4GXg79x9X3KZVr601mxIX91WzU5Wmuu2anan9vpDh1pERERE\npK/0hyEfIiIiIiJ9Rh1qEREREZEyqEMtIiIiIlIGdahFRERERMqgDrWIiIiISBnUoZaqYmY5M1tR\n8Jh7ENseb2YVeT9NEZE0Us2WalHTfYhIquwJpxEVEZHKp5otVUFnqKVfMLP1ZnaLmS0PH0eH2480\ns1+Y2avhz3Hh9lFmtsTMXgkffxk2lTWzu8xstZk9Gc6uhJl91sxeC9t5IKGXKSJSFVSzJW3UoZZq\nM6jT14eXFPxut7ufAHybYBY2wuV73X0ysAi4Pdx+O7DM3Y8HpgGrw+0Tge+4+yRgJ3BBuH0uMDVs\n5zN99eJERKqMarZUBc2UKFXFzJrdfUjE9vXAKe6+1sxqgXfdfaSZbQNGu3tLuH2Tux9mZluB+sLp\nRs1sPPCUu08M178E1Lr7V83scaCZYMreh9y9uY9fqohI6qlmS7XQGWrpTzxmOS4myr6C5Rwd1yGc\nBXwH+HPgRTPT9QkiIuVRzZbUUIda+pNLCn4+Gy7/BpgVLl8GPBMu/wK4CsDMsmb2R3GNmlkGGOvu\njcAXgWFAlzMuIiLSI6rZkhr6RCbVZpCZrShYf9zd227DdIiZPUfwQfLScNtngbvN7FpgK3B5uP1q\n4E4z+zTBWY2rgE0x+8wCPzSzOsCAb7r7zoP2ikREqpdqtlQFjaGWfiEcj9fg7tuSzkVERIpTzZa0\n0ZAPEREREZEy6Ay1iIiIiEgZdIZaRERERKQM6lCLiIiIiJRBHWoRERERkTKoQy0iIiIiUgZ1qEVE\nREREyqAOtYiIiIhIGf4fuvSf1WgsdxsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1fc9e7a2dd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 透過趨勢圖來觀察訓練與驗證的走向 (特別去觀察是否有\"過擬合(overfitting)\"的現象)\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def plot_train_history(history, train_metrics, val_metrics):\n",
    "    plt.plot(history.history.get(train_metrics),'-o')\n",
    "    plt.plot(history.history.get(val_metrics),'-o')\n",
    "    plt.ylabel(train_metrics)\n",
    "    plt.xlabel('Epochs')\n",
    "    plt.legend(['train', 'validation'])\n",
    "    \n",
    "    \n",
    "plt.figure(figsize=(12,4))\n",
    "plt.subplot(1,2,1)\n",
    "plot_train_history(history, 'loss','val_loss')\n",
    "\n",
    "plt.subplot(1,2,2)\n",
    "plot_train_history(history, 'acc','val_acc')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12630/12630 [==============================] - 2s 119us/step\n",
      "Test accuracy = 0.9870942201108472\n"
     ]
    }
   ],
   "source": [
    "# 預測與比對\n",
    "y_pred = model.predict_classes(X_test)\n",
    "acc = np.sum(y_pred==y_test)/np.size(y_pred)\n",
    "print(\"Test accuracy = {}\".format(acc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 驗證評估 (Evaluation)\n",
    "\n",
    "透過圖像增強 (Data Augmentation)後再訓練的這個模型，我在測試集上獲得了`98.7%`的準確度(灑花! 灑花!)。\n",
    "\n",
    "說正格的，我並沒有做太多的參數調校。以下列出一些可以嘗試改進模型的東西：\n",
    "* 嘗試不同的網絡架構。嘗試更深和更淺的網絡。\n",
    "* 嘗試添加`BatchNormalization`層到網絡。\n",
    "* 嘗試不同的權重初始化\n",
    "* 嘗試不同的學習速度和時間表\n",
    "* 製作一個集成模型(ensemble models)\n",
    "* 嘗試輸入圖像的正規化\n",
    "* 更積極的數據增強(Data Augmentation)\n",
    "\n",
    "這只是初學者的一種快速入門模型。對於這個問題的還有一些先進的解決方案，你可以看看[這個](http://torch.ch/blog/2015/09/07/spatial_transformers.html)，作者用一個稱為`Spatial Transformer`層的專門層\n",
    "達到`99.61%`的準確度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 總結 (Conclusion)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在這篇文章中有一些個人學習到的一些有趣的重點:\n",
    "* 對於圖像的成像亮度透過直方圖均衡化(Histogram equalization)可以輕鬆的調整 (又學到一招)\n",
    "* 深度學習的卷積網絡真的對圖像的辨識有很好的效果\n",
    "* 透過使用Keras的callback模組可以對模型的訓練過程有更多的監控與控制\n",
    "* Keras的`ModelCheckpoint`真的很好用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "參考:\n",
    "* [Keras Tutorial - Traffic Sign Recognition](https://chsasank.github.io/keras-tutorial.html)\n",
    "* [Keras官網](http://keras.io/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}