exercise-cross-validation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ac5a689f",
   "metadata": {
    "papermill": {
     "duration": 0.006291,
     "end_time": "2023-01-25T10:49:23.242491",
     "exception": false,
     "start_time": "2023-01-25T10:49:23.236200",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**This notebook is an exercise in the [Intermediate Machine Learning](https://www.kaggle.com/learn/intermediate-machine-learning) course.  You can reference the tutorial at [this link](https://www.kaggle.com/alexisbcook/cross-validation).**\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "701e9552",
   "metadata": {
    "papermill": {
     "duration": 0.005026,
     "end_time": "2023-01-25T10:49:23.252513",
     "exception": false,
     "start_time": "2023-01-25T10:49:23.247487",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "In this exercise, you will leverage what you've learned to tune a machine learning model with **cross-validation**.\n",
    "\n",
    "# Setup\n",
    "\n",
    "The questions below will give you feedback on your work. Run the following cell to set up the feedback system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "6611e7fd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-01-25T10:49:23.264874Z",
     "iopub.status.busy": "2023-01-25T10:49:23.264338Z",
     "iopub.status.idle": "2023-01-25T10:49:23.325590Z",
     "shell.execute_reply": "2023-01-25T10:49:23.324326Z"
    },
    "papermill": {
     "duration": 0.072172,
     "end_time": "2023-01-25T10:49:23.329591",
     "exception": false,
     "start_time": "2023-01-25T10:49:23.257419",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Setup Complete\n"
     ]
    }
   ],
   "source": [
    "# Set up code checking\n",
    "import os\n",
    "if not os.path.exists(\"../input/train.csv\"):\n",
    "    os.symlink(\"../input/home-data-for-ml-course/train.csv\", \"../input/train.csv\")  \n",
    "    os.symlink(\"../input/home-data-for-ml-course/test.csv\", \"../input/test.csv\") \n",
    "from learntools.core import binder\n",
    "binder.bind(globals())\n",
    "from learntools.ml_intermediate.ex5 import *\n",
    "print(\"Setup Complete\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ddc03911",
   "metadata": {
    "papermill": {
     "duration": 0.004669,
     "end_time": "2023-01-25T10:49:23.340451",
     "exception": false,
     "start_time": "2023-01-25T10:49:23.335782",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "You will work with the [Housing Prices Competition for Kaggle Learn Users](https://www.kaggle.com/c/home-data-for-ml-course) from the previous exercise. \n",
    "\n",
    "![Ames Housing dataset image](https://i.imgur.com/lTJVG4e.png)\n",
    "\n",
    "Run the next code cell without changes to load the training and test data in `X` and `X_test`.  For simplicity, we drop categorical variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "da31ceed",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-01-25T10:49:23.352663Z",
     "iopub.status.busy": "2023-01-25T10:49:23.352191Z",
     "iopub.status.idle": "2023-01-25T10:49:24.711675Z",
     "shell.execute_reply": "2023-01-25T10:49:24.710354Z"
    },
    "papermill": {
     "duration": 1.369668,
     "end_time": "2023-01-25T10:49:24.715214",
     "exception": false,
     "start_time": "2023-01-25T10:49:23.345546",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Read the data\n",
    "train_data = pd.read_csv('../input/train.csv', index_col='Id')\n",
    "test_data = pd.read_csv('../input/test.csv', index_col='Id')\n",
    "\n",
    "# Remove rows with missing target, separate target from predictors\n",
    "train_data.dropna(axis=0, subset=['SalePrice'], inplace=True)\n",
    "y = train_data.SalePrice              \n",
    "train_data.drop(['SalePrice'], axis=1, inplace=True)\n",
    "\n",
    "# Select numeric columns only\n",
    "numeric_cols = [cname for cname in train_data.columns if train_data[cname].dtype in ['int64', 'float64']]\n",
    "X = train_data[numeric_cols].copy()\n",
    "X_test = test_data[numeric_cols].copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2b7d08d",
   "metadata": {
    "papermill": {
     "duration": 0.007218,
     "end_time": "2023-01-25T10:49:24.730292",
     "exception": false,
     "start_time": "2023-01-25T10:49:24.723074",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Use the next code cell to print the first several rows of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "00c1b3f8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-01-25T10:49:24.747487Z",
     "iopub.status.busy": "2023-01-25T10:49:24.747070Z",
     "iopub.status.idle": "2023-01-25T10:49:24.775870Z",
     "shell.execute_reply": "2023-01-25T10:49:24.775063Z"
    },
    "papermill": {
     "duration": 0.041406,
     "end_time": "2023-01-25T10:49:24.779144",
     "exception": false,
     "start_time": "2023-01-25T10:49:24.737738",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>OverallQual</th>\n",
       "      <th>OverallCond</th>\n",
       "      <th>YearBuilt</th>\n",
       "      <th>YearRemodAdd</th>\n",
       "      <th>MasVnrArea</th>\n",
       "      <th>BsmtFinSF1</th>\n",
       "      <th>BsmtFinSF2</th>\n",
       "      <th>...</th>\n",
       "      <th>GarageArea</th>\n",
       "      <th>WoodDeckSF</th>\n",
       "      <th>OpenPorchSF</th>\n",
       "      <th>EnclosedPorch</th>\n",
       "      <th>3SsnPorch</th>\n",
       "      <th>ScreenPorch</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>60</td>\n",
       "      <td>65.0</td>\n",
       "      <td>8450</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2003</td>\n",
       "      <td>2003</td>\n",
       "      <td>196.0</td>\n",
       "      <td>706</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>548</td>\n",
       "      <td>0</td>\n",
       "      <td>61</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>20</td>\n",
       "      <td>80.0</td>\n",
       "      <td>9600</td>\n",
       "      <td>6</td>\n",
       "      <td>8</td>\n",
       "      <td>1976</td>\n",
       "      <td>1976</td>\n",
       "      <td>0.0</td>\n",
       "      <td>978</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>460</td>\n",
       "      <td>298</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>60</td>\n",
       "      <td>68.0</td>\n",
       "      <td>11250</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2001</td>\n",
       "      <td>2002</td>\n",
       "      <td>162.0</td>\n",
       "      <td>486</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>608</td>\n",
       "      <td>0</td>\n",
       "      <td>42</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>2008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>70</td>\n",
       "      <td>60.0</td>\n",
       "      <td>9550</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>1915</td>\n",
       "      <td>1970</td>\n",
       "      <td>0.0</td>\n",
       "      <td>216</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>642</td>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>272</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2006</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>60</td>\n",
       "      <td>84.0</td>\n",
       "      <td>14260</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>2000</td>\n",
       "      <td>2000</td>\n",
       "      <td>350.0</td>\n",
       "      <td>655</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>836</td>\n",
       "      <td>192</td>\n",
       "      <td>84</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>2008</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 36 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    MSSubClass  LotFrontage  LotArea  OverallQual  OverallCond  YearBuilt  \\\n",
       "Id                                                                          \n",
       "1           60         65.0     8450            7            5       2003   \n",
       "2           20         80.0     9600            6            8       1976   \n",
       "3           60         68.0    11250            7            5       2001   \n",
       "4           70         60.0     9550            7            5       1915   \n",
       "5           60         84.0    14260            8            5       2000   \n",
       "\n",
       "    YearRemodAdd  MasVnrArea  BsmtFinSF1  BsmtFinSF2  ...  GarageArea  \\\n",
       "Id                                                    ...               \n",
       "1           2003       196.0         706           0  ...         548   \n",
       "2           1976         0.0         978           0  ...         460   \n",
       "3           2002       162.0         486           0  ...         608   \n",
       "4           1970         0.0         216           0  ...         642   \n",
       "5           2000       350.0         655           0  ...         836   \n",
       "\n",
       "    WoodDeckSF  OpenPorchSF  EnclosedPorch  3SsnPorch  ScreenPorch  PoolArea  \\\n",
       "Id                                                                             \n",
       "1            0           61              0          0            0         0   \n",
       "2          298            0              0          0            0         0   \n",
       "3            0           42              0          0            0         0   \n",
       "4            0           35            272          0            0         0   \n",
       "5          192           84              0          0            0         0   \n",
       "\n",
       "    MiscVal  MoSold  YrSold  \n",
       "Id                           \n",
       "1         0       2    2008  \n",
       "2         0       5    2007  \n",
       "3         0       9    2008  \n",
       "4         0       2    2006  \n",
       "5         0      12    2008  \n",
       "\n",
       "[5 rows x 36 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7e1272b",
   "metadata": {
    "papermill": {
     "duration": 0.00808,
     "end_time": "2023-01-25T10:49:24.795575",
     "exception": false,
     "start_time": "2023-01-25T10:49:24.787495",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "So far, you've learned how to build pipelines with scikit-learn.  For instance, the pipeline below will use [`SimpleImputer()`](https://scikit-learn.org/stable/modules/generated/sklearn.impute.SimpleImputer.html) to replace missing values in the data, before using [`RandomForestRegressor()`](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html) to train a random forest model to make predictions.  We set the number of trees in the random forest model with the `n_estimators` parameter, and setting `random_state` ensures reproducibility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "89ca1021",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-01-25T10:49:24.814268Z",
     "iopub.status.busy": "2023-01-25T10:49:24.813602Z",
     "iopub.status.idle": "2023-01-25T10:49:25.032603Z",
     "shell.execute_reply": "2023-01-25T10:49:25.031385Z"
    },
    "papermill": {
     "duration": 0.232521,
     "end_time": "2023-01-25T10:49:25.036385",
     "exception": false,
     "start_time": "2023-01-25T10:49:24.803864",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.impute import SimpleImputer\n",
    "\n",
    "my_pipeline = Pipeline(steps=[\n",
    "    ('preprocessor', SimpleImputer()),\n",
    "    ('model', RandomForestRegressor(n_estimators=50, random_state=0))\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dfeb512",
   "metadata": {
    "papermill": {
     "duration": 0.006878,
     "end_time": "2023-01-25T10:49:25.051847",
     "exception": false,
     "start_time": "2023-01-25T10:49:25.044969",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "You have also learned how to use pipelines in cross-validation.  The code below uses the [`cross_val_score()`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_score.html) function to obtain the mean absolute error (MAE), averaged across five different folds.  Recall we set the number of folds with the `cv` parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "876e510e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-01-25T10:49:25.069185Z",
     "iopub.status.busy": "2023-01-25T10:49:25.068417Z",
     "iopub.status.idle": "2023-01-25T10:49:28.269775Z",
     "shell.execute_reply": "2023-01-25T10:49:28.268098Z"
    },
    "papermill": {
     "duration": 3.214627,
     "end_time": "2023-01-25T10:49:28.273222",
     "exception": false,
     "start_time": "2023-01-25T10:49:25.058595",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average MAE score: 18276.410356164386\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "# Multiply by -1 since sklearn calculates *negative* MAE\n",
    "scores = -1 * cross_val_score(my_pipeline, X, y,\n",
    "                              cv=5,\n",
    "                              scoring='neg_mean_absolute_error')\n",
    "\n",
    "print(\"Average MAE score:\", scores.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ad05937",
   "metadata": {
    "papermill": {
     "duration": 0.006314,
     "end_time": "2023-01-25T10:49:28.285858",
     "exception": false,
     "start_time": "2023-01-25T10:49:28.279544",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Step 1: Write a useful function\n",
    "\n",
    "In this exercise, you'll use cross-validation to select parameters for a machine learning model.\n",
    "\n",
    "Begin by writing a function `get_score()` that reports the average (over three cross-validation folds) MAE of a machine learning pipeline that uses:\n",
    "- the data in `X` and `y` to create folds,\n",
    "- `SimpleImputer()` (with all parameters left as default) to replace missing values, and\n",
    "- `RandomForestRegressor()` (with `random_state=0`) to fit a random forest model.\n",
    "\n",
    "The `n_estimators` parameter supplied to `get_score()` is used when setting the number of trees in the random forest model.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4103197f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-01-25T10:49:28.299608Z",
     "iopub.status.busy": "2023-01-25T10:49:28.299128Z",
     "iopub.status.idle": "2023-01-25T10:49:29.336892Z",
     "shell.execute_reply": "2023-01-25T10:49:29.335682Z"
    },
    "papermill": {
     "duration": 1.047929,
     "end_time": "2023-01-25T10:49:29.339811",
     "exception": false,
     "start_time": "2023-01-25T10:49:28.291882",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"1_GetScore\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def get_score(n_estimators):\n",
    "    \"\"\"Return the average MAE over 3 CV folds of random forest model.\n",
    "    \n",
    "    Keyword argument:\n",
    "    n_estimators -- the number of trees in the forest\n",
    "    \"\"\"\n",
    "    my_pipeline = Pipeline(steps=[\n",
    "    ('preprocessor', SimpleImputer()),\n",
    "    ('model', RandomForestRegressor(n_estimators, random_state=0))\n",
    "    ])\n",
    "    \n",
    "    scores = -1 * cross_val_score(my_pipeline, X, y,\n",
    "                              cv=3,\n",
    "                              scoring='neg_mean_absolute_error')\n",
    "    \n",
    "    # Replace this body with your own code\n",
    "    return scores.mean()\n",
    "\n",
    "# Check your answer\n",
    "step_1.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3479b416",
   "metadata": {
    "papermill": {
     "duration": 0.005401,
     "end_time": "2023-01-25T10:49:29.351145",
     "exception": false,
     "start_time": "2023-01-25T10:49:29.345744",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Step 2: Test different parameter values\n",
    "\n",
    "Now, you will use the function that you defined in Step 1 to evaluate the model performance corresponding to eight different values for the number of trees in the random forest: 50, 100, 150, ..., 300, 350, 400.\n",
    "\n",
    "Store your results in a Python dictionary `results`, where `results[i]` is the average MAE returned by `get_score(i)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "5dba6b1f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-01-25T10:49:29.364915Z",
     "iopub.status.busy": "2023-01-25T10:49:29.364430Z",
     "iopub.status.idle": "2023-01-25T10:50:26.349896Z",
     "shell.execute_reply": "2023-01-25T10:50:26.348765Z"
    },
    "papermill": {
     "duration": 57.000003,
     "end_time": "2023-01-25T10:50:26.356956",
     "exception": false,
     "start_time": "2023-01-25T10:49:29.356953",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"2_GetDict\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "results = {} # Your code here\n",
    "for i in range(50,450,50):\n",
    "    results[i] = get_score(i)\n",
    "\n",
    "# Check your answer\n",
    "step_2.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0341bc4f",
   "metadata": {
    "papermill": {
     "duration": 0.005437,
     "end_time": "2023-01-25T10:50:26.368594",
     "exception": false,
     "start_time": "2023-01-25T10:50:26.363157",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Use the next cell to visualize your results from Step 2.  Run the code without changes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "bb919d78",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-01-25T10:50:26.382975Z",
     "iopub.status.busy": "2023-01-25T10:50:26.381828Z",
     "iopub.status.idle": "2023-01-25T10:50:26.632385Z",
     "shell.execute_reply": "2023-01-25T10:50:26.631427Z"
    },
    "papermill": {
     "duration": 0.260466,
     "end_time": "2023-01-25T10:50:26.634794",
     "exception": false,
     "start_time": "2023-01-25T10:50:26.374328",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD5CAYAAAAndkJ4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAqVElEQVR4nO3deXyV9Zn38c+VhYQ1QLYiBBDZElEDRLSKohJatHW3U21tnY5PqVg7ik+fqdiO1mlr69KqrR2XGawWW+tSrNhqOwKO1A0bMAjIjiABJQlLgEAgy/X8ce4jMSQkHJKcc3K+79crr9zndy/nOrd4vrl/v3sxd0dERCQp2gWIiEhsUCCIiAigQBARkYACQUREAAWCiIgEFAgiIgJASmsLmNljwBeBcncfE7QVAg8D6UAdcL27v9NoncHA+8AP3f3eoG088DjQHXgJuNHd3czSgN8C44HtwJfdfWNrdWVlZfnQoUPb+jlFRARYvHhxpbtnNzev1UAg9CX+IKEv7bC7gTvc/WUzuyB4fU6j+fcBLzfZzkPANOBtQoEwNVjmWmCnuw83syuBu4Avt1bU0KFDKSkpaUP5IiISZmabWprXapeRuy8EdjRtBvoE0xnA1kZvdgmwAVjRqG0A0Mfd3/LQlXC/BS4JZl8MPBFMPwdMNjNrrS4REWlfbTlCaM5NwN/M7F5CoXIGgJn1BL4HTAG+22j5gUBZo9dlQVt43mYAd68zsyogE6iMsDYREYlApIPK04EZ7p4HzABmBe13APe5+94myzf3F7+3Yd6nN2I2zcxKzKykoqIigrJFRKQlkR4hXAPcGEw/C/x3MH0acIWZ3Q30BRrMrAb4IzCo0fqDONTNVAbkAWVmlkKoC6ppFxUA7v4o8ChAUVGRbsIkItKOIj1C2ApMCqbPA9YCuPtZ7j7U3YcC9wN3uvuD7v4RsMfMTg/GB74OvBCsP5dQwABcASxw3XFPRKTTteW006cInUGUZWZlwO3AN4EHgr/oawidPdSa6Rw67fRlDp2FNAuYbWbrCB0ZXHl0H0FERNqDxesf40VFRa7TTkVEjo6ZLXb3oubm6UrlGPdC6RY279gX7TJEJAEoEGLYM//YzI1/KOUXr6yJdikikgAUCDFq+ZYqfvDCcpIMXl1dTl19Q7RLEpEuToEQg3ZWH+S6JxeT1bMb/3HxGHbtq2Xxpp3RLktEujgFQoypb3BufLqU8t0H+M+rx3PJ2IF0S05i/qryaJcmIl2cAiHGPDB/LQvXVHD7RQUU5vWlV1oKp5+Qybz3t0W7NBHp4hQIMWTBqm38cv5arhg/iK9MGPxJe3F+Dhsqq1lf0fSOICIi7UeBECM+3L6Pm/5QSsGAPvz4kjE0vuHr5PxcAOav1FGCiHQcBUIM2H+wnm89uRgz4+Grx5Oemvyp+QP7dqdgQB/mva9xBBHpOAqEKHN3vv+nZaz6eDf3X1nI4MwezS5XnJ9DyaYd7Kw+2MkVikiiUCBE2e8WfcicJVu4cfIIzh2V0+JyxQW5NHjomgQRkY6gQIiidz/cyR0vruDcUdn863kjjrjsmOMyyO2TxjyNI4hIB1EgREnl3gNc/7slfCYjnfu+XEhS0pGfGpqUZJw3OpfXVldwoK6+k6oUkUSiQIiCuvoGvvP7d9lRfZCHvjqevj26tWm9KQU5VB+sZ9GGZp8fJCJyTBQIUXDv/6zhrQ3b+fElYxgzMKPN651xQhbdU5PVbSQiHUKB0Mn+uvxjHn5tPV85bTBfKso7qnXTU5OZOCKLee9vI16fYyEisUuB0Ik2VOzlu88u5ZRBGdx+YUFE25iSn8vWqhpWfrSnnasTkUSnQOgk1QfquO7JxXRLSeI/rx5PWkpy6ys149zROZihbiMRaXetBoKZPWZm5Wa2vFFboZm9bWalZlZiZhOC9glBW6mZLTWzSxutc5WZLTOz98zsr2aWFbSnmdnTZrbOzBaZ2dAO+JxR5e5874/vsa58L7+6aiwD+3aPeFvZvdMozOurQBCRdteWI4THgalN2u4G7nD3QuC24DXAcqAoaJ8KPGJmKWaWAjwAnOvuJwPvATcE61wL7HT34cB9wF0Rf5oY9Zs3NvLn9z7iu58fxZnDs455e8X5ubxXVsW23TXtUJ2ISEirgeDuC4Gm5zk60CeYzgC2Bsvuc/e6oD09WA7Agp+eFrprW5/wOsDFwBPB9HPAZGt8Z7c4984HO7jzpZV8riCX6ZNOaJdtTikI3+xOVy2LSPuJdAzhJuAeM9sM3AvMDM8ws9PMbAWwDLjO3evcvRaYHrRtBQqAWcEqA4HNAEGYVAGZEdYVU8p31/Dt3y8hr38P7v2nU2ivnBuR04u8/t3VbSQi7SrSQJgOzHD3PGAGh77ccfdF7n4icCow08zSzSw1WGcscByhLqNwiDT3LdnsOZVmNi0YsyipqKiIsPTOUVvfwLd/v4S9NXU8fPV4+qSnttu2zYzi/FzeWFfJvoN1ra8gItIGkQbCNcCcYPpZYELTBdx9JVANjAEKg7b1HjqB/hngjGDRMiAPIBhryODwLqrwNh919yJ3L8rOzo6w9M7xs5dX8Y+NO/nZ5Scx6jO92337U/JzOVDXwOtrK9t92yKSmCINhK3ApGD6PGAtgJkdH3ypY2ZDgFHARmALUGBm4W/xKcDKYHouoYABuAJY4HF+1dWLS7cy6/UP+OczhnJx4cAOeY9Tj+9P7/QUdRuJSLtJaW0BM3sKOAfIMrMy4Hbgm8ADwZd/DTAtWHwicIuZ1QINwPXuXhls5w5gYTBvE/DPwTqzgNlmto7QkcGV7fPRomPNtj1874/vUTSkH7dekN9h75OanMQ5o3JYsKqchgZv9eZ4IiKtaTUQ3P2qFmaNb2bZ2cDsFrbzMPBwM+01wJdaqyMe7Kmp5brZi+nRLYVff3Uc3VI69rq/4vwcXly6ldKyXYwb3K9D30tEuj5dqdxO3J3vPruUTTv28euvjCW3T3qHv+c5I3NITjLmva9uIxE5dgqEdvLIwg38bcU2Zp4/mtOGdc5Zsxk9UpkwtL+uRxCRdqFAaAdvrqvk7r+u4gsnD+Daicd36ntPzs9h9bY9fLh9X6e+r4h0PQqEY/RR1X6+89S7DMvuxd2Xn9xuF5+1VfiqZZ1tJCLHSoFwDA7U1TP9ySUcqGvg4avH0zOt1TH6djcksycjcnoxf5UCQUSOjQLhGPz4zysp3byLe644meE5vaJWx+T8XBZt2EHV/tqo1SAi8U+BEKE/Li5j9tub+NbZwzj/pAFRrWVKQQ51Dc5ra2L7dh4iEtsUCBFYsbWKW59fxunD+vP/Pj8q2uVQmNePzJ7dmK9xBBE5BgqEo1S1r5bpTy6hX49u/OqqcaQkR38XJicZ547O4dVV5dTWN0S7HBGJU9H/NosjDQ3OjGdK+ahqP7/+6jiye6dFu6RPFOfnsrumjn9sbPa+gCIirVIgHIUHX13HglXl3PbFAsYPia1bRZw1IotuKUm6SE1EIqZAaKPX1lRw37w1XDZ2IFefPiTa5RymZ1oKZ5yQybyV24jzm8WKSJQoENpg84593PiHdxmV25ufXHpSp1981lbF+bls2r6PdeV7o12KiMQhBUIramrrmf67xdQ3OI98bTzduyVHu6QWTc7PAWCeuo1EJAIKhFbc/sIKlm/Zzf1fLmRIZs9ol3NEAzK6M2ZgH93GQkQiokA4gj+88yFPl2zmO+cNZ3J+brTLaZPi/FyWfLiTyr0Hol2KiMQZBUILlm7exW0vrOCsEVncVDwy2uW0WXF+Lu7w6ip1G4nI0VEgNGNH9UGu/90Ssnun8csrx5IcR4+nPPG4PgzISFe3kYgctVYDwcweM7NyM1veqK3QzN42s1IzKzGzCUH7hKCt1MyWmtmljdbpZmaPmtkaM1tlZpcH7Wlm9rSZrTOzRWY2tAM+Z5vVNzg3/uFdKvYe4KGrx9GvZ7dolnPUzIzJ+TksXFNJTW19tMsRkTjSliOEx4GpTdruBu5w90LgtuA1wHKgKGifCjxiZuF7Qn8fKHf3kUAB8FrQfi2w092HA/cBd0X0SdrJ/fPW8Pe1lfzo4hM5eVDfaJYSseL8XPbX1vPWhu3RLkVE4kirgeDuC4Gm90NwoE8wnQFsDZbd5+51QXt6sFzYvwA/DZZrcPfKoP1i4Ilg+jlgskXpRP9572/jVwvWceWpeXz51MHRKKFdnD4skx7dkvWsZRE5KpGOIdwE3GNmm4F7gZnhGWZ2mpmtAJYB17l7nZn1DWb/yMyWmNmzZhY+bWcgsBkgCJMqoHMeStzIxspqZjxTykkDM/jhRSd29tu3q/TUZM4ekc38leW6allE2izSQJgOzHD3PGAGMCs8w90XufuJwKnATDNLB1KAQcAb7j4OeItQkAA0dzTQ7LeYmU0LxixKKira797/+w/Wc92Ti0lOMh66ehzpqbF78VlbFRfk8vHuGlZs3R3tUkQkTkQaCNcAc4LpZ4EJTRdw95VANTAG2A7sA55vtM64YLoMyAMIxhsyOLyLKrzNR929yN2LsrOzIyz9sG1y6/PLWL1tD7+8ciyD+vVol+1G27mjsjGDV9RtJCJtFGkgbAUmBdPnAWsBzOz48CCymQ0BRgEbPdRv8SJwTrDOZOD9YHouoYABuAJY4J3YzzH77U08/+4Wbi4eydkj2ydkYkFmrzTGD+6n009FpM1afSq8mT1F6Is8y8zKgNuBbwIPBF/+NcC0YPGJwC1mVgs0ANc3Gjz+HjDbzO4HKoBvBO2zgvZ1hI4MrmyHz9Umizft4D9efJ/Jo3P49rnDO+ttO01xQS4/e3kVH1XtZ0BG92iXIyIxzuJ10LGoqMhLSkoiXr9izwG++Ku/k56azNwbJpLRPbUdq4sN68r3UPyLhfzokjF8LQZv2S0inc/MFrt7UXPzEvJK5br6Br7z1BKq9tfy0FfHd8kwADghuxdDM3vo9FMRaZOEDIR7/raatzfs4KeXnUTBcX1aXyFOmRnF+bm8tX471QfqWl9BRBJawgXCy8s+4pGFG/j6Z4dw6dhB0S6nw03Oz+VgfQN/X9t+p+mKSNeUcIHQKz2FyaNz+MEXCqJdSqcoGtqPjO6pvPK+7n4qIkfW6llGXc1ZI7I5a0TXOb20NanJSZw7KptXV5dT3+BxdedWEelcCXeEkIgm5+eyo/og7364M9qliEgMUyAkgEmjsklJMl7RRWoicgQKhATQJz2V04dlMn+lxhFEpGUKhAQxOT+HdeV7+aCyOtqliEiMUiAkiOL80N3G56vbSERaoEBIEHn9ezD6M711szsRaZECIYFMzs/hHxt3smvfwWiXIiIxSIGQQIrzc6lvcP53ta5aFpHDKRASyCmD+pLVK03dRiLSLAVCAklKMiaPzuG11RUcrGuIdjkiEmMUCAmmuCCXPQfqeOeDZp9SKiIJTIGQYCYOzyItJUndRiJyGAVCguneLZmJw7OYt3Ib8fq0PBHpGK0Ggpk9ZmblZra8UVuhmb1tZqVmVmJmE4L2CUFbqZktNbNLm9ne3CbbSjOzp81snZktMrOh7fTZpAXFBbmU7dzP6m17ol2KiMSQthwhPA5MbdJ2N3CHuxcCtwWvAZYDRUH7VOARM/vkFttmdhmwt8m2rgV2uvtw4D7grqP7CHK0Jo/OAdC9jUTkU1oNBHdfCDQdgXQg/OzJDGBrsOw+dw8/qzE9WA4AM+sF3Az8uMm2LgaeCKafAyabmW7a34Fy+qRzyqAMXtGzlkWkkUjHEG4C7jGzzcC9wMzwDDM7zcxWAMuA6xoFxI+AnwP7mmxrILAZIFi2CsiMsC5po+L8XEo376J8T020SxGRGBFpIEwHZrh7HjADmBWe4e6L3P1E4FRgppmlm1khMNzdn29mW80dDTQ72mlm04Ixi5KKCl1teyyKC0I3u3t1lbqNRCQk0kC4BpgTTD8LTGi6gLuvBKqBMcBngfFmthF4HRhpZv8bLFoG5AEE4w0ZHN5FFd7mo+5e5O5F2dmJ8xjMjjD6M70Z2Le7nrUsIp+INBC2ApOC6fOAtQBmdnx4ENnMhgCjgI3u/pC7H+fuQ4GJwBp3PydYfy6hgAG4AljgOh+yw5kZxfk5vL6ugpra+miXIyIxoC2nnT4FvAWMMrMyM7sW+CbwczNbCtwJTAsWnwgsNbNS4HngenevbOUtZgGZZraO0KDzLRF9EjlqxQW51NQ28Ma61v4TiUgiSGltAXe/qoVZ45tZdjYwu5XtbSTUjRR+XQN8qbU6pP2ddnwmvdJSmLdyG5ODB+iISOLSlcoJrFtKEpNGZjNvZTkNDeqlE0l0CoQEV1yQQ8WeAyzbUhXtUkQkyhQICe6ckTkkGbrZnYgoEBJdv57dKBraX1cti4gCQaA4P4dVH++hbGfTi8hFJJEoEITi4Awj3exOJLEpEIRh2b0Ylt1T4wgiCU6BIEDoKOHtDdvZU1Mb7VJEJEoUCAKEAqG23lm4RlctiyQqBYIAMG5wX/r1SFW3kUgCUyAIACnJSZw7KodXV5dTV98Q7XJEJAoUCPKJ4oJcdu2rZfGmndEuRUSiQIEgnzh7ZDbdkpPUbSSSoBQI8oleaSmcNqy/rkcQSVAKBPmUKQW5bKisZn3F3miXIiKdTIEgnxJ+LsI83dtIJOEoEORTBvbtTv6APuo2EklACgQ5zJT8HEo27WBH9cFolyIinagtz1R+zMzKzWx5o7ZCM3vbzErNrMTMJgTtE4K2UjNbamaXBu09zOwvZrbKzFaY2c8abSvNzJ42s3VmtsjMhnbA55SjUFyQS4PDq6t0lCCSSNpyhPA4MLVJ293AHe5eCNwWvAZYDhQF7VOBR8ws/Nzme919NDAWONPMzg/arwV2uvtw4D7grsg+irSXMcdlkNM7jfmrNI4gkkhaDQR3XwjsaNoM9AmmM4CtwbL73L0uaE8Plgu3vxpMHwSWAIOC5S4GngimnwMmm5lF9GmkXSQlGZPzc3ltdQUH6uqjXY6IdJJIxxBuAu4xs83AvcDM8AwzO83MVgDLgOsaBUR4fl/gQmB+0DQQ2AwQLFsFZDb3pmY2LeiiKqmoqIiwdGmLKQU5VB+s5+0NTf8WEJGuKtJAmA7McPc8YAYwKzzD3Re5+4nAqcBMM0sPzwu6j54CfunuG8LNzWzfm3tTd3/U3YvcvSg7OzvC0qUtzjghi/TUJObrqmWRhBFpIFwDzAmmnwUmNF3A3VcC1cCYRs2PAmvd/f5GbWVAHnwSGBkc3kUlnSw9NZmzRmQz7/1tuDebzyLSxUQaCFuBScH0ecBaADM7PjyIbGZDgFHAxuD1jwl92d/UZFtzCQUMwBXAAtc3UEyYkp/L1qoaVn60J9qliEgnSGltATN7CjgHyDKzMuB24JvAA8GXfw0wLVh8InCLmdUCDcD17l5pZoOA7wOrgCXBmPGD7v7fhLqbZpvZOkJHBle24+eTY3Du6BzMYN7KbRQc16f1FUQkrlm8/jFeVFTkJSUl0S6jy7v0P9+gvsGZe8PEaJciIu3AzBa7e1Fz83SlshxRcX4u75VVsW13TbRLEZEOpkCQIyoObnanexuJdH0KBDmikbm9yOvfXQ/NEUkACgQ5IjOjOD+XN9ZVsu9gXesriEjcUiBIq4rzczlQ18DrayujXYqIdCAFgrRqwvH96Z2eom4jkS5OgSCtSk1O4pxROSxYVU5DQ3yepiwirVMgSJsU5+dQufcgpWW7ol2KiHQQBYK0yTkjc0hOMj1rWaQLUyBIm2T0SGXC0P66HkGkC1MgSJtNzs9h9bY9fLh9X7RLEZEOoECQNptSELpqWWcbiXRNCgRpsyGZPRmR00vPWhbpohQIclQm5+eyaMMOqvbXRrsUEWlnCgQ5KlMKcqhrcF5bo2dai3Q1CgQ5KoV5/cjs2U3PWhbpghQIclSSk4xzR+fw6qpyausbol2OiLQjBYIcteL8XHbX1PGPjTuiXYqItKNWA8HMHjOzcjNb3qit0MzeNrNSMysxswlB+4SgrdTMlprZpY3WGW9my8xsnZn90oIHK5tZmpk9HbQvMrOhHfA5pR2dNSKLbilJukhNpItpyxHC48DUJm13A3e4eyFwW/AaYDlQFLRPBR4xs5Rg3kPANGBE8BPe5rXATncfDtwH3BXJB5HO0zMthTNOyGTeym3E6zO5ReRwrQaCuy8EmvYNONAnmM4AtgbL7nP38FNU0oPlMLMBQB93f8tD3yC/BS4JlrsYeCKYfg6YHD56kNhVnJ/Lpu37WFe+N9qliEg7iXQM4SbgHjPbDNwLzAzPMLPTzGwFsAy4LgiIgUBZo/XLgjaC35sBgmWrgMzm3tTMpgVdVCUVFTrtMZom5+cAME/dRiJdRqSBMB2Y4e55wAxgVniGuy9y9xOBU4GZZpYONPcXf7iv4UjzPt3o/qi7F7l7UXZ2doSlS3sYkNGdMQP76DYWIl1IpIFwDTAnmH4WmNB0AXdfCVQDYwgdEQxqNHsQQTdTMC8PIBhvyODwLiqJQcX5uSz5cCeVew9EuxQRaQeRBsJWYFIwfR6wFsDMjg8PIpvZEGAUsNHdPwL2mNnpwfjA14EXgvXnEgoYgCuABa6RyrhQnJ+LO7y6St1GIl1BSmsLmNlTwDlAlpmVAbcD3wQeCL78awidPQQwEbjFzGqBBuB6dw8/mX06oTOWugMvBz8Q6m6abWbrCB0ZXHnsH0s6w4nH9WFARjrzVm7jS0V50S5HRI5Rq4Hg7le1MGt8M8vOBma3sJ0SQt1HTdtrgC+1VofEHjNjcn4Of1y8hZraetJTk6NdkogcA12pLMekOD+X/bX1vLVhe7RLEZFjpECQY3L6sEx6dEvWs5ZFugAFghyT9NRkzh6RzfyV5bpqWSTOKRDkmBUX5PLx7hpWbN0d7VJE5BgoEOSYnTsqGzN4Rd1GInFNgSDHLLNXGuMH99NVyyJxToEg7aK4IJcVW3fzUdX+aJciIhFSIEi7KNbN7kTingJB2sUJ2b0YmtlDp5+KxDEFgrQLM2NKQS5vrq9kWVlVtMsRkQgoEKTdXDfpBHJ6p/Ot2SW6A6pIHFIgSLvJ7JXGI18bz/bqg3z7d0uorW+IdkkichQUCNKuxgzM4KeXncSiD3Zw50sro12OiByFVu92KnK0Lhs3iGVbqvjNGxs5aWAGl40b1PpKIhJ1OkKQDnHrBfmcPqw/M+cs0yCzSJxQIEiHSE1O4tdfGUdWrzQNMovECQWCdBgNMovEFwWCdCgNMovEj1YDwcweM7NyM1veqK3QzN42s1IzKzGzCUH7FDNbbGbLgt/nNVrnqqD9PTP7q5llBe1pZva0ma0zs0VmNrQDPqdE0WXjBvGNM4fymzc2MmdJWbTLEZEWtOUI4XFgapO2u4E73L0QuC14DVAJXOjuJwHXEDxf2cxSgAeAc939ZOA94IZgnWuBne4+HLgPuCvSDyOxS4PMIrGv1UBw94XAjqbNQJ9gOgPYGiz7rrtvDdpXAOlmlgZY8NPTzCxYN7zcxcATwfRzwORgGelCNMgsEvsiHUO4CbjHzDYD9wIzm1nmcuBddz/g7rXAdGAZoSAoAGYFyw0ENgO4ex1QBWQ296ZmNi3ooiqpqKiIsHSJFg0yi8S2SANhOjDD3fOAGRz6cgfAzE4k1PXzreB1arDOWOA4Ql1G4RBp7mig2Yfzuvuj7l7k7kXZ2dkRli7RpEFmkdgVaSBcA8wJpp8FJoRnmNkg4Hng6+6+PmguBHD39R56EvszwBnBvDIgL1g3hVAXVNMuKulCNMgsEpsiDYStwKRg+jxgLYCZ9QX+Asx09zcaLb8FKDCz8J/1U4Dwn4dzCQUMwBXAgiA0pAvTILNI7GnLaadPAW8Bo8yszMyuBb4J/NzMlgJ3AtOCxW8AhgP/HpySWmpmOcFA8x3AQjN7j9ARw53BOrOATDNbB9wM3NJ+H09ilQaZRWKPxesf40VFRV5SUhLtMuQYLd9SxeUPvUlhXl+e/D+nkZqsayW7AndHJwvGJjNb7O5Fzc3T3U4lqsKDzDc/s5Q7X1rJ7ReeGO2S5BjUNzi/fWsj972yhuQkY1h2L07I7smw7F4Mywr9HpLZQ8EfoxQIEnW6XXbX8F7ZLm59fhnLt+zmrBFZDOrXgw0Ve1mwqoJnSg6dPJCSZAzu34NhjYLihJzQ7/49u+nIIooUCBITbr0gn5Uf7WbmnGWMyOnNSYMyol2StNHumlp+/rfV/PbtTWT3SuPBr4zlCycN+NQXe9X+Wj6orGZDxV7WV+xlQ0U1GyqqWbi2koN1h65HyeieyrDsnpyQ3SsUGFmhI4whmT3plqKjio6mMQSJGdv3HuCiB9/A3Zn7nYlk9UqLdklyBO7On9/7iP/48/tU7j3ANZ8dys2fG0mf9NQ2b6O+wdmycz/rK8MhEfq9vmIv5XsOnWiQZARHFYe6nsJdUVm9dFRxNI40hqBAkJiiQeb4sLGymn9/YTl/X1vJmIF9uPPSkzh5UN92fY89NeGjiupPjirWV+zlg8pqDjQ6quidnhIKiEZdT+GxivTU5HatqStQIEhcmbOkjJufWco3zhyqQeYYc6Cunkdf28CvXl1Ht+Qkvvu5kXzts0NJTuq8v9AbGpwtu/azofLQEcWGyr2sL6/m4901nyyXZDCwX/dQ91NW0AUVdEfl9E5L2KMKnWUkcUWDzLHpzfWV/OBPy9lQUc0XThrAbRcWkNsnvdPrSEoy8vr3IK9/DyaN/PQtbKoP1PFBZehIYn2jLqhFG3awv7b+k+V6paUEYxSho4nPZKSTZBa6C2eQE2ZgGI1zw4Jlmpt/aF1rNJ9Pgqe5bfOpdY+8bcLrAMOye3bIvlcgSEzSIHPsqNx7gDtfWsmcJVvI69+d33zjVM4dlRPtsprVMy2FMQMzGDPw0/9eGhqcj3fXNBrQ3suGymre+WAHfyrd2sLWYtePLxnD1acPafftqstIYtb2vQe48FevA2iQOQoaGpynSzbzs5dXse9gHdPOHsYN546ge7eu1S+/72Ad2/cexB08uK9maDo0cA7h6fBU4/kE8z3U1uR1WHjbh+YfxbbdD3uvYVmho5pIqMtI4lLodtlFXPHwm3z7d0s0yNyJVn28m+8/v5zFm3Yy4fj+/OSSMYzI7R3tsjpEj24p9Oivr0LQM5Ulxp00SLfL7kz7Dtbx05dW8oVfvs4HldXc+6VTeHra6V02DOTTFIsS8zTI3Dnmvb+N2+euYMuu/Vx5ah7fmzqafj27Rbss6UQKBIkLGmTuOFt37eeHc1fwP+9vY2RuL5697rOcOrR/tMuSKFCXkcSF8O2yM3t241uzS9iu22Ufs9r6Bv5r4QaKf/EaC9dWcMv5o/nLv56lMEhgCgSJG+FB5u3VB/n27/VM5mOx5MOdXPir1/nJSys5fVgmr8yYxHWTTtCgfYLTf32JK+FB5rc3aJA5ElX7arn1+WVc/tCb7NpXy8NXj2fWNUXk9e8R7dIkBmgMQeKOBpmPnrvzp9It/OQvK9lRfZB/OfN4ZkwZSa80fQXIIfrXIHFJg8xtt75iL//+p+W8uX47p+T15fFvTDjsSl4RaNszlR8zs3IzW96ordDM3g6emVxiZhOC9ilmttjMlgW/z2u0Tjcze9TM1pjZKjO7PGhPM7OnzWydmS0ys6Ed8Dmli0lNTuJBDTIfUU1tPb94ZQ3n3/93lm2p4keXjGHO9DMUBtKitowhPA5MbdJ2N3CHuxcCtwWvASqBC939JOAaYHajdb4PlLv7SKAAeC1ovxbY6e7DgfuAu47+Y0giytIgc4v+vraCqfcv5Jfz13L+SZ9h/v+dxNdOH9KpdyWV+NNqILj7QmBH02agTzCdAWwNln3X3cN3iloBpJtZ+AY0/wL8NFiuwd0rg/aLgSeC6eeAyZao96WVo6ZB5k8r31PDvz71Ll+b9Q5mxpPXnsYDV44lp3fn35VU4k+kYwg3AX8zs3sJhcoZzSxzOfCuux8ws75B24/M7BxgPXCDu28DBgKbAdy9zsyqgExCRxufYmbTgGkAgwcPjrB06Wo0yBx68tjv3/mQu/+6igO1Ddw4eQTTzzlBD4iRoxLpaafTgRnungfMAGY1nmlmJxLq+vlW0JQCDALecPdxwFvAveHFm9l+s7dgdfdH3b3I3Yuys7ObW0QS1K0X5HP6sP7MnLOMZWVV0S6nUy3fUsVlD73Jv/9pOScPyuCvN53FjCkjFQZy1CINhGuAOcH0s8CE8AwzGwQ8D3zd3dcHzduBfUF7eJ1xwXQZkBesm0KoC6ppF5XIESXiIPPeA3X86M/vc9GDr7Nl5z7u/3IhT157GsOye0W7NIlTkQbCVmBSMH0esBYg6Br6CzDT3d8IL+yhG3+/CJwTNE0G3g+m5xIKGIArgAUerw9pkKhKlEFmd+evyz+i+Oev8dgbH3DVhMHMv/kcLhk7MGEfCynto9UH5JjZU4S+yLOAbcDtwGrgAUJdQTXA9e6+2Mx+AMwkCIjA59y93MyGEDrrqC9QAXzD3T80s/SgfSyhI4Mr3X1Da4XrATnSkq78TObNO/Zx+9wVLFhVTv6APvzk0jGMG9wv2mVJHDnSA3L0xDTpku54cQW/eWMjv/inU7rEIPP+g/U8/uZGHpi/hiQzbp4ykn8+YygpuveQHCU9MU0Szq0X5PP+1vi8krmhwVlfsZd3N++idPMuSj/cxepte6hvcD5XkMsPLzqR4/p2j3aZ0gXpCEG6rMq9B7goeCbzi9+ZSGaMPpO5fE8NSzdXUbp5J6Wbd/He5ir2HKgDoHdaCqfk9eWUvAwmDs/msydkRrlaiXc6QpCElNX4mcy/X8Lsa6P/TOb9B+tZvrWK0g+Dv/4372LLrv0AJCcZoz/Tm4sKj6Mwry9jB/dlWFYvknR1sXQSBYJ0aeErmW9+Zil3vrSyUweZG3f9LA2+/Fd9HOr6ARjYtzuFg/vyjTOHUpjXlxOPy6B7N107INGjQJAur7OuZK7YcyD4q7/lrp/rJg2jMK8fhXl9ye4dm11YkrgUCJIQ2nuQWV0/0hVpUFkSRqSDzA0NzobKvbzb6Mu/ua6fsXl91fUjMU/XIYgElpVVccXDbzJ2cN8WB5nb0vVzSl6Gun4kLuksI5FA00Hmf/v86ENdP2Whc/7V9SOJSoEgCafxIPNv39qks35EAgoESUi3XpBPanIS3ZKTOCXo+1fXjyQ6BYIkpNTkJG69ID/aZYjEFN0ZS0REAAWCiIgEFAgiIgIoEEREJKBAEBERQIEgIiIBBYKIiAAKBBERCcTtze3MrALYFOHqWUBlO5bT0eKp3niqFeKr3niqFeKr3niqFY6t3iHunt3cjLgNhGNhZiUt3e0vFsVTvfFUK8RXvfFUK8RXvfFUK3RcveoyEhERQIEgIiKBRA2ER6NdwFGKp3rjqVaIr3rjqVaIr3rjqVbooHoTcgxBREQOl6hHCCIi0kRCBIKZbTSzZWZWamYlQVt/M3vFzNYGv/tFqbbHzKzczJY3amuxNjObaWbrzGy1mX0+Rur9oZltCfZvqZldEAv1mlmemb1qZivNbIWZ3Ri0x9z+PUKtsbpv083sHTNbGtR7R9Aei/u2pVpjct82qiHZzN41sz8Hrzt+37p7l/8BNgJZTdruBm4Jpm8B7opSbWcD44DlrdUGFABLgTTgeGA9kBwD9f4Q+G4zy0a1XmAAMC6Y7g2sCWqKuf17hFpjdd8a0CuYTgUWAafH6L5tqdaY3LeN6rgZ+D3w5+B1h+/bhDhCaMHFwBPB9BPAJdEowt0XAjuaNLdU28XAH9z9gLt/AKwDJnRGnWEt1NuSqNbr7h+5+5Jgeg+wEhhIDO7fI9TakmjvW3f3vcHL1ODHic1921KtLYn6/2dmNgj4AvDfTerq0H2bKIHgwP+Y2WIzmxa05br7RxD6nxHIiVp1h2uptoHA5kbLlXHkL43OdIOZvRd0KYUPZWOmXjMbCowl9NdhTO/fJrVCjO7boEujFCgHXnH3mN23LdQKMbpvgfuBfwMaGrV1+L5NlEA4093HAecD3zazs6NdUISsmbZYOE3sIeAEoBD4CPh50B4T9ZpZL+CPwE3uvvtIizbT1qn1NlNrzO5bd69390JgEDDBzMYcYfGo1ttCrTG5b83si0C5uy9u6yrNtEVUb0IEgrtvDX6XA88TOpzaZmYDAILf5dGr8DAt1VYG5DVabhCwtZNrO4y7bwv+h2sA/otDh6tRr9fMUgl9wf7O3ecEzTG5f5urNZb3bZi77wL+F5hKjO7bsMa1xvC+PRO4yMw2An8AzjOzJ+mEfdvlA8HMeppZ7/A08DlgOTAXuCZY7BrghehU2KyWapsLXGlmaWZ2PDACeCcK9X1K+B9p4FJC+xeiXK+ZGTALWOnuv2g0K+b2b0u1xvC+zTazvsF0d6AYWEVs7ttma43VfevuM919kLsPBa4EFrj71XTGvu3skfPO/gGGERqBXwqsAL4ftGcC84G1we/+UarvKUKHq7WEkv7aI9UGfJ/QWQSrgfNjpN7ZwDLgveAf54BYqBeYSOjQ+T2gNPi5IBb37xFqjdV9ezLwblDXcuC2oD0W921Ltcbkvm1S+zkcOsuow/etrlQWEREgAbqMRESkbRQIIiICKBBERCSgQBAREUCBICIiAQWCiIgACgQREQkoEEREBID/D4E+2QSSvMjbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "plt.plot(list(results.keys()), list(results.values()))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a73bb74a",
   "metadata": {
    "papermill": {
     "duration": 0.005971,
     "end_time": "2023-01-25T10:50:26.647528",
     "exception": false,
     "start_time": "2023-01-25T10:50:26.641557",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Step 3: Find the best parameter value\n",
    "\n",
    "Given the results, which value for `n_estimators` seems best for the random forest model?  Use your answer to set the value of `n_estimators_best`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7fc09753",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-01-25T10:50:26.663050Z",
     "iopub.status.busy": "2023-01-25T10:50:26.661542Z",
     "iopub.status.idle": "2023-01-25T10:50:26.672340Z",
     "shell.execute_reply": "2023-01-25T10:50:26.670979Z"
    },
    "papermill": {
     "duration": 0.021183,
     "end_time": "2023-01-25T10:50:26.674883",
     "exception": false,
     "start_time": "2023-01-25T10:50:26.653700",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.3333333333333333, \"interactionType\": 1, \"questionType\": 2, \"questionId\": \"3_BestEst\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Correct</span>"
      ],
      "text/plain": [
       "Correct"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_estimators_best = 200\n",
    "\n",
    "# Check your answer\n",
    "step_3.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c579a02b",
   "metadata": {
    "papermill": {
     "duration": 0.006232,
     "end_time": "2023-01-25T10:50:26.688072",
     "exception": false,
     "start_time": "2023-01-25T10:50:26.681840",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "In this exercise, you have explored one method for choosing appropriate parameters in a machine learning model.  \n",
    "\n",
    "If you'd like to learn more about [hyperparameter optimization](https://en.wikipedia.org/wiki/Hyperparameter_optimization), you're encouraged to start with **grid search**, which is a straightforward method for determining the best _combination_ of parameters for a machine learning model.  Thankfully, scikit-learn also contains a built-in function [`GridSearchCV()`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html) that can make your grid search code very efficient!\n",
    "\n",
    "# Keep going\n",
    "\n",
    "Continue to learn about **[gradient boosting](https://www.kaggle.com/alexisbcook/xgboost)**, a powerful technique that achieves state-of-the-art results on a variety of datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcd2f841",
   "metadata": {
    "papermill": {
     "duration": 0.006007,
     "end_time": "2023-01-25T10:50:26.700479",
     "exception": false,
     "start_time": "2023-01-25T10:50:26.694472",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "*Have questions or comments? Visit the [course discussion forum](https://www.kaggle.com/learn/intermediate-machine-learning/discussion) to chat with other learners.*"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 73.759685,
   "end_time": "2023-01-25T10:50:27.832452",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2023-01-25T10:49:14.072767",
   "version": "2.3.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}