chap03/AUC_ROC.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "AUC_ROC.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "meXJVUb3dDT1"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/oreilly-japan/ml-at-work/blob/master/chap03/AUC_ROC.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "source": [
        "![表紙](https://www.oreilly.co.jp/books/images/picture978-4-87311-947-2.gif)\n",
        "\n",
        "このノートブックはオライリー・ジャパンより発行の書籍[『仕事ではじめる機械学習 第2版』](https://www.oreilly.co.jp/books/9784873119472/)のサンプルコードです。コードの解説等は書籍をご参照ください。なお、このコードを動作させた結果について、著者およびオライリー・ジャパンは一切の責任を負いません。"
      ],
      "cell_type": "markdown",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_9xbc_uqc5a0"
      },
      "source": [
        "# 3章: 学習結果を評価しよう\n",
        "\n",
        "このノートブックでは、評価指標の一つであるAUCとROCについて説明します。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0vCqfdg4dv7T"
      },
      "source": [
        "元となるデータを設定します。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_D9xfqBjYgww",
        "outputId": "e3440fa0-c8fd-46ad-87e7-f61211fecfc6"
      },
      "source": [
        "from sklearn.metrics import roc_curve\r\n",
        "\r\n",
        "y_pass = [0, 0, 0, 1, 0, 0, 1, 0, 1, 1]\r\n",
        "y_score = [0.1, 0.2, 0.4, 0.45, 0.5, 0.65, 0.7, 0.8, 0.85, 0.95]\r\n",
        "\r\n",
        "fpr, tpr, thresholds = roc_curve(y_pass, y_score)\r\n",
        "\r\n",
        "print(fpr)\r\n",
        "print(tpr)\r\n",
        "print(thresholds)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[0.         0.         0.         0.16666667 0.16666667 0.5\n",
            " 0.5        1.        ]\n",
            "[0.   0.25 0.5  0.5  0.75 0.75 1.   1.  ]\n",
            "[1.95 0.95 0.85 0.8  0.7  0.5  0.45 0.1 ]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-ONZK7h2l9yY"
      },
      "source": [
        "matplotlibを使いROC曲線を描画してみましょう。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "id": "y09P2LMxahu4",
        "outputId": "104461c6-024d-4e46-c07c-b917b648a12c"
      },
      "source": [
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "plt.plot(fpr, tpr, marker='o')\r\n",
        "plt.xlabel('FPR: False Positive Rate')\r\n",
        "plt.ylabel('TPR: True Positive Rate')\r\n",
        "plt.grid()\r\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAdq0lEQVR4nO3de5hcVZnv8e+PEKQ1gTgGM9LABDFGI6CNLYg5aEfQIMMhPeogODiDovGGOqKNRDzooDPEyYijDqIZxdsjJJgTc3I00j5zQotHwyWZhoSA7cQIIR08KlJooDGh854/9m6olN3Vu7trV6V6/z7PU0/2Xvv2rqpKvb32ZS1FBGZmVlwHNToAMzNrLCcCM7OCcyIwMys4JwIzs4JzIjAzK7iDGx3AWM2cOTNmz549rm0fffRRnvGMZ9Q2oAOc61wMrnMxTKTOmzZt+m1EHDHcsqZLBLNnz2bjxo3j2ranp4eOjo7aBnSAc52LwXUuhonUWdL9Iy3zqSEzs4JzIjAzKzgnAjOzgnMiMDMrOCcCM7OCy+2uIUnXAWcDv46I44dZLuBzwFnAY8CFEfGfecVjlpc1vf0s6+5jV2mAI2e00LVwLp1trY0OyyaRoe9Yf2mA1lvX1/w7lmeL4OvAmVWWvw6Yk74WA9fmGItZLtb09rNk9Rb6SwME0F8aYMnqLazp7W90aDZJlH/HIJ/vWG4tgoi4RdLsKqssAr4ZST/Yt0qaIek5EfFgXjGZ1dqy7j4G9g7uVzawd5BLV23mhtt31C2OUmmAa/s21O14B4Ki1Ll3R4k9g/v2KxvYO8iy7r6atQoa+UBZK/BA2fzOtOxPEoGkxSStBmbNmkVPT8+4Drh79+5xb9usXOd8Df2VVmnP4D5KpVJdYgAYHBys6/EOBEWpc2USGNJfGqjZ97wpniyOiOXAcoD29vYY75N1fhKxGOpZ59Zb1w+bDFpntND9kVfXJQbw5zyZzV868nesVvVv5F1D/cDRZfNHpWVmTaNr4Vxapk7Zr6xl6hS6Fs5tUEQ22dTjO9bIRLAW+FslXg484usD1mw621q56vUncMiU5L9S64wWrnr9Cb5ryGpm6DvWOqMFyOc7luftozcAHcBMSTuBjwNTASLiS8A6kltHt5HcPvrWvGIxy1NnW+uTF4ZXvvPUBkdjk1FnWyudba25nQ7L866h80dZHsB78zq+mZll4yeLzcwKzonAzKzgnAjMzArOicDMrOCcCMzMCs6JwMys4JwIzMwKzonAzKzgnAjMzArOicDMrOCcCMzMCs6JwMys4JwIzMwKzonAzKzgnAjMzArOicDMrOCcCMzMCs6JwMys4JwIzMwKzonAzKzgnAjMzArOicDMrOCcCMzMCs6JwMys4JwIzMwKzonAzKzgnAjMzArOicDMrOCcCMzMCs6JwMys4JwIzMwKLtdEIOlMSX2Stkm6bJjlx0i6WVKvpM2SzsozHjMz+1O5JQJJU4BrgNcB84DzJc2rWO1jwI0R0QacB3wxr3jMzGx4ebYITga2RcT2iNgDrAAWVawTwGHp9OHArhzjMTOzYSgi8tmx9EbgzIh4ezr/FuCUiLi4bJ3nAD8Engk8AzgjIjYNs6/FwGKAWbNmvXTFihXjimn37t1MmzZtXNs2K9e5Pq66bQCAJae01PW4Q/w5F8NE6rxgwYJNEdE+3LKDJxTVxJ0PfD0iPiPpVOBbko6PiH3lK0XEcmA5QHt7e3R0dIzrYD09PYx322blOtfHtX0bAOjoOLWuxx3iz7kY8qpznqeG+oGjy+aPSsvKXQTcCBARG4BDgZk5xmRmZhXyTAR3AHMkHSvpEJKLwWsr1tkBnA4g6YUkieA3OcZkZmYVMicCSU8fy44j4gngYqAbuJfk7qCtkq6UdE662oeAd0i6C7gBuDDyumhhZmbDGvUagaRXAF8BpgHHSHox8M6IeM9o20bEOmBdRdkVZdP3APPHGrSZmdVOlhbBZ4GFwEMAEXEX8Mo8gzIzs/rJdNdQRDwgqbxoMJ9wbDRrevtZ1t3HrtIAR85ooWvhXDrbWhsdVqGt6e2nd0eJPYP7mL90vT8TazpZEsED6emhkDQV+ADJOX+rszW9/SxZvYWBvUke7i8NsGT1FgD/8DTI0GeyZzC549mfiTWjLIngXcDngFaS2z9/CIx6fcBqb1l335NJYMjA3kEuXbWZG27fMew2pdLAk/e4F0U96zzUEig3sHeQZd19TgTWNLIkgrkR8TflBZLmAz/JJyQbya7SwLDllT9EVj8jvfcjfVZmB6IsieALwEkZyixnR85ooX+YH5jWGS2sfOfwT7QmTyI25mnXRqlnnecvXT/sZ3LkjMZ0NWE2HiPeNSTpVEkfAo6QdEnZ6xPAlLpFaE/qWjiXlqn7v/UtU6fQtXBugyIyfyY2GVRrERxC8uzAwcD0svLfA2/MMygb3tA550tXbWbP4D5afddQww29976Ty5rZiIkgIn4E/EjS1yPi/jrGZFV0trU+eWF4pNNBVl+dba3+4bemluUawWOSlgEvIukLCICIeHVuUZmZWd1kebL428DPgGOBfwDuI+lQzszMJoEsieBZEfFVYG9E/Cgi3ga4NWBmNklkOTW0N/33QUl/STKc5J/lF5KZmdVTlkTwKUmHk3QZ/QWSMYb/PteozMysbkZNBBHxvXTyEWABPPlksZmZTQIjJgJJU4BzSfoYuiki7pZ0NvBRoAVoq0+IZmaWp2otgq+SjDl8O/B5SbuAduCyiFhTj+DMzCx/1RJBO3BiROyTdCjwK+C4iHioPqGZmVk9VLt9dE9E7AOIiMeB7U4CZmaTT7UWwQskbU6nBRyXzguIiDgx9+jMzCx31RLBC+sWhZmZNUy1Tufc0ZyZWQFk6WLCzMwmMScCM7OCy5QIJLVI8pBLZmaT0KiJQNJ/B+4EbkrnXyJpbd6BmZlZfWRpEXwCOBkoAUTEnSRjE5iZ2SSQJRHsjYhHKsoij2DMzKz+snRDvVXSm4EpkuYA7wd+mm9YZmZWL1laBO8jGa/4j8D1JN1RezwCM7NJIksieEFEXB4RL0tfH0v7HhqVpDMl9UnaJumyEdY5V9I9krZKun5M0ZuZ2YRlOTX0GUl/DqwCVkbE3Vl2nI5ncA3wGmAncIektRFxT9k6c4AlwPyIeFjSs8dcAzMzm5BRWwQRsYBkZLLfAF+WtEXSxzLs+2RgW0Rsj4g9wApgUcU67wCuiYiH02P9ekzRm5nZhCki+w1Akk4ALgXeFBGHjLLuG4EzI+Lt6fxbgFMi4uKyddYAPwfmA1OAT0TETcPsazGwGGDWrFkvXbFiReaYy+3evZtp06aNa9sDyVW3DQCw5JSWUdedLHUeC9e5GFznsVmwYMGmiGgfbtmop4YkvRB4E/AG4CFgJclA9rVwMDAH6ACOAm6RdEJElMpXiojlwHKA9vb26OjoGNfBenp6GO+2B5Jr+zYA0NFx6qjrTpY6j4XrXAyuc+1kuUZwHcmP/8KI2DWGffeTDHU55Ki0rNxO4LaI2Av8UtLPSRLDHWM4jpmZTUCWawSnRsS/jjEJQPJjPkfSsZIOAc4DKrumWEPSGkDSTOD5wPYxHsfMzCZgxBaBpBsj4lxJW9j/SeJMI5RFxBOSLga6Sc7/XxcRWyVdCWyMiLXpstdKugcYBLo8HKaZWX1VOzX0gfTfs8e784hYB6yrKLuibDqAS9KXmZk1wIinhiLiwXTyPRFxf/kLeE99wjMzs7xlebL4NcOUva7WgZiZWWNUu0bwbpK//J8raXPZounAT/IOzMzM6qPaNYLrgR8AVwHl/QT9ISJ+l2tUZmZWN9USQUTEfZLeW7lA0p85GZiZTQ6jtQjOBjaR3D6qsmUBPDfHuMzMrE5GTAQRcXb6r4elNDObxLIMXj9f0jPS6QskXS3pmPxDMzOzeshy++i1wGOSXkzS2dwvgG/lGpWZmdVNlkTwRPoE8CLg3yLiGpJbSM3MbBLI0vvoHyQtAd4CnCbpIGBqvmGZmVm9ZGkRvIlk4Pq3RcSvSLqTXpZrVDW2pref+UvXc+FNjzJ/6XrW9Fb2ht081vT207ujxG2//F3T18XMDgxZuqH+FfBt4HBJZwOPR8Q3c4+sRtb09rNk9Rb6S8moXv2lAZas3tKUP6BDddkzuA9o7rqY2YEjywhl55K0AHpIniX4gqSuiFiVc2w1say7j4G9g/uVDewd5NJVm7nh9h0Nimp8eneUnkwCQwb2DrKsu4/OttYGRWVmzS7LNYLLgZcNDSwv6QjgP4CmSAS70pZApcof1GYwUswj1dHMLIssieCgoSSQeohs1xYOCEfOaHnytFC51hktrHzn6GP+HkjmL10/bF2OnDH6IPZmZiPJ8oN+k6RuSRdKuhD4PhWDzRzIuhbOpWXqlP3KWqZOoWvh3AZFNH6TqS5mduAYtUUQEV2SXg/8t7RoeUR8N9+wamfo3PmlqzazZ3AfrTNa6Fo4tynPqQ/FvKy7j12lAY5s4rqY2YGj2ngEc4B/AY4DtgAfjoimvD2ls62VG27fQalUovsjr250OBPS2dbqH34zq6lqp4auA74HvIGkB9Iv1CUiMzOrq2qnhqZHxL+n032S/rMeAZmZWX1VSwSHSmrjqXEIWsrnI8KJwcxsEqiWCB4Eri6b/1XZfADNfbLdzMyA6gPTLKhnIGZm1hhN82CYmZnlw4nAzKzgnAjMzAouy5jFSscqviKdP0bSyfmHZmZm9ZClRfBF4FTg/HT+D8A1uUVkZmZ1laX30VMi4iRJvQAR8bCkQ3KOy8zM6iRLi2CvpCkkzw4MjUfQfJ35m5nZsLIkgs8D3wWeLekfgf8L/FOWnUs6U1KfpG2SLquy3hskhaT2TFGbmVnNZOmG+tuSNgGnk3Qv0RkR9462XdqKuAZ4DbATuEPS2oi4p2K96cAHgNvGEb+ZmU1QlruGjgEeA/43sBZ4NC0bzcnAtojYHhF7gBXAomHW+yTwaeDxzFGbmVnNZLlY/H2S6wMCDgWOBfqAF42yXSvwQNn8TuCU8hUknQQcHRHfl9Q10o4kLQYWA8yaNYuenp4MYe+vVBpgcHBwXNs2s927d7vOBeA6F0Nedc5yauiE8vn0x/s9Ez2wpINIOrG7MEMMy4HlAO3t7dHR0THm413bt4FSqcR4tm1mPT09rnMBuM7FkFedx/xkcdr99Cmjrgj9wNFl80elZUOmA8cDPZLuA14OrPUFYzOz+hq1RSDpkrLZg4CTgF0Z9n0HMEfSsSQJ4DzgzUMLI+IRYGbZcXpIhsPcmClyMzOriSwtgullr6eRXDMY7qLvfiLiCeBioBu4F7gxIrZKulLSOeMP2czMaqlqiyC9BXR6RHx4PDuPiHXAuoqyK0ZYt2M8xzAzs4mp2iKIiEFgfp1iMTOzBhixRSDp4PT0zp2S1gLfAR4dWh4Rq+sQn5mZ5azaqaHbSS4MHwo8xP5jFAfgRGBmNglUSwQCiIi31ikWMzNrgGqJ4IiKW0f3ExFX5xCPmZnVWbVEMAWYRtoyMDOzyalaIngwIq6sWyRmZtYQ1W4fdUvAzKwAqiWC0+sWhZmZNcyIiSAiflfPQMzMrDHG3PuomZlNLk4EZmYFlykRSFpebd7MzJpX1hbBl0eZNzOzJpUpEUTEpqHpdIjJF+QWkZmZ1dWIiUDSYZKWSPo3Sa9V4n3AduDc+oVoZmZ5qvZk8beAh4ENwNuBj5I8ZNYZEXfWITYzM6uDaonguRFxAoCkrwAPAsdExON1iczMzOqi2jWCvUMT6UhlO50EzMwmn2otghdL+j1P9TnUUjYfEXFY7tGZmVnuqnUxMSUiDouI6enr4LL5pkoCa3r76d1Rou/hfcxfup41vf2NDsnM7IBRbcziQ4F3Ac8DNgPXpWMYN5U1vf0sWb2FPYP7AOgvDbBk9RYAOttaGxmamdkBodo1gm8A7cAW4CzgM3WJqMaWdfcxsHdwv7KBvYMs6+5rUERmZgeWatcI5pXdNfRVksHsm86u0sCYys3MiibrXUNNd0poyJEzWsZUbmZWNNUSwUsk/T59/QE4cWg6vXuoKXQtnEvL1Cn7lbVMnULXwrkNisjM7MBS7dTQXRHRVrdIcjJ0QfjSVZvZM7iP1hktdC2c6wvFZmapaokg6hZFzjrbWrnh9h2USiW6P/LqRodjZnZAqZYIni3pkpEWRsTVOcRjZmZ1Vi0RTAGm8dSTxWZmNglVSwQPRsSVE9m5pDOBz5Ekla9ExNKK5ZeQ9Gz6BPAb4G0Rcf9EjmlmZmNT7a6hCbUEJE0BrgFeB8wDzpc0r2K1XqA9Ik4EVgH/PJFjmpnZ2FVLBKdPcN8nA9siYntE7AFWAIvKV4iImyPisXT2VuCoCR7TzMzGaMRTQxHxuwnuuxV4oGx+J3BKlfUvAn4w3AJJi4HFALNmzaKnp2fMwZRKAwwODo5r22a2e/du17kAXOdiyKvO1a4R1I2kC0j6NXrVcMsjYjmwHKC9vT06OjrGfIxr+zZQKpUYz7bNrKenx3UuANe5GPKqc56JoB84umz+qLRsP5LOAC4HXhURf8wxHjMzG0a1awQTdQcwR9Kxkg4BzgPWlq8gqQ34MnBORPw6x1jMzGwEuSWCtKO6i4Fu4F7gxojYKulKSeekqy0jeVbhO5LulLR2hN2ZmVlOcr1GEBHrgHUVZVeUTZ+R5/HNzGx0eZ4aMjOzJuBEYGZWcE4EZmYF50RgZlZwTgRmZgXnRGBmVnBOBGZmBedEYGZWcE4EZmYF50RgZlZwTgRmZgXnRGBmVnBOBGZmBedEYGZWcE4EZmYF50RgZlZwTgRmZgXnRGBmVnBOBGZmBedEYGZWcE4EZmYF50RgZlZwTgRmZgXnRGBmVnBOBGZmBedEYGZWcE4EZmYF50RgZlZwTgRmZgXnRGBmVnBOBGZmBZdrIpB0pqQ+SdskXTbM8qdJWpkuv03S7DziWNPbT++OEn0P72P+0vWs6e3P4zBmZk0pt0QgaQpwDfA6YB5wvqR5FatdBDwcEc8DPgt8utZxrOntZ8nqLewZ3AdAf2mAJau3OBmYmaXybBGcDGyLiO0RsQdYASyqWGcR8I10ehVwuiTVMohl3X0M7B3cr2xg7yDLuvtqeRgzs6Z1cI77bgUeKJvfCZwy0joR8YSkR4BnAb8tX0nSYmAxwKxZs+jp6ckcRH9pYMTyseynWe3evbsQ9SznOheD61w7eSaCmomI5cBygPb29ujo6Mi8beut64dNBq0zWhjLfppVT09PIepZznUuBte5dvI8NdQPHF02f1RaNuw6kg4GDgceqmUQXQvn0jJ1yn5lLVOn0LVwbi0PY2bWtPJMBHcAcyQdK+kQ4DxgbcU6a4G/S6ffCKyPiKhlEJ1trVz1+hNondECJC2Bq15/Ap1trbU8jJlZ08rt1FB6zv9ioBuYAlwXEVslXQlsjIi1wFeBb0naBvyOJFnUXGdbK51trYVsSpqZjSbXawQRsQ5YV1F2Rdn048Bf5xmDmZlV5yeLzcwKzonAzKzgnAjMzArOicDMrOBU47s1cyfpN8D949x8JhVPLReA61wMrnMxTKTOfxERRwy3oOkSwURI2hgR7Y2Oo55c52JwnYshrzr71JCZWcE5EZiZFVzREsHyRgfQAK5zMbjOxZBLnQt1jcDMzP5U0VoEZmZWwYnAzKzgJmUikHSmpD5J2yRdNszyp0lamS6/TdLs+kdZWxnqfImkeyRtlvR/JP1FI+KspdHqXLbeGySFpKa/1TBLnSWdm37WWyVdX+8Yay3Dd/sYSTdL6k2/32c1Is5akXSdpF9LunuE5ZL0+fT92CzppAkfNCIm1Yuky+tfAM8FDgHuAuZVrPMe4Evp9HnAykbHXYc6LwCenk6/uwh1TtebDtwC3Aq0NzruOnzOc4Be4Jnp/LMbHXcd6rwceHc6PQ+4r9FxT7DOrwROAu4eYflZwA8AAS8HbpvoMSdji+BkYFtEbI+IPcAKYFHFOouAb6TTq4DTJamOMdbaqHWOiJsj4rF09laSEeOaWZbPGeCTwKeBx+sZXE6y1PkdwDUR8TBARPy6zjHWWpY6B3BYOn04sKuO8dVcRNxCMj7LSBYB34zErcAMSc+ZyDEnYyJoBR4om9+Zlg27TkQ8ATwCPKsu0eUjS53LXUTyF0UzG7XOaZP56Ij4fj0Dy1GWz/n5wPMl/UTSrZLOrFt0+chS508AF0jaSTL+yfvqE1rDjPX/+6iaYvB6qx1JFwDtwKsaHUueJB0EXA1c2OBQ6u1gktNDHSStvlsknRARpYZGla/zga9HxGcknUoy6uHxEbGv0YE1i8nYIugHji6bPyotG3YdSQeTNCcfqkt0+chSZySdAVwOnBMRf6xTbHkZrc7TgeOBHkn3kZxLXdvkF4yzfM47gbURsTcifgn8nCQxNKssdb4IuBEgIjYAh5J0zjZZZfr/PhaTMRHcAcyRdKykQ0guBq+tWGct8Hfp9BuB9ZFehWlSo9ZZUhvwZZIk0OznjWGUOkfEIxExMyJmR8Rskusi50TExsaEWxNZvttrSFoDSJpJcqpoez2DrLEsdd4BnA4g6YUkieA3dY2yvtYCf5vePfRy4JGIeHAiO5x0p4Yi4glJFwPdJHccXBcRWyVdCWyMiLXAV0maj9tILsqc17iIJy5jnZcB04DvpNfFd0TEOQ0LeoIy1nlSyVjnbuC1ku4BBoGuiGja1m7GOn8I+HdJHyS5cHxhM/9hJ+kGkmQ+M73u8XFgKkBEfInkOshZwDbgMeCtEz5mE79fZmZWA5Px1JCZmY2BE4GZWcE5EZiZFZwTgZlZwTkRmJkVnBOB1YWkQUl3lr1mS+qQ9Eg6f6+kj6frlpf/TNK/ZNh/+TZ3SvqPUdb9Xg3qdJ+kLWkPkD+U9Ofj2MdP039nS3pzWXm7pM/XIMbxvJedkuZN9NjWPJwIrF4GIuIlZa/70vIfR8RLSLq9uKCsS92h8jbgbEnzMxzjx2X7P6P2VRjWgog4EdgIfHSsG0fEK9LJ2cCby8o3RsT7axLh2N/LTpJePK0gnAjsgBARjwKbgOdVlA8AdzKOTrUknSxpQ9pP/U8lzR1mnVeVtSJ6JU1Py7sk3ZH+tf8PGQ53C/A8SYdK+lraUuiVtCDd34sk3Z4eZ7OkOWn57nT7pcBp6fIPDrVaJB2UtjxmlMX8X5JmSTpC0v9M47xjtB/4yvdS0jvS7e5K9/N0Sa8AzgGWpbEcl75ukrRJ0o8lvSDD+2FNxInA6qWl7Af3u5ULJT2LpD+grRXlzyTpK+eWdP5dkt41wjFOKzvG5cDPgNMiog24AvinYbb5MPDe9C/m04ABSa9Nj3ky8BLgpZJeOUr9zga2AO8FIiJOIOkM7RuSDgXeBXyurPWzs2L7y3iqRfPZocK047T/BfxVWv9TgPsj4v8BnwM+GxEvA94AfKVagJXvJbA6Il4WES8G7gUuioifknRh0JXG8guS/v7fFxEvTd+vL47yXliTmXRdTNgBayD9Eax0mqReYB+wNO0+oCMtv4vkh+tfI+JX8OQj9iP5cUScPTQj6WiSH+I5JF0PTB1mm58AV0v6NskP4840EbyWZIAXSLrmKP8BLXezpEFgM/Ax4GvAF9JYfybpfpL+fjYAl0s6Kj3Of1WpR6WVJInsa6QDKaXlZwDz9NRQGodJmhYRuyu2H/a9BI6X9ClgRlrH7soDS5oGvIKnuiYBeNoYYrcm4ERgjbbfj3dluaRjgVsl3RgRd45x358Ebo6Iv1IyHGlP5QoRsVTS90n6bvmJpIUkIz9dFRFfznCMBRHx26EZjTC+UURcL+k24C+BdZLeGRHrM9ZjA8lppyNIzt9/Ki0/CHh5RIw26M5I7+XXgc6IuEvShaSd1VU4CCiNkMRtkvCpITugpV0pLwU+Mo7ND+ep7nkvHG4FScdFxJaI+DRJT5cvIPnL+G3pX8NIapX07IzH/DHwN+l2zweOAfokPRfYHhGfJznVc2LFdn8g6Tr7T6QdqH2XZHyFe8s6kfshZYOwSKr6Yz3MezkdeFDS1KGYK2OJiN8Dv5T01+kxJOnF1Y5jzceJwJrBl4BXprdYVrtGUOmfgavSU08jtX7/XtLdkjYDe4EfRMQPgeuBDZK2kAxnOuyP9DC+CByUbreSpCfMPwLnAndLupNknIRvVmy3GRhML9x+cJj9rgQu4KnTQgDvB9rTi8/3kFyHGM2T7yXwP4DbSE6P/axsnRVAV3qx+ziSJHFRenppK8MPCWpNzL2PmpkVnFsEZmYF50RgZlZwTgRmZgXnRGBmVnBOBGZmBedEYGZWcE4EZmYF9/8B+vWdKSBA3r4AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0IlLUh3ipwJs"
      },
      "source": [
        "scikit-learnでは、AUCの値を求めます。\r\n",
        "\r\n",
        "ROC曲線を描画した場合は `fpr` と `tpr` を `auc` 関数に入力することで、またROC曲線を描画していない場合も真の合格者の系列 `y_pass` とそのスコアの系列 `y_score` を `roc_auc_score` に入力することで求められます。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "USKY26owr22_",
        "outputId": "883a3b7d-b13e-433c-9966-9c46d5abfa8a"
      },
      "source": [
        "from sklearn.metrics import auc, roc_auc_score\r\n",
        "\r\n",
        "print(f\"AUC (auc func): {auc(fpr, tpr)}\")\r\n",
        "print(f\"AUC (roc_auc_score): {roc_auc_score(y_pass, y_score)}\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "AUC (auc func): 1.0\n",
            "AUC (roc_auc_score): 0.8333333333333333\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RoGfIo9wuFWh"
      },
      "source": [
        "AUCが1になるケースを描画してみましょう。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 296
        },
        "id": "WqEOYXyCs2U8",
        "outputId": "c10060ce-4aef-47f7-b012-e562b5997c7c"
      },
      "source": [
        "y_perfect = [0, 0, 0, 0, 1, 1, 1, 1, 1, 1]\r\n",
        "y_score_perfect = [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]\r\n",
        "\r\n",
        "fpr, tpr, _ = roc_curve(y_perfect, y_score)\r\n",
        "\r\n",
        "plt.plot(fpr, tpr, marker='o')\r\n",
        "plt.xlabel('FPR: False Positive Rate')\r\n",
        "plt.ylabel('TPR: True Positive Rate')\r\n",
        "plt.grid()\r\n",
        "plt.show()\r\n",
        "\r\n",
        "print(f\"AUC (perfect): {roc_auc_score(y_perfect, y_score)}\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAbdUlEQVR4nO3df7xVdZ3v8debIyQpykwoc+9BBzXCSFTshCbXOqSJOl5htEwbZ8ZyYswfNTkxaXqtse5I42STjZlMmtUjRXOMObdIeszVk15DBS4IotIQ+YOjXs3cFoqBh8/9Yy1ysz1nn3U4e63D3uv9fDzOg7W+69fnu4H9Od/vWuv7VURgZmblNWK4AzAzs+HlRGBmVnJOBGZmJedEYGZWck4EZmYlt9twBzBY48aNi4kTJ+7UsS+//DJ77LFHYwPaxbnO5eA6l8NQ6rxixYpfRcQ+fW1rukQwceJEli9fvlPHdnd309nZ2diAdnGuczm4zuUwlDpLeqK/be4aMjMrOScCM7OScyIwMys5JwIzs5JzIjAzK7ncEoGkGyU9J+nhfrZL0jWS1ktaLemIvGJZtLKHGfPv4uw7X2bG/LtYtLInr0uZmTVc3t9hebYIbgJOqLP9RGBS+jMXuC6PIBat7OGSO9bQU9kMQE9lM5fcscbJwMyaQhHfYcpzGGpJE4EfRsQhfWy7HuiOiFvS9XVAZ0Q8U++cHR0dMZj3CGbMv+v3H2C1UW0jmLb/2MznaVaVSoWxY1u/ntVc53IoS51XPllhS++2N5S3jx3NfRe/L/N5JK2IiI6+tg3nC2XtwFNV6xvTsjckAklzSVoNjB8/nu7u7swX6SsJAGzp3UalUskebZPq7e0tRT2ruc7lUJY695UEIPluG8x3YT1N8WZxRCwAFkDSIhjMm3Xt9/fdImgfO5oln8meTZuV374sB9e5dfXXq9E+dnTD6j+cTw31APtVrU9Iyxpq3qzJjB7ZtkPZ6JFtzJs1udGXMjNruCK+w4YzEXQBf5E+PXQU8NJA9wd2xpxp7Vx56lRGtSVVbR87mitPncqcae2NvpSZWcNt/w5rHzsayOc7LLeuIUm3AJ3AOEkbgc8BIwEi4hvAYuAkYD3wCvCRvGKZM62dWx58kkqlUoruIDNrLXOmtTNnWntu3WG5JYKIOHOA7QGcn9f1zcwsG79ZbGZWck4EZmYl50RgZlZyTgRmZiXnRGBmVnJOBGZmJedEYGZWck4EZmYl50RgZlZyTgRmZiXnRGBmVnJOBGZmJedEYGZWck4EZmYl50RgZlZyTgRmZiXnRGBmVnJOBGZmJedEYGZWck4EZmYl50RgZlZyTgRmZiXnRGBmVnJOBGZmJedEYGZWck4EZmYl50RgZlZyTgRmZiXnRGBmVnJOBGZmJedEYGZWcrkmAkknSFonab2ki/vYvr+kuyWtlLRa0kl5xmNmZm+UWyKQ1AZcC5wITAHOlDSlZrfLgNsiYhpwBvD1vOIxM7O+5dkimA6sj4gNEbEFWAjMrtkngL3S5b2Bp3OMx8zM+rBbjuduB56qWt8IHFmzz+eBn0i6ENgDOK6vE0maC8wFGD9+PN3d3YMOplLZTG9v704d28w2bdrkOpeA61wOedU5z0SQxZnATRHxZUnvBr4r6ZCI2Fa9U0QsABYAdHR0RGdn56AvdN26pVQqFXbm2GbW3d3tOpeA61wOedU5z66hHmC/qvUJaVm1c4DbACJiKbA7MC7HmMzMrEaeiWAZMEnSAZJGkdwM7qrZ50ngWABJbydJBM/nGJOZmdXInAgkvXkwJ46I14ALgCXAoyRPB62VdIWkU9Ld/hb4mKSHgFuAsyMiBnMdMzMbmgHvEUg6GvgmsCewv6TDgL+OiPMGOjYiFgOLa8our1p+BJgx2KDNzKxxsrQIvgLMAl4AiIiHgPfkGZSZmRUnU9dQRDxVU9SbQyxmZjYMsjw++lTaPRSSRgKfJOnzNzOzFpClRXAucD7JC2I9wOHAgPcHzMysOWRpEUyOiD+rLpA0A7gvn5DMzKxIWVoEX8tYZmZmTajfFkE65MPRwD6SLqratBfQlndgZmZWjHpdQ6NI3h3YDRhTVf4b4AN5BmVmZsXpNxFExE+Bn0q6KSKeKDAmMzMrUJabxa9Iugp4B8lYQABExPtyi8rMzAqT5Wbx94DHgAOAvwceJxlQzszMWkCWRPCWiLgB2BoRP42IjwJuDZiZtYgsXUNb0z+fkfQnJNNJ/mF+IZmZWZGyJIIvStqbZMjor5E8Pvo3uUZlZmaFGTARRMQP08WXgJnw+zeLzcysBdR7oawNOJ1kjKE7I+JhSScDnwVGA9OKCdHMzPJUr0VwA8mcww8C10h6GugALo6IRUUEZ2Zm+auXCDqAQyNim6TdgWeBgyLihWJCMzOzItR7fHRLRGwDiIhXgQ1OAmZmradei+BgSavTZQEHpesCIiIOzT06MzPLXb1E8PbCojAzs2FTb9A5DzRnZlYCmSavNzOz1uVEYGZWcpkSgaTRkibnHYyZmRVvwEQg6b8Dq4A70/XDJXXlHZiZmRUjS4vg88B0oAIQEatI5iYwM7MWkCURbI2Il2rKIo9gzMyseFmGoV4r6cNAm6RJwCeAn+UblpmZFSVLi+BCkvmKfwfcTDIctecjMDNrEVkSwcERcWlEvCv9uSwde2hAkk6QtE7SekkX97PP6ZIekbRW0s2Dit7MzIYsS9fQlyX9EXA7cGtEPJzlxOl8BtcC7wc2AsskdUXEI1X7TAIuAWZExIuS9h10DczMbEgGbBFExEySmcmeB66XtEbSZRnOPR1YHxEbImILsBCYXbPPx4BrI+LF9FrPDSp6MzMbMkVkfwBI0lTg74APRcSoAfb9AHBCRPxVuv7nwJERcUHVPouAnwMzgDbg8xFxZx/nmgvMBRg/fvw7Fy5cmDnm7a58YDO9vb1cdvSegz62mW3atIk993SdW53rXA5DqfPMmTNXRERHX9sG7BqS9HbgQ8BpwAvArSQT2TfCbsAkoBOYANwjaWpEVKp3iogFwAKAjo6O6OzsHPSFrlu3lEqlws4c28y6u7td5xJwncshrzpnuUdwI8mX/6yIeHoQ5+4hmepyuwlpWbWNwAMRsRX4paSfkySGZYO4jpmZDUGWewTvjoh/HmQSgOTLfJKkAySNAs4AaoemWETSGkDSOOBtwIZBXsfMzIag3xaBpNsi4nRJa9jxTeJMM5RFxGuSLgCWkPT/3xgRayVdASyPiK502/GSHgF6gXmeDtPMrFj1uoY+mf558s6ePCIWA4tryi6vWg7govTHzMyGQb9dQxHxTLp4XkQ8Uf0DnFdMeGZmlrcsbxa/v4+yExsdiJmZDY969wg+TvKb/4GSVldtGgPcl3dgZmZWjHr3CG4GfgxcCVSPE/TbiPh1rlGZmVlh6iWCiIjHJZ1fu0HSHzoZmJm1hoFaBCcDK0geH1XVtgAOzDEuMzMrSL+JICJOTv/0tJRmZi0sy+T1MyTtkS6fJelqSfvnH5qZmRUhy+Oj1wGvSDqMZLC5XwDfzTUqMzMrTJZE8Fr6BvBs4F8i4lqSR0jNzKwFZBl99LeSLgH+HDhG0ghgZL5hmZlZUbK0CD5EMnH9RyPiWZLhpK/KNSozMytMlmGonwW+B+wt6WTg1Yj4Tu6RmZlZIbI8NXQ68CDwQeB04IF0GkozM2sBWe4RXAq8a/vE8pL2Af4DuD3PwMzMrBhZ7hGM2J4EUi9kPM7MzJpAlhbBnZKWALek6x+iZrIZMzNrXgMmgoiYJ+lU4L+lRQsi4gf5hmVmZkWpNx/BJOCfgIOANcCnI6KnqMDMzKwY9fr6bwR+CJxGMgLp1wqJyMzMClWva2hMRPxrurxO0v8tIiAzMytWvUSwu6RpvD4Pwejq9YhwYjAzawH1EsEzwNVV689WrQfwvryCMjOz4tSbmGZmkYGYmdnw8IthZmYl50RgZlZyTgRmZiWXZfRRpXMVX56u7y9pev6hmZlZEbK0CL4OvBs4M13/LXBtbhGZmVmhsgw6d2REHCFpJUBEvChpVM5xmZlZQbK0CLZKaiN5d2D7fATbco3KzMwKkyURXAP8ANhX0v8E/g/wD1lOLukESeskrZd0cZ39TpMUkjoyRW1mZg2TZRjq70laARxLMrzEnIh4dKDj0lbEtcD7gY3AMkldEfFIzX5jgE8CD+xE/GZmNkRZnhraH3gF+F9AF/ByWjaQ6cD6iNgQEVuAhcDsPvb7AvAl4NXMUZuZWcNkuVn8I5L7AwJ2Bw4A1gHvGOC4duCpqvWNwJHVO0g6AtgvIn4kaV5/J5I0F5gLMH78eLq7uzOEvaNKZTO9vb07dWwz27Rpk+tcAq5zOeRV5yxdQ1Or19Mv7/OGemFJI0gGsTs7QwwLgAUAHR0d0dnZOejrXbduKZVKhZ05tpl1d3e7ziXgOpdDXnUe9JvF6fDTRw64I/QA+1WtT0jLthsDHAJ0S3ocOAro8g1jM7NiDdgikHRR1eoI4Ajg6QznXgZMknQASQI4A/jw9o0R8RIwruo63STTYS7PFLmZmTVElhbBmKqfN5HcM+jrpu8OIuI14AJgCfAocFtErJV0haRTdj5kMzNrpLotgvQR0DER8emdOXlELAYW15Rd3s++nTtzDTMzG5q6LYKI6AVmFBSLmZkNg35bBJJ2S7t3VknqAr4PvLx9e0TcUUB8ZmaWs3pdQw+S3BjeHXiBHecoDsCJwMysBdRLBAKIiI8UFIuZmQ2Deolgn5pHR3cQEVfnEI+ZmRWsXiJoA/YkbRmYmVlrqpcInomIKwqLxMzMhkW9x0fdEjAzK4F6ieDYwqIwM7Nh028iiIhfFxmImZkNj0GPPmpmZq3FicDMrOQyJQJJC+qtm5lZ88raIrh+gHUzM2tSmRJBRKzYvpxOMXlwbhGZmVmh+k0EkvaSdImkf5F0vBIXAhuA04sL0czM8lTvzeLvAi8CS4G/Aj5L8pLZnIhYVUBsZmZWgHqJ4MCImAog6ZvAM8D+EfFqIZGZmVkh6t0j2Lp9IZ2pbKOTgJlZ66nXIjhM0m94fcyh0VXrERF75R6dmZnlrt9EEBFtRQZiZmbDo96cxbsD5wJvBVYDN6ZzGJuZWQupd4/g20AHsAY4CfhyIRGZmVmh6t0jmFL11NANJJPZm5lZi8n61JC7hMzMWlS9FsHh6VNCkDwp5KeGzMxaUL1E8FBETCssEjMzGxb1uoaisCjMzGzY1GsR7Cvpov42RsTVOcRjZmYFq5cI2oA9ef3NYjMza0H1EsEzEXHFUE4u6QTgqyRJ5ZsRMb9m+0UkI5u+BjwPfDQinhjKNc3MbHDq3SMYUktAUhtwLXAiMAU4U9KUmt1WAh0RcShwO/CPQ7mmmZkNXr1EcOwQzz0dWB8RGyJiC7AQmF29Q0TcHRGvpKv3AxOGeE0zMxukeoPO/XqI524Hnqpa3wgcWWf/c4Af97VB0lxgLsD48ePp7u4edDCVymZ6e3t36thmtmnTJte5BFzncsirzvXuERRG0lkk4xq9t6/tEbEAWADQ0dERnZ2dg77GdeuWUqlU2Jljm1l3d7frXAKucznkVec8E0EPsF/V+oS0bAeSjgMuBd4bEb/LMR4zM+tDvXsEQ7UMmCTpAEmjgDOAruodJE0DrgdOiYjncozFzMz6kVsiSAequwBYAjwK3BYRayVdIemUdLerSN5V+L6kVZK6+jndkCxa2cPKJyuse3EbM+bfxaKVb2iYmJmVVq73CCJiMbC4puzyquXj8rw+JEngkjvWsKV3GwA9lc1ccscaAOZMa8/78mZmu7w8u4Z2CVctWcfmrb07lG3e2stVS9YNU0RmZruWlk8ET1c2D6rczKxsWj4R/NexowdVbmZWNi2fCObNmszokW07lI0e2ca8WZOHKSIzs13LLvFCWZ623xD+u9tXs6V3G+1jRzNv1mTfKDYzS7V8IoAkGdzy4JNUKhWWfOZ9wx2OmdkupeW7hszMrD4nAjOzknMiMDMrOScCM7OScyIwMys5JwIzs5JzIjAzKzknAjOzknMiMDMrOScCM7OScyIwMys5JwIzs5JzIjAzKzknAjOzknMiMDMrOScCM7OScyIwMys5JwIzs5JzIjAzKzknAjOzknMiMDMrOScCM7OScyIwMys5JwIzs5JzIjAzK7lcE4GkEyStk7Re0sV9bH+TpFvT7Q9ImphHHItW9rDyyQrrXtzGjPl3sWhlTx6XMTNrSrklAkltwLXAicAU4ExJU2p2Owd4MSLeCnwF+FKj41i0sodL7ljDlt5tAPRUNnPJHWucDMzMUnm2CKYD6yNiQ0RsARYCs2v2mQ18O12+HThWkhoZxFVL1rF5a+8OZZu39nLVknWNvIyZWdPaLcdztwNPVa1vBI7sb5+IeE3SS8BbgF9V7yRpLjAXYPz48XR3d2cOoqeyud/ywZynWW3atKkU9azmOpeD69w4eSaChomIBcACgI6Ojujs7Mx8bPv9d/WZDNrHjmYw52lW3d3dpahnNde5HFznxsmza6gH2K9qfUJa1uc+knYD9gZeaGQQ82ZNZvTIth3KRo9sY96syY28jJlZ08ozESwDJkk6QNIo4Aygq2afLuAv0+UPAHdFRDQyiDnT2rny1Km0jx0NJC2BK0+dypxp7Y28jJlZ08qtayjt878AWAK0ATdGxFpJVwDLI6ILuAH4rqT1wK9JkkXDzZnWzpxp7aVsSpqZDSTXewQRsRhYXFN2edXyq8AH84zBzMzq85vFZmYl50RgZlZyTgRmZiXnRGBmVnJq8NOauZP0PPDETh4+jpq3lkvAdS4H17kchlLnP46Iffra0HSJYCgkLY+IjuGOo0iuczm4zuWQV53dNWRmVnJOBGZmJVe2RLBguAMYBq5zObjO5ZBLnUt1j8DMzN6obC0CMzOr4URgZlZyLZkIJJ0gaZ2k9ZIu7mP7myTdmm5/QNLE4qNsrAx1vkjSI5JWS/rfkv54OOJspIHqXLXfaZJCUtM/apilzpJOT/+u10q6uegYGy3Dv+39Jd0taWX67/uk4YizUSTdKOk5SQ/3s12Srkk/j9WSjhjyRSOipX5Ihrz+BXAgMAp4CJhSs895wDfS5TOAW4c77gLqPBN4c7r88TLUOd1vDHAPcD/QMdxxF/D3PAlYCfxBur7vcMddQJ0XAB9Pl6cAjw933EOs83uAI4CH+9l+EvBjQMBRwANDvWYrtgimA+sjYkNEbAEWArNr9pkNfDtdvh04VpIKjLHRBqxzRNwdEa+kq/eTzBjXzLL8PQN8AfgS8GqRweUkS50/BlwbES8CRMRzBcfYaFnqHMBe6fLewNMFxtdwEXEPyfws/ZkNfCcS9wNjJf2XoVyzFRNBO/BU1frGtKzPfSLiNeAl4C2FRJePLHWudg7JbxTNbMA6p03m/SLiR0UGlqMsf89vA94m6T5J90s6obDo8pGlzp8HzpK0kWT+kwuLCW3YDPb/+4CaYvJ6axxJZwEdwHuHO5Y8SRoBXA2cPcyhFG03ku6hTpJW3z2SpkZEZVijyteZwE0R8WVJ7yaZ9fCQiNg23IE1i1ZsEfQA+1WtT0jL+txH0m4kzckXCokuH1nqjKTjgEuBUyLidwXFlpeB6jwGOATolvQ4SV9qV5PfMM7y97wR6IqIrRHxS+DnJImhWWWp8znAbQARsRTYnWRwtlaV6f/7YLRiIlgGTJJ0gKRRJDeDu2r26QL+Ml3+AHBXpHdhmtSAdZY0DbieJAk0e78xDFDniHgpIsZFxMSImEhyX+SUiFg+POE2RJZ/24tIWgNIGkfSVbShyCAbLEudnwSOBZD0dpJE8HyhURarC/iL9Omho4CXIuKZoZyw5bqGIuI1SRcAS0ieOLgxItZKugJYHhFdwA0kzcf1JDdlzhi+iIcuY52vAvYEvp/eF38yIk4ZtqCHKGOdW0rGOi8Bjpf0CNALzIuIpm3tZqzz3wL/KulTJDeOz27mX+wk3UKSzMel9z0+B4wEiIhvkNwHOQlYD7wCfGTI12ziz8vMzBqgFbuGzMxsEJwIzMxKzonAzKzknAjMzErOicDMrOScCKwQknolrar6mSipU9JL6fqjkj6X7ltd/pikf8pw/upjVkn6jwH2/WED6vS4pDXpCJA/kfRHO3GOn6V/TpT04aryDknXNCDGnfks50iaMtRrW/NwIrCibI6Iw6t+Hk/L742Iw0mGvTirakjd7eXTgJMlzchwjXurzn9c46vQp5kRcSiwHPjsYA+OiKPTxYnAh6vKl0fEJxoS4eA/yzkko3haSTgR2C4hIl4GVgBvrSnfDKxiJwbVkjRd0tJ0nPqfSZrcxz7vrWpFrJQ0Ji2fJ2lZ+tv+32e43D3AWyXtLulbaUthpaSZ6fneIenB9DqrJU1Kyzelx88Hjkm3f2p7q0XSiLTlMbYq5v+UNF7SPpL+LY1z2UBf8LWfpaSPpcc9lJ7nzZKOBk4BrkpjOSj9uVPSCkn3Sjo4w+dhTcSJwIoyuuoL9we1GyW9hWQ8oLU15X9AMlbOPen6uZLO7ecax1Rd41LgMeCYiJgGXA78Qx/HfBo4P/2N+Rhgs6Tj02tOBw4H3inpPQPU72RgDXA+EBExlWQwtG9L2h04F/hqVetnY83xF/N6i+Yr2wvTgdP+HfjTtP5HAk9ExP8Dvgp8JSLeBZwGfLNegLWfJXBHRLwrIg4DHgXOiYifkQxhMC+N5Rck4/1fGBHvTD+vrw/wWViTabkhJmyXtTn9Eqx1jKSVwDZgfjp8QGda/hDJF9c/R8Sz8PtX7Ptzb0ScvH1F0n4kX8STSIYeGNnHMfcBV0v6HskX48Y0ERxPMsELJENzVH+BVrtbUi+wGrgM+BbwtTTWxyQ9QTLez1LgUkkT0uv8Z5161LqVJJF9i3QipbT8OGCKXp9KYy9Je0bEpprj+/wsgUMkfREYm9ZxSe2FJe0JHM3rQ5MAvGkQsVsTcCKw4bbDl3dtuaQDgPsl3RYRqwZ57i8Ad0fEnyqZjrS7doeImC/pRyRjt9wnaRbJzE9XRsT1Ga4xMyJ+tX1F/cxvFBE3S3oA+BNgsaS/joi7MtZjKUm30z4k/fdfTMtHAEdFxECT7vT3Wd4EzImIhySdTTpYXY0RQKWfJG4twl1DtktLh1KeD3xmJw7fm9eH5z27rx0kHRQRayLiSyQjXR5M8pvxR9PfhpHULmnfjNe8F/iz9Li3AfsD6yQdCGyIiGtIunoOrTnutyRDZ79BOoDaD0jmV3i0ahC5n1A1CYukul/WfXyWY4BnJI3cHnNtLBHxG+CXkj6YXkOSDqt3HWs+TgTWDL4BvCd9xLLePYJa/whcmXY99df6/RtJD0taDWwFfhwRPwFuBpZKWkMynWmfX9J9+DowIj3uVpKRMH8HnA48LGkVyTwJ36k5bjXQm964/VQf570VOIvXu4UAPgF0pDefHyG5DzGQ33+WwP8AHiDpHnusap+FwLz0ZvdBJEninLR7aS19TwlqTcyjj5qZlZxbBGZmJedEYGZWck4EZmYl50RgZlZyTgRmZiXnRGBmVnJOBGZmJff/AYZteibmdCGvAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "text": [
            "AUC (perfect): 1.0\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}