Decorator Taxonomy A-E EuroPython Slides.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from sys import getsizeof\n",
    "from datetime import datetime\n",
    "import sys, inspect, dis, asyncio\n",
    "from functools import wraps\n",
    "from unittest.mock import MagicMock\n",
    "import random\n",
    "\n",
    "# file touch\n",
    "with open('hello_world.txt', 'a') as f:\n",
    "    pass\n",
    "\n",
    "def null_it(func):\n",
    "    'Replace with no-op'\n",
    "    def null(*args,**kwargs): pass\n",
    "    return null"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# A Taxonomy of Decorators: A-E\n",
    "\n",
    "Andy Fundinger, Senior Engineer\n",
    "\n",
    "EuroPython, Edinburgh, Scotland, July 29, 2018 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Introduction \n",
    "* Who I am\n",
    "* What is Bloomberg?\n",
    "* What is this talk?\n",
    "  - we use decorators, but when we go to talk about them it's hard to get them into categories \n",
    "  - with a common terminology we can discuss more easily\n",
    "* Who is this talk for?  \n",
    "  - Intermediate developers who can write decorators, but maybe aren't sure when and why\n",
    "  - Architects who may need to work across teams to implement and manage their design"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Decorator Syntax and Implementation \n",
    "\n",
    "Any decorator can be replaced with code in the decorated functions.  However, decorators allow this code to be reused and factored out of the functions.\n",
    "- Decorators without the @ sign\n",
    "- Function Decorators\n",
    "- Class Decorators\n",
    "- Decorators with arguments\n",
    "- Decorators written as classes\n",
    "\n",
    "Generally decorators insert a section of code of arbitrary complexity in a single line.  Debuggers generally skip over this code--for better or worse."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Basic Syntax\n",
    "### Decorators without the @ sign\n",
    "\n",
    "The oldest decorators in Python are @staticmethod and @classmethod.  Dating back to Python 2.2 we used those like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "class Util(object):\n",
    "    def wibble():\n",
    "        print(\"Wobble\")\n",
    "    wibble = staticmethod(wibble)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### The @ sign\n",
    "\n",
    "The @ sign was added in 2.4 as a syntactic sugar for this pattern"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "class Util(object):\n",
    "    @staticmethod\n",
    "    def wibble():\n",
    "        print(\"Wobble\")\n",
    "    @null_it\n",
    "    def disabled():\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can write our own decorators as long as we accept a function and  return a replacement function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "def null_it(func):\n",
    "    'Replace with no-op'\n",
    "    def null(*args,**kwargs): pass\n",
    "    return null"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Nonsense\n"
     ]
    }
   ],
   "source": [
    "class Printer(object):\n",
    "    @null_it\n",
    "    def printOut(self, stuff):\n",
    "        sys.stdout.write(stuff)\n",
    "    def print_out(self, stuff):\n",
    "        print(stuff)\n",
    "              \n",
    "Printer().print_out('Nonsense')        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "Printer().printOut('something')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Class Decorators\n",
    "\n",
    "In Python 2.6 we can decorate classes too,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@null_it\n",
    "class DeadClass:\n",
    "    pass\n",
    "\n",
    "DeadClass() is None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "### More advanced declarations\n",
    "\n",
    "#### @wraps - impersonate the orginal function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n",
      "HelloHello\n"
     ]
    }
   ],
   "source": [
    "from functools import wraps\n",
    "\n",
    "def double_arg1(func):\n",
    "    #@wraps(func)\n",
    "    def wrapper(arg_one,*args, **kwargs):\n",
    "        return func(arg_one*2, *args, **kwargs)\n",
    "    wrapper.__wrapped__ = func\n",
    "    wrapper.__module__ = func.__module__\n",
    "    wrapper.__doc__ = func.__doc__\n",
    "        \n",
    "    return wrapper \n",
    "\n",
    "@double_arg1\n",
    "def print_x(x):\n",
    "    print(x)\n",
    "    \n",
    "print_x(1)\n",
    "print_x('Hello')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function __main__.double_arg1.<locals>.wrapper>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print_x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Decorators with arguments \n",
    "\n",
    "Decorators with arguments aren't actually doing anything all that special.  They simply call the function with the arguments and __that__ function returns the actual decorator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3\n",
      "Hello Hello Hello \n"
     ]
    }
   ],
   "source": [
    "def mult_arg(mult):\n",
    "    def deco(func):\n",
    "        def wrapper(arg_one, *args, **kwargs):\n",
    "            return func(arg_one*mult, *args, **kwargs)\n",
    "        return wrapper\n",
    "    return deco\n",
    "\n",
    "@mult_arg(3)\n",
    "def print_x(x):\n",
    "    print(x)\n",
    "    \n",
    "print_x(1)\n",
    "print_x('Hello ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice, this is two closures, one for the argument (mult) and one for the function (func).  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Decorators written as classes\n",
    "\n",
    "Normally we write decorators that are closures, but there's no particular benefit to this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10, 30) {}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "13"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import random\n",
    "\n",
    "class trace_it:\n",
    "    def __init__(self, func):\n",
    "        self.func = func\n",
    "    def __call__(self, *args, **kwargs):\n",
    "        print(args, kwargs)\n",
    "        return self.func(*args, **kwargs)\n",
    "        \n",
    "@trace_it\n",
    "def rand(min_val, max_val):\n",
    "    return random.randint(min_val, max_val)\n",
    "rand(10,30)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## A - Argument Changing Decorators \n",
    "\n",
    "* add or remove an argument when the function is called\n",
    "* change the value or type of an argument at call time \n",
    "* similarly alter the return value\n",
    "\n",
    "### Problems \n",
    "\n",
    "* calling the apparent signature does not actually work\n",
    "* calling a function for a test requires injecting data to drive the decorator properly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Example:  pytest.mark.parametrize()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pytest\n",
    "\n",
    "@pytest.mark.parametrize(\"test_input,expected\", [\n",
    "    (\"3+5\", 8),\n",
    "    (\"2+4\", 6),\n",
    "    (\"6*9\", 42),\n",
    "])\n",
    "\n",
    "def test_eval(test_input, expected):\n",
    "    assert eval(test_input) == expected"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Example implementation -- adding the func name in the call"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "self_aware: here\n"
     ]
    }
   ],
   "source": [
    "def fn_with_name(func):\n",
    "    def wrapper(*args, **kwargs):\n",
    "        func(func.__name__,*args,**kwargs)\n",
    "    return wrapper\n",
    "        \n",
    "@fn_with_name\n",
    "def self_aware(name, oth):\n",
    "    print('{name}: {oth}'.format(name=name, oth=oth))\n",
    "    \n",
    "self_aware('here')\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## B - Binding Decorators\n",
    "\n",
    "* implement the Descriptor Protocol to change how functions behave\n",
    "* the standard library includes @staticmethod, @classmethod, and @property\n",
    "\n",
    "### Problems\n",
    "\n",
    "* creates an alternative to instance methods and attributes\n",
    "* new language patterns arguably better fitting other languages \n",
    "* time shifts otherwise normal exceptions or introduces new ones"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Example: SQLAlchemy Hybrid Properties"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "#from sqlalchemy import func\n",
    "hybrid_property= MagicMock()\n",
    "\n",
    "class Interval(object):\n",
    "    @hybrid_property\n",
    "    def radius(self):\n",
    "        return abs(self.length) / 2\n",
    "\n",
    "    @radius.expression\n",
    "    def radius(cls):\n",
    "        return func.abs(cls.length) / 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Example implementation -- instance method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "class instance_method:\n",
    "    def __init__(self, func):\n",
    "        self.func = func\n",
    "    def __get__(self, inst, cls):\n",
    "        if inst is None:\n",
    "            raise TypeError(f'{self.func.__name__} is only valid on instances.')\n",
    "        return self.func.__get__(inst, cls)\n",
    "    \n",
    "class GoodClass:\n",
    "    @instance_method\n",
    "    def simple_method(self):\n",
    "        print('simple_method')\n",
    "    def normal_method(self):\n",
    "        print('normal_method')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "normal_method\n",
      "simple_method\n"
     ]
    }
   ],
   "source": [
    "GoodClass().normal_method()\n",
    "GoodClass().simple_method()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "normal_method() missing 1 required positional argument: 'self'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-18-63f2a7128816>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mGoodClass\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnormal_method\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: normal_method() missing 1 required positional argument: 'self'"
     ]
    }
   ],
   "source": [
    "GoodClass.normal_method()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "simple_method is only valid on instances.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-19-065d630c3609>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mGoodClass\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msimple_method\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m<ipython-input-16-53c5e8f52716>\u001b[0m in \u001b[0;36m__get__\u001b[0;34m(self, inst, cls)\u001b[0m\n\u001b[1;32m      4\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__get__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minst\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcls\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0minst\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mTypeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mf'{self.func.__name__} is only valid on instances.'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      7\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__get__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minst\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcls\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTypeError\u001b[0m: simple_method is only valid on instances."
     ]
    }
   ],
   "source": [
    "GoodClass.simple_method()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function __main__.GoodClass.normal_method>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GoodClass.normal_method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "simple_method is only valid on instances.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-21-5ffb98a5765b>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mGoodClass\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msimple_method\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m<ipython-input-16-53c5e8f52716>\u001b[0m in \u001b[0;36m__get__\u001b[0;34m(self, inst, cls)\u001b[0m\n\u001b[1;32m      4\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__get__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minst\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcls\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0minst\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mTypeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mf'{self.func.__name__} is only valid on instances.'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      7\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__get__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minst\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcls\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTypeError\u001b[0m: simple_method is only valid on instances."
     ]
    }
   ],
   "source": [
    "GoodClass.simple_method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "datetime.datetime(2018, 8, 17, 12, 13, 9, 829535)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datetime.now() # 6 min B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## C - Control Flow Decorators\n",
    "\n",
    "* change whether a function will be called and how many times\n",
    "\n",
    "### Problems\n",
    "\n",
    "* a predictable control flow now has a hidden conditional\n",
    "* a single invocation might now lead to 0, 1, or many executions of the function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Example: Retry Decorator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<MagicMock name='mock()()' id='140154847706696'>\n"
     ]
    }
   ],
   "source": [
    "#from retrying import retry\n",
    "retry = MagicMock()\n",
    "\n",
    "@retry\n",
    "def do_something_unreliable():\n",
    "    if random.randint(0, 10) > 1:\n",
    "        raise IOError(\"Broken sauce, everything is hosed!!!111one\")\n",
    "    else:\n",
    "        return \"Awesome sauce!\"\n",
    "\n",
    "print(do_something_unreliable())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Example implementation -- infinite retry"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Invalid negative number -82\n",
      "Invalid negative number -82\n",
      "Invalid negative number -5\n",
      "Invalid negative number -47\n",
      "Invalid negative number -67\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def infinite_retry(func):\n",
    "    def wrapper(*args, **kwargs):\n",
    "        while True:\n",
    "            try:\n",
    "                return func(*args, **kwargs)\n",
    "            except RuntimeError as e:\n",
    "                print(e)\n",
    "    return wrapper\n",
    "\n",
    "@infinite_retry\n",
    "def random_fail(max_value):\n",
    "    ret = random.randint(-100, max_value)\n",
    "    if ret<0:\n",
    "        raise RuntimeError(\"Invalid negative number {ret}\".format(ret=ret))\n",
    "    return ret\n",
    "\n",
    "random_fail(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "datetime.datetime(2018, 8, 17, 12, 13, 15, 514301)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datetime.now() # 2 min C "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## D - Descriptive Decorators\n",
    "\n",
    "* add the decorated object to some sort of collection\n",
    "* this collection will serve some other purpose such as:\n",
    "  - documentation\n",
    "  - dispatching\n",
    "  - plugins\n",
    "\n",
    "### Problems \n",
    "\n",
    "* it's unclear how dispatching will be done as a result of registration\n",
    "* similarly it's hard to see where the registration is maintained"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Example pytest.marks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "#import pytest\n",
    "pytest = MagicMock()\n",
    "\n",
    "@pytest.mark.webtest\n",
    "def test_send_http():\n",
    "    pass # perform some webtest test for your app\n",
    "def test_something_quick():\n",
    "    pass\n",
    "def test_another():\n",
    "    pass\n",
    "class TestClass(object):\n",
    "    def test_method(self):\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Example:  flask.app.route"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "#app = Flask(__name__)\n",
    "app = MagicMock()\n",
    "\n",
    "@app.route('/')\n",
    "def hello_world():\n",
    "    return 'Hello, World!'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Example implementation -- qa list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings \n",
    "\n",
    "to_qa=[]\n",
    "def qa(func):\n",
    "    to_qa.append(func.__name__)\n",
    "    return func\n",
    "\n",
    "@qa\n",
    "def new_code(): pass\n",
    "\n",
    "@qa\n",
    "def refactored_code(): pass\n",
    "\n",
    "def well_trusted_code(): pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['new_code', 'refactored_code']"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "to_qa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "new_code()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## E - Execution Decorators \n",
    "\n",
    "* reads the method/class code\n",
    "* may reinterpret the source code to basically not be python\n",
    "\n",
    "### Problems\n",
    "* many\n",
    "* and more\n",
    "\n",
    "This truly means that the code you wrote is changed--by the decorator--to some other code that is then executed.  It might:\n",
    "* be analyzed for dependencies\n",
    "* have objects in the ast swapped out, injected or removed\n",
    "* be recompiled with different rules "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Example: cython"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "cython = MagicMock()\n",
    "\n",
    "@cython.locals(a=cython.double, b=cython.double, n=cython.p_double)\n",
    "def foo(a, b, x, y):\n",
    "    n = a*b\n",
    "    ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Example implementation -- code replacer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "def replacer(old, new):\n",
    "    def deco(func):\n",
    "        source = inspect.getsource(func.__code__)\n",
    "        lines = source.split('\\n')\n",
    "        new_source = lines[1]+'\\n'+('\\n'.join(lines[2:]).replace(old, new))\n",
    "        exec(new_source,globals())\n",
    "        return globals()[func.__name__]\n",
    "    return deco"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample: 10\n"
     ]
    }
   ],
   "source": [
    "def sample(a, b):\n",
    "    x = a + b\n",
    "    y = x * 2\n",
    "    print('Sample: ' + str(y))\n",
    "sample(1,4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample: 26\n"
     ]
    }
   ],
   "source": [
    "@replacer('b','b*3')\n",
    "def sample(a, b):\n",
    "    x = a + b\n",
    "    y = x * 2\n",
    "    print('Sample: ' + str(y))\n",
    "sample(1,4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sa*3mple: 14\n"
     ]
    }
   ],
   "source": [
    "@replacer('a','a*3')\n",
    "def sample(a, b):\n",
    "    x = a + b\n",
    "    y = x * 2\n",
    "    print('Sample: ' + str(y))\n",
    "sample(1,4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Other tools available for execution decorators\n",
    "\n",
    "* bytecode manipulation\n",
    "* ast manipulation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Conclusion\n",
    "\n",
    "Our Taxonomy\n",
    "* Argument changing\n",
    "  - @click.option\n",
    "  - @flask.templated\n",
    "  - @django.views.decorators.gzip.gzip_page\n",
    "* Binding\n",
    "  - @variants.primary\n",
    "  - @pyramid.decorator.reify\n",
    "* Control flow\n",
    "  - @functools.lru_cache\n",
    "  - @django.views.decorators.http.require_http_methods\n",
    "  - @twisted.internet.defer.inlineCallbacks\n",
    "* Descriptive\n",
    "  - @numpy.testing.decorators.setastest\n",
    "* Execution\n",
    "  - @numba.jit\n",
    "* ?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "## Feedback Please\n",
    "* Pacing \n",
    "  - Syntax vs Examples\n",
    "  - Examples themselves\n",
    "* choice of examples\n",
    "  - Real world examples\n",
    "  - Example implementations\n",
    "  - Execution decorator example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bytecode manipulation scratch work "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'|\\x00d\\x01\\x14\\x00|\\x01\\x17\\x00}\\x02|\\x02d\\x02\\x14\\x00}\\x03t\\x00d\\x03t\\x01|\\x03\\x83\\x01\\x17\\x00\\x83\\x01\\x01\\x00d\\x00S\\x00'"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample.__code__.co_code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "          0 LOAD_FAST                0 (0)\n",
      "          2 LOAD_CONST               1 (1)\n",
      "          4 BINARY_MULTIPLY\n",
      "          6 LOAD_FAST                1 (1)\n",
      "          8 BINARY_ADD\n",
      "         10 STORE_FAST               2 (2)\n",
      "         12 LOAD_FAST                2 (2)\n",
      "         14 LOAD_CONST               2 (2)\n",
      "         16 BINARY_MULTIPLY\n",
      "         18 STORE_FAST               3 (3)\n",
      "         20 LOAD_GLOBAL              0 (0)\n",
      "         22 LOAD_CONST               3 (3)\n",
      "         24 LOAD_GLOBAL              1 (1)\n",
      "         26 LOAD_FAST                3 (3)\n",
      "         28 CALL_FUNCTION            1\n",
      "         30 BINARY_ADD\n",
      "         32 CALL_FUNCTION            1\n",
      "         34 POP_TOP\n",
      "         36 LOAD_CONST               0 (0)\n",
      "         38 RETURN_VALUE\n"
     ]
    }
   ],
   "source": [
    "dis.dis(sample.__code__.co_code)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [
    {
     "ename": "OSError",
     "evalue": "could not get source code",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mOSError\u001b[0m                                   Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-39-c8b071f070c3>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0msource\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minspect\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetsource\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msample\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__code__\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0msource\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/bb/lib/python3.6/inspect.py\u001b[0m in \u001b[0;36mgetsource\u001b[0;34m(object)\u001b[0m\n\u001b[1;32m    966\u001b[0m     \u001b[0;32mor\u001b[0m \u001b[0mcode\u001b[0m \u001b[0mobject\u001b[0m\u001b[0;34m.\u001b[0m  \u001b[0mThe\u001b[0m \u001b[0msource\u001b[0m \u001b[0mcode\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0mreturned\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0ma\u001b[0m \u001b[0msingle\u001b[0m \u001b[0mstring\u001b[0m\u001b[0;34m.\u001b[0m  \u001b[0mAn\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    967\u001b[0m     OSError is raised if the source code cannot be retrieved.\"\"\"\n\u001b[0;32m--> 968\u001b[0;31m     \u001b[0mlines\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlnum\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgetsourcelines\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobject\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    969\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlines\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    970\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/bb/lib/python3.6/inspect.py\u001b[0m in \u001b[0;36mgetsourcelines\u001b[0;34m(object)\u001b[0m\n\u001b[1;32m    953\u001b[0m     raised if the source code cannot be retrieved.\"\"\"\n\u001b[1;32m    954\u001b[0m     \u001b[0mobject\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0munwrap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobject\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 955\u001b[0;31m     \u001b[0mlines\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlnum\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfindsource\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobject\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    956\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    957\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mismodule\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobject\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/bb/lib/python3.6/inspect.py\u001b[0m in \u001b[0;36mfindsource\u001b[0;34m(object)\u001b[0m\n\u001b[1;32m    784\u001b[0m         \u001b[0mlines\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlinecache\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetlines\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfile\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    785\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mlines\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 786\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mOSError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'could not get source code'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    787\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    788\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mismodule\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobject\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mOSError\u001b[0m: could not get source code"
     ]
    }
   ],
   "source": [
    "source = inspect.getsource(sample.__code__)\n",
    "source"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample: 10\n"
     ]
    }
   ],
   "source": [
    "def sample(a, b):\n",
    "    x = a + b\n",
    "    y = x * 2\n",
    "    print('Sample: ' + str(y))\n",
    "sample(1,4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"def sample(a, b):\\n    x = a + b\\n    y = x * 2\\n    print('Sample: ' + str(y))\\n\""
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "source = inspect.getsource(sample.__code__)\n",
    "source"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [],
   "source": [
    "lines = source.split('\\n')\n",
    "new_source = lines[0]+'\\n'+('\\n'.join(lines[1:]).replace('b','b*3'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"def sample(a, b):\\n    x = a + b*3\\n    y = x * 2\\n    print('Sample: ' + str(y))\\n\""
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_source"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [],
   "source": [
    "glb = globals()\n",
    "exec(new_source,glb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample: 26\n",
      "Sample: 26\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(None, None)"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "glb['sample'](1,4), sample(1,4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  2           0 LOAD_FAST                0 (a)\n",
      "              2 LOAD_FAST                1 (b)\n",
      "              4 LOAD_CONST               1 (3)\n",
      "              6 BINARY_MULTIPLY\n",
      "              8 BINARY_ADD\n",
      "             10 STORE_FAST               2 (x)\n",
      "\n",
      "  3          12 LOAD_FAST                2 (x)\n",
      "             14 LOAD_CONST               2 (2)\n",
      "             16 BINARY_MULTIPLY\n",
      "             18 STORE_FAST               3 (y)\n",
      "\n",
      "  4          20 LOAD_GLOBAL              0 (print)\n",
      "             22 LOAD_CONST               3 ('Sample: ')\n",
      "             24 LOAD_GLOBAL              1 (str)\n",
      "             26 LOAD_FAST                3 (y)\n",
      "             28 CALL_FUNCTION            1\n",
      "             30 BINARY_ADD\n",
      "             32 CALL_FUNCTION            1\n",
      "             34 POP_TOP\n",
      "             36 LOAD_CONST               0 (None)\n",
      "             38 RETURN_VALUE\n"
     ]
    }
   ],
   "source": [
    "dis.dis(sample)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}