Generic capture of argument / return types for functions

[bwo]

Feature

Here's how the mypy docs propose typing a generic decorator:

from typing import Any, Callable, TypeVar, Tuple, cast

F = TypeVar('F', bound=Callable[..., Any])

# A decorator that preserves the signature.
def my_decorator(func: F) -> F:
    def wrapper(*args, **kwds):
        print("Calling", func)
        return func(*args, **kwds)
    return cast(F, wrapper)

The documentation notes that the cast is necessary because wrapper is untyped, which it says is ok because "wrapper functions are typically small enough that this is not a big problem." This obscures a deeper justification: you can't give it an accurate type. (If you can, this part of the docs would be a great place to put how!)

The closest I've been able to come is something like this:

from typing import Generic, TypeVar, Callable, Any, cast
from mypy_extensions import VarArg, KwArg

V = TypeVar('V')
K = TypeVar('K')
R = TypeVar('R')


def custom_greeting_decorator(greeting: str) -> Callable[[Callable[[VarArg(V), KwArg(K)], R]], Callable[[VarArg(V), KwArg(K)], R]]:
    def decorator(func: Callable[[VarArg(V), KwArg(K)], R]) -> Callable[[VarArg(V), KwArg(K)], R]:
        def wrapper(*args: V, **kwargs: K) -> R:
            print(greeting, func)
            return func(*args, **kwargs)
        return wrapper
    return decorator


@custom_greeting_decorator('hi')  # mypy complains about this decoration
def foo(x: int) -> str:
    return (', '.join(['world'] * x))

reveal_type(foo)  # mypy reports this as `def (*builtins.int*, **<nothing>) -> builtins.str*`, which is *almost* right
reveal_type(foo(3))  # mypy accurately reports this as `str`
foo(3, y=5)  # mypy accurately reports this as an error
foo('hi')  # mypy accurately reports *this* as an error

The interesting thing here is that mypy has assigned foo a type that actually is mostly accurate. If you have a value of type Any (or, presumably, <nothing>), you can pass it as a keyword arg to foo. But you can't pass a keyword arg with any non-Any type. And if foo had any keyword arguments, it would accept any keyword arguments with the right type, regardless of whether it had the right name.

Here's what mypy reports for this:

foo.py:18: error: Argument 1 has incompatible type "Callable[[int], str]"; expected "Callable[[VarArg(int), KwArg(<nothing>)], str]

So, the feature request is: it would be really useful to be able to refer generically to the argument and return types of a function, when they are known, somehow, so that they could then be used to annotate a further function with the same unknown types.

Motivation

The example function is really simple. But sometimes your example function is not so simple, and sometimes it's not a function at all. What if you had this?

def custom_greeting_decorator2(greeting: str) -> Callable[[Callable[[VarArg(V), KwArg(K)], R]], Greeter[V, K, R]]:
    def decorator(func: Callable[[VarArg(V), KwArg(K)], R]) -> Greeter[V, K, R]:
        return Greeter(greeting, func)
    return decorator


class Greeter(Generic[V, K, R]):
    def __init__(self, greeting : str, func: Callable[[VarArg(V), KwArg(K)], R]) -> None:
        self.greeting = greeting
        self.func = func

    def __call__(self, *args : V, **kwargs: K) -> R:
        print(self.greeting)
        return self.func(*args, **kwargs)


@custom_greeting_decorator2('hello')
def bar(x: str) -> int:
    return len(x)

This still won't work, for the same reasons that the first one doesn't work, but it's hopefully a little clearer why you'd even want the separate argument/return types, rather than just using the "whole" function type. Here, you want to use __call__ on the class with the types from the wrapped function, but you also don't want to just cast(T, Greeter()) in the return value because you also want the other attributes of the class (assuming it has some). And AFAICT there's just no way to annotate __call__ correctly at the moment. I'd be delighted to learn that it is possible, though!

[gvanrossum]

I think PEP 612 will solve this. Implementation is a work in progress: see #8645.