diff --git a/docs/source/casts.rst b/docs/source/casts.rst deleted file mode 100644 index 61eeb3062625..000000000000 --- a/docs/source/casts.rst +++ /dev/null @@ -1,49 +0,0 @@ -.. _casts: - -Casts and type assertions -========================= - -Mypy supports type casts that are usually used to coerce a statically -typed value to a subtype. Unlike languages such as Java or C#, -however, mypy casts are only used as hints for the type checker, and they -don't perform a runtime type check. Use the function :py:func:`~typing.cast` to perform a -cast: - -.. code-block:: python - - from typing import cast, List - - o: object = [1] - x = cast(List[int], o) # OK - y = cast(List[str], o) # OK (cast performs no actual runtime check) - -To support runtime checking of casts such as the above, we'd have to check -the types of all list items, which would be very inefficient for large lists. -Casts are used to silence spurious -type checker warnings and give the type checker a little help when it can't -quite understand what is going on. - -.. note:: - - You can use an assertion if you want to perform an actual runtime check: - - .. code-block:: python - - def foo(o: object) -> None: - print(o + 5) # Error: can't add 'object' and 'int' - assert isinstance(o, int) - print(o + 5) # OK: type of 'o' is 'int' here - -You don't need a cast for expressions with type ``Any``, or when -assigning to a variable with type ``Any``, as was explained earlier. -You can also use ``Any`` as the cast target type -- this lets you perform -any operations on the result. For example: - -.. code-block:: python - - from typing import cast, Any - - x = 1 - x.whatever() # Type check error - y = cast(Any, x) - y.whatever() # Type check OK (runtime error) diff --git a/docs/source/common_issues.rst b/docs/source/common_issues.rst index 4de0ead6c7c5..d3c1761bc994 100644 --- a/docs/source/common_issues.rst +++ b/docs/source/common_issues.rst @@ -370,7 +370,7 @@ Complex type tests Mypy can usually infer the types correctly when using :py:func:`isinstance `, :py:func:`issubclass `, or ``type(obj) is some_class`` type tests, -and even user-defined type guards, +and even :ref:`user-defined type guards `, but for other kinds of checks you may need to add an explicit type cast: @@ -385,7 +385,7 @@ explicit type cast: found = a[index] # Has `object` type, despite the fact that we know it is `str` return cast(str, found) # So, we need an explicit cast to make mypy happy - + Alternatively, you can use ``assert`` statement together with some of the supported type inference techniques: @@ -729,6 +729,8 @@ not necessary: def test(self, t: List[int]) -> Sequence[str]: # type: ignore[override] ... +.. _unreachable: + Unreachable code ---------------- diff --git a/docs/source/dynamic_typing.rst b/docs/source/dynamic_typing.rst index cea5248a3712..add445009666 100644 --- a/docs/source/dynamic_typing.rst +++ b/docs/source/dynamic_typing.rst @@ -80,7 +80,7 @@ operations: n = 1 # type: int n = o # Error! -You can use :py:func:`~typing.cast` (see chapter :ref:`casts`) or :py:func:`isinstance` to -go from a general type such as :py:class:`object` to a more specific -type (subtype) such as ``int``. :py:func:`~typing.cast` is not needed with +You can use different :ref:`type narrowing ` +techniques to narrow :py:class:`object` to a more specific +type (subtype) such as ``int``. Type narrowing is not needed with dynamically typed values (values with type ``Any``). diff --git a/docs/source/generics.rst b/docs/source/generics.rst index 817466d2469a..f09e0572ee35 100644 --- a/docs/source/generics.rst +++ b/docs/source/generics.rst @@ -562,7 +562,7 @@ non-function (e.g. ``my_decorator(1)``) will be rejected. Also note that the ``wrapper()`` function is not type-checked. Wrapper functions are typically small enough that this is not a big problem. This is also the reason for the :py:func:`~typing.cast` call in the -``return`` statement in ``my_decorator()``. See :ref:`casts`. +``return`` statement in ``my_decorator()``. See :ref:`casts `. .. _decorator-factories: diff --git a/docs/source/index.rst b/docs/source/index.rst index c9ee1ce1f9ad..8aae6e0a8ac7 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -39,7 +39,7 @@ Mypy is a static type checker for Python 3 and Python 2.7. protocols dynamic_typing python2 - casts + type_narrowing duck_type_compatibility stubs generics diff --git a/docs/source/more_types.rst b/docs/source/more_types.rst index 2d99cdc7bef6..a240ac338988 100644 --- a/docs/source/more_types.rst +++ b/docs/source/more_types.rst @@ -1151,148 +1151,3 @@ section of the docs has a full description with an example, but in short, you wi need to give each TypedDict the same key where each value has a unique unique :ref:`Literal type `. Then, check that key to distinguish between your TypedDicts. - - -User-Defined Type Guards -************************ - -Mypy supports User-Defined Type Guards -(:pep:`647`). - -A type guard is a way for programs to influence conditional -type narrowing employed by a type checker based on runtime checks. - -Basically, a ``TypeGuard`` is a "smart" alias for a ``bool`` type. -Let's have a look at the regular ``bool`` example: - -.. code-block:: python - - from typing import List - - def is_str_list(val: List[object]) -> bool: - """Determines whether all objects in the list are strings""" - return all(isinstance(x, str) for x in val) - - def func1(val: List[object]) -> None: - if is_str_list(val): - reveal_type(val) # Reveals List[object] - print(" ".join(val)) # Error: incompatible type - -The same example with ``TypeGuard``: - -.. code-block:: python - - from typing import List - from typing import TypeGuard # use `typing_extensions` for Python 3.9 and below - - def is_str_list(val: List[object]) -> TypeGuard[List[str]]: - """Determines whether all objects in the list are strings""" - return all(isinstance(x, str) for x in val) - - def func1(val: List[object]) -> None: - if is_str_list(val): - reveal_type(val) # List[str] - print(" ".join(val)) # ok - -How does it work? ``TypeGuard`` narrows the first function argument (``val``) -to the type specified as the first type parameter (``List[str]``). - -.. note:: - - Narrowing is - `not strict `_. - For example, you can narrow ``str`` to ``int``: - - .. code-block:: python - - def f(value: str) -> TypeGuard[int]: - return True - - Note: since strict narrowing is not enforced, it's easy - to break type safety. - - However, there are many ways a determined or uninformed developer can - subvert type safety -- most commonly by using cast or Any. - If a Python developer takes the time to learn about and implement - user-defined type guards within their code, - it is safe to assume that they are interested in type safety - and will not write their type guard functions in a way - that will undermine type safety or produce nonsensical results. - -Generic TypeGuards ------------------- - -``TypeGuard`` can also work with generic types: - -.. code-block:: python - - from typing import Tuple, TypeVar - from typing import TypeGuard # use `typing_extensions` for `python<3.10` - - _T = TypeVar("_T") - - def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]: - return len(val) == 2 - - def func(names: Tuple[str, ...]): - if is_two_element_tuple(names): - reveal_type(names) # Tuple[str, str] - else: - reveal_type(names) # Tuple[str, ...] - -Typeguards with parameters --------------------------- - -Type guard functions can accept extra arguments: - -.. code-block:: python - - from typing import Type, Set, TypeVar - from typing import TypeGuard # use `typing_extensions` for `python<3.10` - - _T = TypeVar("_T") - - def is_set_of(val: Set[Any], type: Type[_T]) -> TypeGuard[Set[_T]]: - return all(isinstance(x, type) for x in val) - - items: Set[Any] - if is_set_of(items, str): - reveal_type(items) # Set[str] - -TypeGuards as methods ---------------------- - - A method can also serve as the ``TypeGuard``: - -.. code-block:: python - - class StrValidator: - def is_valid(self, instance: object) -> TypeGuard[str]: - return isinstance(instance, str) - - def func(to_validate: object) -> None: - if StrValidator().is_valid(to_validate): - reveal_type(to_validate) # Revealed type is "builtins.str" - -.. note:: - - Note, that ``TypeGuard`` - `does not narrow `_ - types of ``self`` or ``cls`` implicit arguments. - - If narrowing of ``self`` or ``cls`` is required, - the value can be passed as an explicit argument to a type guard function: - - .. code-block:: python - - class Parent: - def method(self) -> None: - reveal_type(self) # Revealed type is "Parent" - if is_child(self): - reveal_type(self) # Revealed type is "Child" - - class Child(Parent): - ... - - def is_child(instance: Parent) -> TypeGuard[Child]: - return isinstance(instance, Child) diff --git a/docs/source/runtime_troubles.rst b/docs/source/runtime_troubles.rst index 515a7985dcbe..863d0c9c85eb 100644 --- a/docs/source/runtime_troubles.rst +++ b/docs/source/runtime_troubles.rst @@ -62,7 +62,7 @@ required to be valid Python syntax. For more details, see :pep:`563`. of forward references or generics in: * :ref:`type aliases `; - * :ref:`casts `; + * :ref:`type narrowing `; * type definitions (see :py:class:`~typing.TypeVar`, :py:func:`~typing.NewType`, :py:class:`~typing.NamedTuple`); * base classes. diff --git a/docs/source/type_narrowing.rst b/docs/source/type_narrowing.rst new file mode 100644 index 000000000000..25d6629aa694 --- /dev/null +++ b/docs/source/type_narrowing.rst @@ -0,0 +1,297 @@ +.. _type-narrowing: + +Type narrowing +============== + +This section is dedicated to several type narrowing +techniques which are supported by mypy. + +Type narrowing is when you convince a type checker that a broader type is actually more specific, for instance, that an object of type ``Shape`` is actually of the narrower type ``Square``. + + +Type narrowing expressions +-------------------------- + +The simplest way to narrow a type is to use one of the supported expressions: + +- :py:func:`isinstance` like in ``isinstance(obj, float)`` will narrow ``obj`` to have ``float`` type +- :py:func:`issubclass` like in ``issubclass(cls, MyClass)`` will narrow ``cls`` to be ``Type[MyClass]`` +- :py:func:`type` like in ``type(obj) is int`` will narrow ``obj`` to have ``int`` type + +Type narrowing is contextual. For example, based on the condition, mypy will narrow an expression only within an ``if`` branch: + +.. code-block:: python + + def function(arg: object): + if isinstance(arg, int): + # Type is narrowed within the ``if`` branch only + reveal_type(arg) # Revealed type: "builtins.int" + elif isinstance(arg, str) or isinstance(arg, bool): + # Type is narrowed differently within this ``elif`` branch: + reveal_type(arg) # Revealed type: "builtins.str | builtins.bool" + + # Subsequent narrowing operations will narrow the type further + if isinstance(arg, bool): + reveal_type(arg) # Revealed type: "builtins.bool" + + # Back outside of the ``if`` statement, the type isn't narrowed: + reveal_type(arg) # Revealed type: "builtins.object" + +Mypy understands the implications `return` or exception raising can have for what type an object could be: + +.. code-block:: python + + def function(arg: int | str): + if isinstance(arg, int): + return + + # `arg` can't be `int` at this point: + reveal_type(arg) # Revealed type: "builtins.str" + +We can also use ``assert`` to narrow types in the same context: + +.. code-block:: python + + def function(arg: Any): + assert isinstance(arg, int) + reveal_type(arg) # Revealed type: "builtins.int" + +Mypy can also use :py:func:`issubclass` +for better type inference when working with types and metaclasses: + +.. code-block:: python + + class MyCalcMeta(type): + @classmethod + def calc(cls) -> int: + ... + + def f(o: object) -> None: + t = type(o) # We must use a variable here + reveal_type(t) # Revealed type is "builtins.type" + + if issubtype(t, MyCalcMeta): # `issubtype(type(o), MyCalcMeta)` won't work + reveal_type(t) # Revealed type is "Type[MyCalcMeta]" + t.calc() # Okay + +.. note:: + + With :option:`--warn-unreachable ` + narrowing types to some impossible state will be treated as an error. + + .. code-block:: python + + def function(arg: int): + # error: Subclass of "int" and "str" cannot exist: + # would have incompatible method signatures + assert isinstance(arg, str) + + # error: Statement is unreachable + print("so mypy concludes the assert will always trigger") + + Without ``--warn-unreachable`` mypy will simply not check code it deems to be + unreachable. See :ref:`unreachable` for more information. + + .. code-block:: python + + x: int = 1 + assert isinstance(x, str) + reveal_type(x) # Revealed type is "builtins.int" + print(x + '!') # Typechecks with `mypy`, but fails in runtime. + + +.. _casts: + +Casts +----- + +Mypy supports type casts that are usually used to coerce a statically +typed value to a subtype. Unlike languages such as Java or C#, +however, mypy casts are only used as hints for the type checker, and they +don't perform a runtime type check. Use the function :py:func:`~typing.cast` +to perform a cast: + +.. code-block:: python + + from typing import cast, List + + o: object = [1] + x = cast(List[int], o) # OK + y = cast(List[str], o) # OK (cast performs no actual runtime check) + +To support runtime checking of casts such as the above, we'd have to check +the types of all list items, which would be very inefficient for large lists. +Casts are used to silence spurious +type checker warnings and give the type checker a little help when it can't +quite understand what is going on. + +.. note:: + + You can use an assertion if you want to perform an actual runtime check: + + .. code-block:: python + + def foo(o: object) -> None: + print(o + 5) # Error: can't add 'object' and 'int' + assert isinstance(o, int) + print(o + 5) # OK: type of 'o' is 'int' here + +You don't need a cast for expressions with type ``Any``, or when +assigning to a variable with type ``Any``, as was explained earlier. +You can also use ``Any`` as the cast target type -- this lets you perform +any operations on the result. For example: + +.. code-block:: python + + from typing import cast, Any + + x = 1 + x.whatever() # Type check error + y = cast(Any, x) + y.whatever() # Type check OK (runtime error) + + +.. _type-guards: + +User-Defined Type Guards +------------------------ + +Mypy supports User-Defined Type Guards (:pep:`647`). + +A type guard is a way for programs to influence conditional +type narrowing employed by a type checker based on runtime checks. + +Basically, a ``TypeGuard`` is a "smart" alias for a ``bool`` type. +Let's have a look at the regular ``bool`` example: + +.. code-block:: python + + from typing import List + + def is_str_list(val: List[object]) -> bool: + """Determines whether all objects in the list are strings""" + return all(isinstance(x, str) for x in val) + + def func1(val: List[object]) -> None: + if is_str_list(val): + reveal_type(val) # Reveals List[object] + print(" ".join(val)) # Error: incompatible type + +The same example with ``TypeGuard``: + +.. code-block:: python + + from typing import List + from typing import TypeGuard # use `typing_extensions` for Python 3.9 and below + + def is_str_list(val: List[object]) -> TypeGuard[List[str]]: + """Determines whether all objects in the list are strings""" + return all(isinstance(x, str) for x in val) + + def func1(val: List[object]) -> None: + if is_str_list(val): + reveal_type(val) # List[str] + print(" ".join(val)) # ok + +How does it work? ``TypeGuard`` narrows the first function argument (``val``) +to the type specified as the first type parameter (``List[str]``). + +.. note:: + + Narrowing is + `not strict `_. + For example, you can narrow ``str`` to ``int``: + + .. code-block:: python + + def f(value: str) -> TypeGuard[int]: + return True + + Note: since strict narrowing is not enforced, it's easy + to break type safety. + + However, there are many ways a determined or uninformed developer can + subvert type safety -- most commonly by using cast or Any. + If a Python developer takes the time to learn about and implement + user-defined type guards within their code, + it is safe to assume that they are interested in type safety + and will not write their type guard functions in a way + that will undermine type safety or produce nonsensical results. + +Generic TypeGuards +~~~~~~~~~~~~~~~~~~ + +``TypeGuard`` can also work with generic types: + +.. code-block:: python + + from typing import Tuple, TypeVar + from typing import TypeGuard # use `typing_extensions` for `python<3.10` + + _T = TypeVar("_T") + + def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]: + return len(val) == 2 + + def func(names: Tuple[str, ...]): + if is_two_element_tuple(names): + reveal_type(names) # Tuple[str, str] + else: + reveal_type(names) # Tuple[str, ...] + +Typeguards with parameters +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Type guard functions can accept extra arguments: + +.. code-block:: python + + from typing import Type, Set, TypeVar + from typing import TypeGuard # use `typing_extensions` for `python<3.10` + + _T = TypeVar("_T") + + def is_set_of(val: Set[Any], type: Type[_T]) -> TypeGuard[Set[_T]]: + return all(isinstance(x, type) for x in val) + + items: Set[Any] + if is_set_of(items, str): + reveal_type(items) # Set[str] + +TypeGuards as methods +~~~~~~~~~~~~~~~~~~~~~ + + A method can also serve as the ``TypeGuard``: + +.. code-block:: python + + class StrValidator: + def is_valid(self, instance: object) -> TypeGuard[str]: + return isinstance(instance, str) + + def func(to_validate: object) -> None: + if StrValidator().is_valid(to_validate): + reveal_type(to_validate) # Revealed type is "builtins.str" + +.. note:: + + Note, that ``TypeGuard`` + `does not narrow `_ + types of ``self`` or ``cls`` implicit arguments. + + If narrowing of ``self`` or ``cls`` is required, + the value can be passed as an explicit argument to a type guard function: + + .. code-block:: python + + class Parent: + def method(self) -> None: + reveal_type(self) # Revealed type is "Parent" + if is_child(self): + reveal_type(self) # Revealed type is "Child" + + class Child(Parent): + ... + + def is_child(instance: Parent) -> TypeGuard[Child]: + return isinstance(instance, Child)