Skip to content

Commit

Permalink
typing.TypedDict x 2.
Browse files Browse the repository at this point in the history 
This commit is the last in a commit chain adding support for
superficially type-checking **typed dictionaries** (i.e.,
`typing.TypedDict` and `typing_extensions.TypedDict` subclasses),
resolving issue #55 kindly submitted by Kyle (@KyleKing), the
undisputed King of Parexel AI Labs. Specifically, this commit:

* Refines our recently added
  `beartype._util.hint.pep.proposal.utilpep593.is_hint_pep593()` tester
  to additionally test for the existence of the `__total__` dunder
  attribute necessarily added to typed dictionaries.
* Improves our private
  `beartype._util.hint.pep.utilpepget.get_hint_pep_sign_or_none()`
  getter to uniquely identify typed dictionaries as `HintSignTypedDict`.
* Exhaustively tests `@beartype` support for typed dictionaries,
  including those defined via both:
  * The official `typing.TypedDict` factory under Python ≥ 3.8.
  * The third-party `typing_extensions.TypedDict` factory under Python
    3.6 and 3.7.

(*Unrequited requiem!*)
  • Loading branch information
@leycec
leycec committed Oct 13, 2021
1 parent 1816402 commit 342e5fc
Show file tree
Hide file tree
Showing 13 changed files with 507 additions and 123 deletions.
10 changes: 8 additions & 2 deletions beartype/_data/hint/pep/sign/datapepsignset.py
View file
Expand Up @@ -64,6 +64,7 @@
HintSignSized,
HintSignTuple,
HintSignType,
HintSignTypedDict,
HintSignTypeVar,
HintSignUnion,
HintSignValuesView,
Expand Down Expand Up @@ -267,12 +268,17 @@
_HINT_SIGNS_SUPPORTED_SHALLOW = frozenset((
# ..................{ PEP 484 }..................
HintSignTypeVar,

# ..................{ PEP 589 }..................
#FIXME: Shift into "HINT_SIGNS_SUPPORTED_DEEP" *AFTER* deeply type-checking
#typed dictionaries.
HintSignTypedDict,
))
'''
Frozen set of all **shallowly supported non-originative signs** (i.e.,
arbitrary objects uniquely identifying PEP-compliant type hints *not*
originating from a non-:mod:`typing` origin type for which the
:func:`beartype.beartype` decorator generates shallow type-checking code).
originating from an isinstanceable type for which the :func:`beartype.beartype`
decorator generates shallow type-checking code).
'''


Expand Down
26 changes: 21 additions & 5 deletions beartype/_util/hint/pep/proposal/pep484585/utilpepgeneric.py
View file
Expand Up @@ -33,11 +33,11 @@
def is_hint_pep484585_generic(hint: object) -> bool:
'''
``True`` only if the passed object is either a :pep:`484`- or
:pep:`585`-compliant **generic** (i.e., class superficially subclassing at
least one PEP-compliant type hint that is possibly *not* an actual class).
:pep:`585`-compliant **generic** (i.e., object that may *not* actually be a
class despite subclassing at least one PEP-compliant type hint that also
may *not* actually be a class).
Specifically, this tester returns ``True`` only if this object is a class
that is either:
Specifically, this tester returns ``True`` only if this object is either:
* A :pep:`585`-compliant generic as tested by the lower-level
:func:`is_hint_pep585_generic` function.
Expand All @@ -49,6 +49,23 @@ def is_hint_pep484585_generic(hint: object) -> bool:
one-liner are non-negligible. Moreover, this tester is called frequently
enough to warrant its reduction to an efficient lookup.
Caveats
----------
**Generics are not necessarily classes,** despite originally being declared
as classes. Although *most* generics are classes, subscripting a generic
class usually produces a generic non-class that *must* nonetheless be
transparently treated as a generic class: e.g.,
.. code-block:: python
>>> from typing import Generic, TypeVar
>>> S = TypeVar('S')
>>> T = TypeVar('T')
>>> class MuhGeneric(Generic[S, T]): pass
>>> non_class_generic = MuhGeneric[S, T]
>>> isinstance(non_class_generic, type)
False
Parameters
----------
hint : object
Expand Down Expand Up @@ -410,7 +427,6 @@ class such that *all* objects satisfying this hint are instances of that
isinstanceable class). Callers *must* perform subsequent tests to
distinguish these two cases.
Parameters
----------
hint : object
Expand Down
71 changes: 28 additions & 43 deletions beartype/_util/hint/pep/proposal/utilpep484.py
View file
Expand Up @@ -299,28 +299,28 @@ def is_hint_pep484_generic(hint: object) -> bool:
# >>> MuhList.__mro__
# (__main__.MuhList, list, typing.Generic, object)
#
# Note that this issubclass() call implicitly performs a surprisingly
# inefficient search over the method resolution order (MRO) of all
# superclasses of this hint. In theory, the cost of this search might
# be circumventable by observing that this ABC is expected to reside at
# the second-to-last index of the tuple exposing this MRO far all
# generics by virtue of fragile implementation details violating
# privacy encapsulation. In practice, this codebase is fragile enough.
#
# Note lastly that the following logic superficially appears to
# implement the same test *WITHOUT* the onerous cost of a search:
# return len(get_hint_pep484_generic_bases_unerased_or_none(hint)) > 0
#
# Why didn't we opt for that, then? Because this tester is routinely
# passed objects that *CANNOT* be guaranteed to be PEP-compliant.
# Indeed, the high-level is_hint_pep() tester establishing the
# PEP-compliance of arbitrary objects internally calls this lower-level
# tester to do so. Since the
# get_hint_pep484_generic_bases_unerased_or_none() getter internally
# reduces to returning the tuple of the general-purpose
# "__orig_bases__" dunder attribute formalized by PEP 560, testing
# whether that tuple is non-empty or not in no way guarantees this
# object to be a PEP-compliant generic.
# Note that:
# * This issubclass() call implicitly performs a surprisingly
# inefficient search over the method resolution order (MRO) of all
# superclasses of this hint. In theory, the cost of this search might
# be circumventable by observing that this ABC is expected to reside
# at the second-to-last index of the tuple exposing this MRO far all
# generics by virtue of fragile implementation details violating
# privacy encapsulation. In practice, this codebase is already
# fragile enough.
# * The following logic superficially appears to implement the same
# test *WITHOUT* the onerous cost of a search:
# return len(get_hint_pep484_generic_bases_unerased_or_none(hint)) > 0
# Why didn't we opt for that, then? Because this tester is routinely
# passed objects that *CANNOT* be guaranteed to be PEP-compliant.
# Indeed, the high-level is_hint_pep() tester establishing the
# PEP-compliance of arbitrary objects internally calls this
# lower-level tester to do so. Since the
# get_hint_pep484_generic_bases_unerased_or_none() getter internally
# reduces to returning the tuple of the general-purpose
# "__orig_bases__" dunder attribute formalized by PEP 560, testing
# whether that tuple is non-empty or not in no way guarantees this
# object to be a PEP-compliant generic.
return is_type_subclass(hint, Generic) # type: ignore[arg-type]
# Else, the active Python interpreter targets Python 3.6.x. In this case,
# implement this function specific to this Python version.
Expand Down Expand Up @@ -378,17 +378,17 @@ def is_hint_pep484_generic(hint: object) -> bool:

# Docstring for this function regardless of implementation details.
is_hint_pep484_generic.__doc__ = '''
``True`` only if the passed object is a :mod:`typing` **generic** (i.e.,
class superficially subclassing at least one non-class PEP-compliant
object defined by the :mod:`typing` module).
``True`` only if the passed object is a :pep:`484`-compliant **generic**
(i.e., object that may *not* actually be a class originally subclassing at
least one PEP-compliant type hint defined by the :mod:`typing` module).
Specifically, this tester returns ``True`` only if this object is a class
subclassing a combination of:
Specifically, this tester returns ``True`` only if this object was
originally defined as a class subclassing a combination of:
* At least one of:
* The :pep:`484`-compliant :mod:`typing.Generic` superclass.
* The `PEP 544`-_compliant :mod:`typing.Protocol` superclass.
* The :pep:`544`-compliant :mod:`typing.Protocol` superclass.
* Zero or more non-class :mod:`typing` pseudo-superclasses (e.g.,
``typing.List[int]``).
Expand All @@ -398,21 +398,6 @@ class superficially subclassing at least one non-class PEP-compliant
:func:`callable_cached` decorator), as the implementation trivially reduces
to an efficient one-liner.
Design
----------
Since *all* :mod:`typing` generics subclass the :pep:`484`-compliant
:mod:`typing.Generic` superclass first introduced with :pep:`484`, this
tester is intentionally:
* Defined in the :pep:`484`-specific submodule rather than either the `PEP
585`_-specific submodule *or* higher-level PEP-agnostic test submodule.
* Named ``is_hint_pep484_generic`` rather than
``is_hint_pep484or544_generic`` or ``is_hint_pep_typing_generic``.
From the end user perspective, *all* :mod:`typing` generics are effectively
indistinguishable from :pep:`484`-compliant generics and should typically
be generically treated as such.
Parameters
----------
hint : object
Expand Down
54 changes: 27 additions & 27 deletions beartype/_util/hint/pep/proposal/utilpep585.py
View file
Expand Up @@ -4,7 +4,7 @@
# See "LICENSE" for further details.

'''
Project-wide :pep:`585`:-compliant type hint utilities.
Project-wide :pep:`585`-compliant type hint utilities.
This private submodule is *not* intended for importation by downstream callers.
'''
Expand Down Expand Up @@ -34,9 +34,9 @@
# ....................{ VALIDATORS }....................
def die_unless_hint_pep585_generic(hint: object) -> None:
'''
Raise an exception unless the passed object is a :pep:`585`:-compliant
Raise an exception unless the passed object is a :pep:`585`-compliant
**generic** (i.e., class superficially subclassing at least one subscripted
:pep:`585`:-compliant pseudo-superclass).
:pep:`585`-compliant pseudo-superclass).
Parameters
----------
Expand All @@ -46,7 +46,7 @@ def die_unless_hint_pep585_generic(hint: object) -> None:
Raises
----------
BeartypeDecorHintPep585Exception
If this hint is *not* a :pep:`585`:-compliant generic.
If this hint is *not* a :pep:`585`-compliant generic.
'''

# If this hint is *NOT* a PEP 585-compliant generic, raise an exception
Expand Down Expand Up @@ -134,7 +134,7 @@ def is_hint_pep585_generic(hint: object) -> bool:
# ....................{ TESTERS ~ doc }....................
# Docstring for this function regardless of implementation details.
is_hint_pep585_builtin.__doc__ = '''
``True`` only if the passed object is a C-based :pep:`585`:-compliant
``True`` only if the passed object is a C-based :pep:`585`-compliant
**builtin type hint** (i.e., C-based type hint instantiated by subscripting
either a concrete builtin container class like :class:`list` or
:class:`tuple` *or* an abstract base class (ABC) declared by the
Expand All @@ -147,18 +147,18 @@ def is_hint_pep585_generic(hint: object) -> bool:
Caveats
----------
**This test returns false for** :pep:`585`:-compliant **generics,** which
fail to satisfy the same API as all other :pep:`585`:-compliant type hints.
**This test returns false for** :pep:`585`-compliant **generics,** which
fail to satisfy the same API as all other :pep:`585`-compliant type hints.
Why? Because :pep:`560`-type erasure erases this API on
:pep:`585`-compliant generics immediately after those generics are
declared, preventing their subsequent detection as :pep:`585`:-compliant.
Instead, :pep:`585`:-compliant generics are only detectable by calling
declared, preventing their subsequent detection as :pep:`585`-compliant.
Instead, :pep:`585`-compliant generics are only detectable by calling
either:
* The high-level PEP-agnostic
:func:`beartype._util.hint.pep.utilpeptest.is_hint_pep484585_generic`
tester.
* The low-level :pep:`585`:-specific :func:`is_hint_pep585_generic` tester.
* The low-level :pep:`585`-specific :func:`is_hint_pep585_generic` tester.
Parameters
----------
Expand All @@ -168,14 +168,14 @@ def is_hint_pep585_generic(hint: object) -> bool:
Returns
----------
bool
``True`` only if this object is a :pep:`585`:-compliant type hint.
``True`` only if this object is a :pep:`585`-compliant type hint.
'''


is_hint_pep585_generic.__doc__ = '''
``True`` only if the passed object is a :pep:`585`:-compliant **generic**
(i.e., class superficially subclassing at least one subscripted `PEP
585`_-compliant pseudo-superclass).
``True`` only if the passed object is a :pep:`585`-compliant **generic**
(i.e., object that may *not* actually be a class originally subclassing at
least one subscripted :pep:`585`-compliant pseudo-superclass).
This tester is memoized for efficiency.
Expand All @@ -187,17 +187,17 @@ def is_hint_pep585_generic(hint: object) -> bool:
Returns
----------
bool
``True`` only if this object is a :pep:`585`:-compliant generic.
``True`` only if this object is a :pep:`585`-compliant generic.
'''

# ....................{ GETTERS }....................
def get_hint_pep585_generic_bases_unerased(hint: Any) -> tuple:
'''
Tuple of all unerased :pep:`585`:-compliant **pseudo-superclasses** (i.e.,
Tuple of all unerased :pep:`585`-compliant **pseudo-superclasses** (i.e.,
:mod:`typing` objects originally listed as superclasses prior to their
implicit type erasure under `PEP 560`_) of the passed `PEP 585`-compliant
**generic** (i.e., class subclassing at least one non-class `PEP
585`-compliant object).
implicit type erasure under :pep:`560`) of the passed :pep:`585`-compliant
**generic** (i.e., class subclassing at least one non-class
:pep:`585`-compliant object).
This getter is intentionally *not* memoized (e.g., by the
:func:`callable_cached` decorator), as the implementation trivially reduces
Expand All @@ -212,12 +212,12 @@ def get_hint_pep585_generic_bases_unerased(hint: Any) -> tuple:
----------
Tuple[object]
Tuple of the one or more unerased pseudo-superclasses of this
:pep:`585`:-compliant generic.
:pep:`585`-compliant generic.
Raises
----------
BeartypeDecorHintPep585Exception
If this hint is *not* a :pep:`585`:-compliant generic.
If this hint is *not* a :pep:`585`-compliant generic.
See Also
----------
Expand Down Expand Up @@ -247,20 +247,20 @@ def get_hint_pep585_generic_bases_unerased(hint: Any) -> tuple:
def get_hint_pep585_generic_typevars(hint: object) -> TupleTypes:
'''
Tuple of all **unique type variables** (i.e., subscripted :class:`TypeVar`
instances of the passed :pep:`585`:-compliant generic listed by the caller
instances of the passed :pep:`585`-compliant generic listed by the caller
at hint declaration time ignoring duplicates) if any *or* the empty tuple
otherwise.
This getter is memoized for efficiency.
Motivation
----------
The current implementation of :pep:`585`: under at least Python 3.9 is
The current implementation of :pep:`585` under at least Python 3.9 is
fundamentally broken with respect to parametrized generics. While `PEP
484`_-compliant generics properly propagate type variables from
pseudo-superclasses to subclasses, :pep:`585`: fails to do so. This function
pseudo-superclasses to subclasses, :pep:`585` fails to do so. This function
"fills in the gaps" by recovering these type variables from parametrized
:pep:`585`:-compliant generics by iteratively constructing a new tuple from
:pep:`585`-compliant generics by iteratively constructing a new tuple from
the type variables parametrizing all pseudo-superclasses of this generic.
Parameters
Expand All @@ -273,14 +273,14 @@ def get_hint_pep585_generic_typevars(hint: object) -> TupleTypes:
Tuple[TypeVar, ...]
Either:
* If this :pep:`585`:-compliant generic defines a ``__parameters__``
* If this :pep:`585`-compliant generic defines a ``__parameters__``
dunder attribute, the value of that attribute.
* Else, the empty tuple.
Raises
----------
BeartypeDecorHintPep585Exception
If this hint is *not* a :pep:`585`:-compliant generic.
If this hint is *not* a :pep:`585`-compliant generic.
'''

# Avoid circular import dependencies.
Expand Down

0 comments on commit 342e5fc

Please sign in to comment.