Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PEP 646 implementation #963

Merged
merged 27 commits into from Feb 12, 2022
Merged

PEP 646 implementation #963

merged 27 commits into from Feb 12, 2022

Conversation

JelleZijlstra
Copy link
Member

@JelleZijlstra JelleZijlstra commented Nov 24, 2021

Fixes python/typing_extensions#28.

This turned out to require even more hacks than usual, because the runtime in old versions of typing really doesn't want things that are not TypeVars to be used as TypeVars.

The tests were copied from https://github.com/python/cpython/pull/24527/files and the implementation is mostly inspired by Required/NotRequired (Unpack) and ParamSpec (TypeVarTuple).

@JelleZijlstra
Copy link
Member Author

The last 3.6 failures are annoying and I don't see an easy fix, I think I'll give it another try tomorrow.

@JelleZijlstra JelleZijlstra marked this pull request as ready for review January 15, 2022 22:45
@JelleZijlstra
Copy link
Member Author

This is now finally ready for review. This is uncommonly hacky even by typing-extensions standards, because the older versions of typing only know about TypeVars and now we need to convince them to somehow also accept unpacked TypeVarTuples. I solved this with monkeypatching (yuck).

For 3.6, which had a very different internal implementation of Generic, it's still not enough. Given that 3.6 is EOL and most usage of TypeVarTuple/Unpack actually works correctly, I decided to just leave this as a documented limitation.

@JelleZijlstra JelleZijlstra changed the title [WIP] PEP 646 implementation PEP 646 implementation Jan 15, 2022
@Fidget-Spinner
Copy link
Member

Fidget-Spinner commented Jan 23, 2022

@JelleZijlstra Sorry I'll try to review this in the next week or two, swamped right now. Ping me again if I forget please!

@JelleZijlstra
Copy link
Member Author

No hurry, take your time!

Copy link
Member

@Fidget-Spinner Fidget-Spinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in general. Thanks Jelle! BTW, should I be reviewing this or Mathew's CPython PR first? I'd imagine any tweaks to his PR would affect yours.

typing_extensions/src/test_typing_extensions.py Outdated Show resolved Hide resolved
typing_extensions/src/typing_extensions.py Outdated Show resolved Hide resolved
_marker = object()


def _check_generic(cls, parameters, elen=_marker):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Mathew pointed out in his documenting typing.py PR that parameters is confusing. Should we use args or type_args instead?

Also what's elen's default for? When would we not want to pass in __parameters__ ourselves?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a modified copy of typing.py's _check_generic, and it's monkeypatched into typing later. We probably shouldn't modify the function's interface, because that makes the monkeypatching even riskier.

- ``ParamSpec`` and ``Concatenate`` will not work with ``get_args`` and
``get_origin``. Certain PEP 612 special cases in user-defined
``Generic``\ s are also not available.
- ``Unpack`` from PEP 646 does not work properly with user-defined
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify, this means even with our monkey patching, subscripting with arbitrary number of type arguments won't work right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defining the class doesn't even work. I'll make the note more precise.

Python 3.6.10 (default, Apr 28 2021, 18:14:25) 
[GCC Apple LLVM 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import Generic
>>> from typing_extensions import Unpack, TypeVarTuple
>>> Ts = TypeVarTuple("Ts")
>>> class X(Generic[Unpack[Ts]]): pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jelle/.pyenv/versions/3.6.10/lib/python3.6/typing.py", line 965, in __new__
    ", ".join(str(g) for g in gvars)))
TypeError: Some type variables (Ts) are not listed in Generic[typing_extensions.Unpack[Ts]]

@JelleZijlstra
Copy link
Member Author

Thanks! I think it's better to focus on this first, since 3.11 has months of time left in any case, but with typing-extensions we can release at any time.

However, I'll look at his current PR for things to improve here; I think he wrote better docstrings.

Copy link
Collaborator

@srittau srittau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't claim to really understand what is going on with typing internals, but I see no red flags.

@JelleZijlstra JelleZijlstra merged commit 9403ccf into python:master Feb 12, 2022
@JelleZijlstra JelleZijlstra deleted the pep646 branch February 12, 2022 01:04
@mrahtz
Copy link

mrahtz commented Feb 13, 2022

Thanks all - especially @JelleZijlstra!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement PEP 646 in typing_extensions
5 participants