Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specialization of namedtuple types #92107

Closed
serhiy-storchaka opened this issue May 1, 2022 · 8 comments
Closed

Specialization of namedtuple types #92107

serhiy-storchaka opened this issue May 1, 2022 · 8 comments
Labels
topic-typing type-feature A feature request or enhancement

Comments

@serhiy-storchaka
Copy link
Member

collections.namedtuple() creates a tuple subclass, and as a subclass it inherits all tuple methods, including __class_getitem__.

The __new__ method has been overridden in a named tuple class. It no longer accept an iterator of arbitrary length. The named tuple has fixed number of elements, and the constructor accept fixed number of arguments (some may be optional).

But the __class_getitem__ is inherited from tuple. It allows to specialize the named tuple type with arbitrary number of arguments.

Point2D = namedtuple('Point2D', 'x y label')
Point2D[int, int, str]
Point2D[int, str]
Point2D[int, ...]

It looks confusing at best. A named tuple is not a general tuple, it cannot have an arbitrary number of items.

There are two options:

  1. Forbid subscription of types created by namedtuple(). If you want to add type annotations, use typing.NamedTuple.
  2. Add __class_getitem__ which checks that the number of types corresponds to the size of the named tuple.

@rhettinger

@serhiy-storchaka serhiy-storchaka added type-feature A feature request or enhancement topic-typing labels May 1, 2022
@serhiy-storchaka
Copy link
Member Author

See also #92106 and #88089.

@AlexWaygood
Copy link
Member

I'm not sure that it is a good idea to forbid parameterisation of collections.namedtuple at runtime.

The standard library has quite a few uses of collections.namedtuple. For almost all of these, typeshed pretends that they are actually typing.NamedTuples, since using collections.namedtuple in a stub gives the type checker very little information to work with.

If #92027 is merged, type checkers will be given the freedom to implement support for generic NamedTuples. If they do implement support, we'll want to make several of these stdlib namedtuple-at-runtime-but-NamedTuple-in-the-stub classes generic. concurrent.futures.DoneAndNotDoneFutures is an obvious example of a collections.namedtuple that we need to be generic in the stub, but there will be others as well.

If we forbid parameterisation of collections.namedtuple at runtime, it could cause a lot of user pain if a user comes across a class that forbids parameterisation at runtime, but which typeshed says is a generic typing.NamedTuple.

@AlexWaygood
Copy link
Member

AlexWaygood commented May 1, 2022

There isn't the same issue with forbidding runtime parameterisation of non-generic TypedDicts, as there isn't a "pre-typing" version of the class the same way as there is with typing.NamedTuple

@MojoVampire
Copy link
Contributor

Is there a reason the PR just goes straight for option #1 (forbidding subscription)? Option #2 seems like a "better than nothing" solution, and doesn't risk breaking existing code that makes sense (code that subscripted with a mismatched number of types would break, but that code would be, by definition, incorrect).

@AlexWaygood
Copy link
Member

AlexWaygood commented May 1, 2022

What about a namedtuple like this (for example):

from collections import namedtuple

Foo = namedtuple("Foo", "dict_field list_field")

In typeshed we might want to write a stub for this class like this:

from typing import NamedTuple, Generic, TypeVar

T = TypeVar("T")
U = TypeVar("U")
V = TypeVar("V")

class Foo(NamedTuple, Generic[T, U, V]):
    dict_field: dict[T, U]
    list_field: list[V]

A type checker will expect 3 parameters for this class, even though it only has two fields. So Option 2 would also be complicated to get right.

@JelleZijlstra
Copy link
Member

Agree with @AlexWaygood; it can be useful to subscript collections.namedtuples if they are conceptually generic. We don't need to change anything here.

@serhiy-storchaka
Copy link
Member Author

@AlexWaygood, in your example the named tuple is generic. This issue is about non-generic named tuples.

@AlexWaygood
Copy link
Member

AlexWaygood commented May 2, 2022

@AlexWaygood, in your example the named tuple is generic. This issue is about non-generic named tuples.

Indeed, but that was my point. If there's a class that's a non-generic collections.namedtuple at runtime in the stdlib, but the same class is stubbed as a generic typing.NamedTuple in typeshed, that has potential for causing a large amount of user pain if the type checker expects a user to parameterise the class, but the runtime raises an exception when they try to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-typing type-feature A feature request or enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants