feat: support generic types and | operator

[PrettyWood]

Change Summary

Now this is possible with python 3.7+

from __future__ import annotations

from pydantic import BaseModel, Field

class User(BaseModel, validate_assignment=True):
    first_name: int | str
    last_name: str | None = None
    friends: list[User] = Field(default_factory=list)
    metadata: dict[str, int] | None = None

user = User(first_name='pika', last_name=666)

assert user.last_name == '666'
assert user.friends == []
user.metadata = {b'1': b'1'}
assert m.user.metadata == {'1': 1}

Related issue number

Checklist

• Unit tests for the changes exist
• Tests pass on CI and coverage remains at 100%
• Documentation reflects the changes where applicable
• changes/<pull request or issue id>-<github username>.md file added describing change
  (see changes/README.md for details)

[samuelcolvin]

I'm assuming this is still a work in progress? let me know when you want me to review it.

(please update (just for the bot to assign you 😄))

[samuelcolvin]

I've just had a quick read of this, it looks great to me.

[samuelcolvin]

If we go with this, I think we'll need a more clear name for this extra.

[PrettyWood]

At first I did something like

all_typing_mod_name = 'pydantic_all_typing___'
all_typing_mod = types.ModuleType(all_typing_mod_name)
exec("from typing import *\nfrom typing_extensions import *", all_typing_mod.__dict__)

But since using eval() is problematic with cython, I went with this approach.

[PrettyWood]

I wrote some documentation. Feedback welcome! Please review

[PrettyWood]

Let's focus on 3.10 and see if people actually want this.
With the switch from PEP 563 to 649, I don't know how it will affect the future.annotations.
And supporting only generics and unions but not some types or forwardrefs may create more support.
I tag this PR as deferred in the meantime

[elyase]

I was surprised to see that pydantic doesnt support the | operator. Would love to have this merged.

[tuchandra]

Let's focus on 3.10 and see if people actually want this. With the switch from PEP 563 to 649, I don't know how it will affect the future.annotations. And supporting only generics and unions but not some types or forwardrefs may create more support. I tag this PR as deferred in the meantime

Hi @PrettyWood - I'd like to give a +1 in support of landing this if still possible. I think the support for unions & generics will cover 80+% of cases; it'd be nice to write x: str | None, and I suspect that the majority of the utility comes from simple stuff like this.

Forward refs in particular have always been a little bit of a thorn (even with the stdlib), and so I don't think it needs to block something like this that supports the simpler cases.

[PrettyWood]

@tuchandra If you use 3.10, it's already supported. This PR is to have a workaround for python 3.7+ using from __future__ import annotations but I don't think it's worth merging it.

[tuchandra]

@PrettyWood sorry, I wasn't clear - yeah, I'm on 3.8/3.9 but I understand not merging it. Thanks for the reply!

[samuelcolvin]

@tuchandra If you use 3.10, it's already supported. This PR is to have a workaround for python 3.7+ using from __future__ import annotations but I don't think it's worth merging it.

agreed

[eseglem]

I understand not merging as well but I would certainly love to see something like this actually get merged. Its not great but unfortunately I have systems stuck with 3.8 for the foreseeable future and it would be nice to use the newer syntax.

It could also be useful for packages that use Pydantic which still support older versions of python as well.

But honestly, it probably better to focus on the future.

[samuelcolvin]

I really think this would have been too confusing for users, we were right to drop it.

On a related note, see #4339.