Type hint issue with pyright and Pydantic 1.9.0 for nested models.

[haoyun]

Checks

• I added a descriptive title to this issue
• I have searched (google, github) for similar issues and couldn't find anything
• I have read and followed the docs and still think this is a bug

(might not be a) Bug

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

             pydantic version: 1.9.0
            pydantic compiled: False
                 install path: /home/yun/.conda/envs/test/lib/python3.8/site-packages/pydantic
               python version: 3.8.12 (default, Oct 12 2021, 13:49:34)  [GCC 7.5.0]
                     platform: Linux-5.15.14-200.fc35.x86_64-x86_64-with-glibc2.17
     optional deps. installed: ['typing-extensions']

The following code is a modification of the example in https://pydantic-docs.helpmanual.io/usage/types/#namedtuple

from typing import NamedTuple

from pydantic import BaseModel


class Point(NamedTuple):
    x: int
    y: int


class Point2(BaseModel):
    x: int
    y: int


class Model(BaseModel):
    p: Point


class Model2(BaseModel):
    p: Point2


print(Model(p=("1", "2")))
# > p=Point(x=1, y=2)

print(Model2(**{"p": {"x": "1", "y": "2"}}))
# > p=Point2(x=1, y=2)

Code works as expected, however in the editor with pyright enabled, there are the following errors:

[Pyright reportGeneralTypeIssues] [E] Argument of type "tuple[Literal['1'], Literal['2']]" cannot be assigned to parameter "p" of type "Point" in function "__init__"
  "tuple[Literal['1'], Literal['2']]" is incompatible with "Point"

[Pyright reportGeneralTypeIssues] [E] Argument of type "dict[str, str]" cannot be assigned to parameter "p" of type "Point2" in function "__init__"
  "dict[str, str]" is incompatible with "Point2"

Actually, pyright is correct on this. Now with 1.9.0, Model has hint (class) Model(*, p: Point), while in, for example, 1.7.4, it has

(class) Model(**data: Any)
---

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

With this more precise typing p: Point than **data: Any, p has to be Point(x=1, y=2) but not ('1', '2'). Moreover, the docstring is missing.

With 1.9.0, I have to use

print(Model.parse_obj({'p': ('1', '2')}))
# > p=Point(x=1, y=2)

to avoid the above errors given by the editor.

Moreover, with the current precise typing, all optional field becomes required optional for type checker, see for example #3710, unless set a default value to be None explicitly.

class A(BaseModel):
    a: Optional[int]

A()  # [E] Argument missing for parameter "a"


class B(BaseModel):
    b: Optional[int] = None
B()  # works

class C(BaseModel):
    c: Optional[int] = Field(None)

C()  # [E] Argument missing for parameter "c"

[tuchandra]

The first part — model initialization not accepting your args — is a consequence of how pyright handles pydantic models. See the Visual Studio Code docs page for more—it's a very good explanation.

In summary, as you've noted, pyright wouldn't do any kind of type checking on the model constructors. This is because it didn't know how to—BaseModel has a lot of 'magic' implemented under the hood, mostly in its metaclass—and this kind of dynamic transformation is near impossible for type checkers to understand.

v1.9.0 changed this via the addition of dataclass_transform in #2721. That PR allows pydantic to mark the BaseModel as something that's 'kind of like a dataclass' in a way that pyright understands (see the spec discussion here). In practice, this mostly means that pyright treats BaseModel like a dataclass — and so you can't initialize models using other types, even though pydantic does its best to coerce things into the model's format.

Ways around it include:

• doing the type transformation yourself, if feasible
• override the type with cast(val, type)
• use parse_obj, which still takes the arg Any
• use #type:ignore on a line

---

The second part about Optional fields is, as far as I can tell, a bug. I've noticed it on my team's codebase as well.

[samuelcolvin]

Thanks for the great explanation @tuchandra.

Also see #3972