New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
✨ Add autocomplete support for VS Code, via dataclass_transform
#2721
Changes from all commits
a8e2528
88fe649
d4c57c9
7c0adbb
c926ed4
a56e3e1
1bed0f7
f5fa6a2
ff06395
e9953a3
a379b86
a22eee9
3ceab48
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Add support for autocomplete in VS Code via `__dataclass_transform__` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,269 @@ | ||
*pydantic* works well with any editor or IDE out of the box because it's made on top of standard Python type annotations. | ||
|
||
When using [Visual Studio Code (VS Code)](https://code.visualstudio.com/), there are some **additional editor features** supported, comparable to the ones provided by the [PyCharm plugin](./pycharm_plugin.md). | ||
|
||
This means that you will have **autocompletion** (or "IntelliSense") and **error checks** for types and required arguments even while creating new *pydantic* model instances. | ||
|
||
![pydantic autocompletion in VS Code](./img/vs_code_01.png) | ||
|
||
## Configure VS Code | ||
|
||
To take advantage of these features, you need to make sure you configure VS Code correctly, using the recommended settings. | ||
|
||
In case you have a different configuration, here's a short overview of the steps. | ||
|
||
### Install Pylance | ||
|
||
You should use the [Pylance](https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-pylance) extension for VS Code. It is the recommended, next-generation, official VS Code plug-in for Python. | ||
|
||
Pylance is installed as part of the [Python Extension for VS Code](https://marketplace.visualstudio.com/items?itemName=ms-python.python) by default, so it should probably just work. Otherwise, you can double check it's installed and enabled in your editor. | ||
|
||
### Configure your environment | ||
|
||
Then you need to make sure your editor knows the [Python environment](https://code.visualstudio.com/docs/python/python-tutorial#_install-and-use-packages) (probably a virtual environment) for your Python project. | ||
|
||
This would be the environment in where you installed *pydantic*. | ||
|
||
### Configure Pylance | ||
|
||
With the default configurations, you will get support for autocompletion, but Pylance might not check for type errors. | ||
|
||
You can enable type error checks from Pylance with these steps: | ||
|
||
* Open the "User Settings" | ||
* Search for `Type Checking Mode` | ||
* You will find an option under `Python › Analysis: Type Checking Mode` | ||
* Set it to `basic` or `strict` (by default it's `off`) | ||
|
||
![Type Checking Mode set to strict in VS Code](./img/vs_code_02.png) | ||
|
||
Now you will not only get autocompletion when creating new *pydantic* model instances but also error checks for **required arguments**. | ||
|
||
![Required arguments error checks in VS Code](./img/vs_code_03.png) | ||
|
||
And you will also get error checks for **invalid data types**. | ||
|
||
![Invalid data types error checks in VS Code](./img/vs_code_04.png) | ||
|
||
!!! note "Technical Details" | ||
Pylance is the VS Code extension, it's closed source, but free to use. Underneath, Pylance uses an open source tool (also from Microsoft) called [Pyright](https://github.com/microsoft/pyright) that does all the heavy lifting. | ||
|
||
You can read more about it in the [Pylance Frequently Asked Questions](https://github.com/microsoft/pylance-release/blob/main/FAQ.md#what-is-the-relationship-between-pylance-pyright-and-the-python-extension). | ||
|
||
### Configure mypy | ||
|
||
You might also want to configure mypy in VS Code to get mypy error checks inline in your editor (alternatively/additionally to Pylance). | ||
|
||
This would include the errors detected by the [*pydantic* mypy plugin](./mypy_plugin.md), if you configured it. | ||
|
||
To enable mypy in VS Code, do the following: | ||
|
||
* Open the "User Settings" | ||
* Search for `Mypy Enabled` | ||
* You will find an option under `Python › Linting: Mypy Enabled` | ||
* Check the box (by default it's unchecked) | ||
|
||
![mypy enabled in VS Code](./img/vs_code_05.png) | ||
|
||
## Tips and tricks | ||
|
||
Here are some additional tips and tricks to improve your developer experience when using VS Code with *pydantic*. | ||
|
||
### Strict errors | ||
|
||
The way this additional editor support works is that Pylance will treat your *pydantic* models as if they were Python's pure `dataclasses`. | ||
|
||
And it will show **strict type error checks** about the data types passed in arguments when creating a new *pydantic* model instance. | ||
|
||
In this example you can see that it shows that a `str` of `'23'` is not a valid `int` for the argument `age`. | ||
|
||
![VS Code strict type errors](./img/vs_code_06.png) | ||
|
||
It would expect `age=23` instead of `age='23'`. | ||
|
||
Nevertheless, the design, and one of the main features of *pydantic*, is that it is very **lenient with data types**. | ||
|
||
It will actually accept the `str` with value `'23'` and will convert it to an `int` with value `23`. | ||
|
||
These strict error checks are **very useful** most of the time and can help you **detect many bugs early**. But there are cases, like with `age='23'`, where they could be inconvenient by reporting a "false positive" error. | ||
|
||
--- | ||
|
||
This example above with `age='23'` is intentionally simple, to show the error and the differences in types. | ||
|
||
But more common cases where these strict errors would be inconvenient would be when using more sophisticated data types, like `int` values for `datetime` fields, or `dict` values for *pydantic* sub-models. | ||
|
||
For example, this is valid for *pydantic*: | ||
|
||
```Python hl_lines="12 17" | ||
from pydantic import BaseModel | ||
|
||
|
||
class Knight(BaseModel): | ||
title: str | ||
age: int | ||
color: str = 'blue' | ||
|
||
|
||
class Quest(BaseModel): | ||
title: str | ||
knight: Knight | ||
|
||
|
||
quest = Quest( | ||
title='To seek the Holy Grail', | ||
knight={'title': 'Sir Lancelot', 'age': 23} | ||
) | ||
``` | ||
|
||
The type of the field `knight` is declared with the class `Knight` (a *pydantic* model) and the code is passing a literal `dict` instead. This is still valid for *pydantic*, and the `dict` would be automatically converted to a `Knight` instance. | ||
|
||
Nevertheless, it would be detected as a type error: | ||
|
||
![VS Code strict type errors with model](./img/vs_code_07.png) | ||
|
||
In those cases, there are several ways to disable or ignore strict errors in very specific places, while still preserving them in the rest of the code. | ||
|
||
Below are several techniques to achieve it. | ||
|
||
#### Disable type checks in a line | ||
|
||
You can disable the errors for a specific line using a comment of: | ||
|
||
``` | ||
# type: ignore | ||
``` | ||
|
||
coming back to the example with `age='23'`, it would be: | ||
|
||
```Python hl_lines="10" | ||
from pydantic import BaseModel | ||
|
||
|
||
class Knight(BaseModel): | ||
title: str | ||
age: int | ||
color: str = 'blue' | ||
|
||
|
||
lancelot = Knight(title='Sir Lancelot', age='23') # type: ignore | ||
``` | ||
|
||
that way Pylance and mypy will ignore errors in that line. | ||
|
||
**Pros**: it's a simple change in that line to remove errors there. | ||
|
||
**Cons**: any other error in that line will also be omitted, including type checks, misspelled arguments, required arguments not provided, etc. | ||
|
||
#### Override the type of a variable | ||
|
||
You can also create a variable with the value you want to use and declare it's type explicitly with `Any`. | ||
|
||
```Python hl_lines="1 11-12" | ||
from typing import Any | ||
from pydantic import BaseModel | ||
|
||
|
||
class Knight(BaseModel): | ||
title: str | ||
age: int | ||
color: str = 'blue' | ||
|
||
|
||
age_str: Any = '23' | ||
lancelot = Knight(title='Sir Lancelot', age=age_str) | ||
``` | ||
|
||
that way Pylance and mypy will interpret the variable `age_str` as if they didn't know its type, instead of knowing it has a type of `str` when an `int` was expected (and then showing the corresponding error). | ||
|
||
**Pros**: errors will be ignored only for a specific value, and you will still see any additional errors for the other arguments. | ||
|
||
**Cons**: it requires importing `Any` and a new variable in a new line for each argument that needs ignoring errors. | ||
|
||
#### Override the type of a value with `cast` | ||
|
||
The same idea from the previous example can be put on the same line with the help of `cast()`. | ||
|
||
This way, the type declaration of the value is overriden inline, without requiring another variable. | ||
|
||
```Python hl_lines="1 11" | ||
from typing import Any, cast | ||
from pydantic import BaseModel | ||
|
||
|
||
class Knight(BaseModel): | ||
title: str | ||
age: int | ||
color: str = 'blue' | ||
|
||
|
||
lancelot = Knight(title='Sir Lancelot', age=cast(Any, '23')) | ||
``` | ||
|
||
`cast(Any, '23')` doesn't affect the value, it's still just `'23'`, but now Pylance and mypy will assume it is of type `Any`, which means, they will act as if they didn't know the type of the value. | ||
|
||
So, this is the equivalent of the previous example, without the additional variable. | ||
|
||
**Pros**: errors will be ignored only for a specific value, and you will still see any additional errors for the other arguments. There's no need for additional variables. | ||
|
||
**Cons**: it requires importing `Any` and `cast`, and if you are not used to using `cast()`, it could seem strange at first. | ||
|
||
### Config in class arguments | ||
|
||
*pydantic* has a rich set of [Model Configurations](./usage/model_config.md) available. | ||
|
||
These configurations can be set in an internal `class Config` on each model: | ||
|
||
```Python hl_lines="9-10" | ||
from pydantic import BaseModel | ||
|
||
|
||
class Knight(BaseModel): | ||
title: str | ||
age: int | ||
color: str = 'blue' | ||
|
||
class Config: | ||
frozen = True | ||
``` | ||
|
||
or passed as keyword arguments when defining the model class: | ||
|
||
```Python hl_lines="4" | ||
from pydantic import BaseModel | ||
|
||
|
||
class Knight(BaseModel, frozen=True): | ||
title: str | ||
age: int | ||
color: str = 'blue' | ||
``` | ||
|
||
The specific configuration **`frozen`** (in beta) has a special meaning. | ||
|
||
It prevents other code from changing a model instance once it's created, keeping it **"frozen"**. | ||
|
||
When using the second version to declare `frozen=True` (with **keyword arguments** in the class definition), Pylance can use it to help you check in your code and **detect errors** when something is trying to set values in a model that is "frozen". | ||
|
||
![VS Code strict type errors with model](./img/vs_code_08.png) | ||
|
||
## Technical Details | ||
|
||
!!! warning | ||
As a *pydantic* user, you don't need the details below. Feel free to skip the rest of this section. | ||
|
||
These details are only useful for other library authors, etc. | ||
|
||
This additional editor support works by implementing the proposed draft standard for [Dataclass Transform](https://github.com/microsoft/pyright/blob/master/specs/dataclass_transforms.md). | ||
|
||
The proposed draft standard is written by Eric Traut, from the Microsoft team, the same author of the open source package Pyright (used by Pylance to provide Python support in VS Code). | ||
|
||
The intention of the standard is to provide a way for libraries like *pydantic* and others to tell editors and tools that they (the editors) should treat these libraries (e.g. *pydantic*) as if they were `dataclasses`, providing autocompletion, type checks, etc. | ||
|
||
The draft standard also includes an [Alternate Form](https://github.com/microsoft/pyright/blob/master/specs/dataclass_transforms.md#alternate-form) for early adopters, like *pydantic*, to add support for it right away, even before the new draft standard is finished and approved. | ||
|
||
This new draft standard, with the Alternate Form, is already supported by Pyright, so it can be used via Pylance in VS Code. | ||
|
||
As it is being proposed as an official standard for Python, other editors can also easily add support for it. | ||
|
||
And authors of other libraries similar to *pydantic* can also easily adopt the standard right away (using the "Alternate Form") and get the benefits of these additional editor features. |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -11,6 +11,7 @@ | |
AbstractSet, | ||
Any, | ||
Callable, | ||
ClassVar, | ||
Dict, | ||
List, | ||
Mapping, | ||
|
@@ -28,7 +29,7 @@ | |
from .config import BaseConfig, Extra, inherit_config, prepare_config | ||
from .error_wrappers import ErrorWrapper, ValidationError | ||
from .errors import ConfigError, DictError, ExtraError, MissingError | ||
from .fields import MAPPING_LIKE_SHAPES, ModelField, ModelPrivateAttr, PrivateAttr, Undefined | ||
from .fields import MAPPING_LIKE_SHAPES, Field, FieldInfo, ModelField, ModelPrivateAttr, PrivateAttr, Undefined | ||
from .json import custom_pydantic_encoder, pydantic_encoder | ||
from .parse import Protocol, load_file, load_str_bytes | ||
from .schema import default_ref_template, model_schema | ||
|
@@ -90,6 +91,18 @@ | |
|
||
__all__ = 'BaseModel', 'compiled', 'create_model', 'validate_model' | ||
|
||
_T = TypeVar('_T') | ||
|
||
|
||
def __dataclass_transform__( | ||
*, | ||
eq_default: bool = True, | ||
order_default: bool = False, | ||
kw_only_default: bool = False, | ||
samuelcolvin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
field_descriptors: Tuple[Union[type, Callable[..., Any]], ...] = (()), | ||
) -> Callable[[_T], _T]: | ||
return lambda a: a | ||
|
||
|
||
def validate_custom_root_type(fields: Dict[str, ModelField]) -> None: | ||
if len(fields) > 1: | ||
|
@@ -114,6 +127,7 @@ def hash_function(self_: Any) -> int: | |
_is_base_model_class_defined = False | ||
|
||
|
||
@__dataclass_transform__(kw_only_default=True, field_descriptors=(Field, FieldInfo)) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there any possibility to add the I can't see how this is possible but @erictraut suggested it should be. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You can pass frozen=True when constructing a class that derives from class CustomerModel(ModelBase, frozen=True): That was based on a suggestion you made in our email discussion — or at least how I interpreted your suggestion. Did I understand you correctly? All classes are assumed to be non-frozen unless otherwise specified. There is a request from the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh, that's awesome. I've just tried it and it's already working! This looks great then, the only problem I can see is the type strictness/laxness which is being discussed on microsoft/pyright#1782 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. EDIT: Dang it, I didn't see all your previous conversation above this message, I guess I had a stale window. Sorry. EDIT 2: discard this message pretty much entirely, already covered above and below. 🤦 😂 Old message below: I didn't see a way to do it from what I read in the spec. If I understand correctly, it would be possible for Pyright (and the spec) to understand if the model was created like: from pydantic import BaseModel
class Fish(BaseModel, frozen=True):
title: str ...which is currently not supported by pydantic (although I think sounds interesting). What is currently supported by pydantic (but I understand not by Pyright nor the spec) is: from pydantic import BaseModel
class Fish(BaseModel):
title: str
class Config:
frozen = True I'll wait for @erictraut's input in case it's possible to support pydantic's internal There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. class Fish(BaseModel, frozen=True):
title: str is actually supported in 1.8 @tiangolo 😉 Config arguments can now be passed as class kwargs thanks to @MrMrRobat's amazing work in #2356 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Dang it again! 🤦 😂 pydantic has evolved so much and I haven't noticed everything! 😅 Thanks for the clarification @PrettyWood, that's awesome! 🎉 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I tried this with v1.9 and Pyright does not seem to recognise # pyright: strict
from typing import Hashable
from pydantic import BaseModel
class Foo(BaseModel, frozen=True):
pass
foo: Hashable = Foo() # Error: Expression of type "Foo" cannot be assigned to declared type "Hashable"
reveal_type(Foo.__hash__) # Type of "Foo.__hash__" is "None"
from dataclasses import dataclass
@dataclass(frozen=True)
class Bar:
pass
bar: Hashable = Bar() # No error
reveal_type(Bar.__hash__) # Type of "Bar.__hash__" is "(self: Bar) -> int" There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Interesting, this probably needs a new issue. I've no idea if this is something pydantic can help with, or an issue with pyright. @erictraut any idea? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
class ModelMetaclass(ABCMeta): | ||
@no_type_check # noqa C901 | ||
def __new__(mcs, name, bases, namespace, **kwargs): # noqa C901 | ||
|
@@ -284,21 +298,21 @@ def is_untouched(v: Any) -> bool: | |
class BaseModel(Representation, metaclass=ModelMetaclass): | ||
if TYPE_CHECKING: | ||
# populated by the metaclass, defined here to help IDEs only | ||
__fields__: Dict[str, ModelField] = {} | ||
__include_fields__: Optional[Mapping[str, Any]] = None | ||
__exclude_fields__: Optional[Mapping[str, Any]] = None | ||
__validators__: Dict[str, AnyCallable] = {} | ||
__pre_root_validators__: List[AnyCallable] | ||
__post_root_validators__: List[Tuple[bool, AnyCallable]] | ||
__config__: Type[BaseConfig] = BaseConfig | ||
__root__: Any = None | ||
__json_encoder__: Callable[[Any], Any] = lambda x: x | ||
__schema_cache__: 'DictAny' = {} | ||
__custom_root_type__: bool = False | ||
__signature__: 'Signature' | ||
__private_attributes__: Dict[str, ModelPrivateAttr] | ||
__class_vars__: SetStr | ||
__fields_set__: SetStr = set() | ||
__fields__: ClassVar[Dict[str, ModelField]] = {} | ||
__include_fields__: ClassVar[Optional[Mapping[str, Any]]] = None | ||
__exclude_fields__: ClassVar[Optional[Mapping[str, Any]]] = None | ||
__validators__: ClassVar[Dict[str, AnyCallable]] = {} | ||
__pre_root_validators__: ClassVar[List[AnyCallable]] | ||
__post_root_validators__: ClassVar[List[Tuple[bool, AnyCallable]]] | ||
__config__: ClassVar[Type[BaseConfig]] = BaseConfig | ||
__root__: ClassVar[Any] = None | ||
__json_encoder__: ClassVar[Callable[[Any], Any]] = lambda x: x | ||
__schema_cache__: ClassVar['DictAny'] = {} | ||
__custom_root_type__: ClassVar[bool] = False | ||
__signature__: ClassVar['Signature'] | ||
__private_attributes__: ClassVar[Dict[str, ModelPrivateAttr]] | ||
__class_vars__: ClassVar[SetStr] | ||
__fields_set__: ClassVar[SetStr] = set() | ||
|
||
Config = BaseConfig | ||
__slots__ = ('__dict__', '__fields_set__') | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a small remark @tiangolo
Why not say
cast(int, '23')
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the review, update to the PR, and the note @PrettyWood!
My intention with using
cast(Any, '23')
was to make it something simple to copy-paste for other use cases and types.For example, if it was something like a
Dict[str, List[SomeModel]]
and the value was a dictionary with lists of dictionaries (instead of Pydantic models), e.g. parsed from some JSON, duplicating the annotation from the original field would be a bit cumbersome, would require duplication (which is always problematic and I think avoiding duplication is one of the greatest advantages of Pydantic), and adding the full type annotation/cast wouldn't provide much more help as it's only for mypy/Pylance.Also, the idea was to make it explicit that this is just to "trick" mypy and Pylance to not pay attention to the actual types. And to avoid any possible confusion from users seeing a
cast(int, '23')
, which is technically wrong (in terms of types).And also to prevent them from thinking that they can just copy that anywhere else and assume it would work (e.g. thinking it would convert the types automatically or something). At least if they use
cast(Any, '23')
in any other place, they lose autocompletion and can (hopefully) realize that the value is not an actual integer.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense! Thanks 👍