New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
✨ Add autocomplete support for VS Code, via dataclass_transform
#2721
Changes from 3 commits
a8e2528
88fe649
d4c57c9
7c0adbb
c926ed4
a56e3e1
1bed0f7
f5fa6a2
ff06395
e9953a3
a379b86
a22eee9
3ceab48
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Add support for autocomplete in VS Code via `__dataclass_transform__` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,230 @@ | ||
*pydantic* works well with any editor or IDE out of the box because it's made on top of standard Python type annotations. | ||
|
||
When using [Visual Studio Code (VS Code)](https://code.visualstudio.com/), there are some **additional editor features** supported, comparable to the ones provided by the [PyCharm plugin](./pycharm_plugin.md). | ||
|
||
This means that you will have **autocompletion** (or "IntelliSense") and **error checks** for types and required arguments even while creating new *pydantic* model instances. | ||
|
||
![pydantic autocompletion in VS Code](./img/vs_code_01.png) | ||
|
||
## Configure VS Code | ||
|
||
To take advantage of these features, you need to make sure you configure VS Code correctly, using the recommended settings. | ||
|
||
In case you have a different configuration, here's a short overview of the steps. | ||
|
||
### Install Pylance | ||
|
||
You should use the [Pylance](https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-pylance) extension for VS Code. It is the recommended, next-generation, official VS Code plug-in for Python. | ||
|
||
Make sure you install and enable it in your editor. | ||
|
||
### Configure your environment | ||
|
||
Then you need to make sure your editor knows the [Python environment](https://code.visualstudio.com/docs/python/python-tutorial#_install-and-use-packages) (probably a virtual environment) for your Python project. | ||
|
||
This would be the environment in where you installed *pydantic*. | ||
|
||
### Configure Pylance and Pyright | ||
|
||
Pylance is the VS Code extension. Underneath, it uses an open source tool called [Pyright](https://github.com/microsoft/pyright) that does all the heavy lifting. | ||
|
||
With the default configurations, you will get support for autocompletion, but Pyright (and Pylance) might not check for type errors. | ||
|
||
You can enable type error checks from Pylance/Pyright with these steps: | ||
|
||
* Open the "User Settings" | ||
* Search for `Type Checking Mode` | ||
* You will find an option under `Python › Analysis: Type Checking Mode` | ||
* Set it to `basic` or `strict` (by default it's `off`) | ||
|
||
![Type Checking Mode set to strict in VS Code](./img/vs_code_02.png) | ||
|
||
Now you will not only get autocompletion when creating new *pydantic* model instances but also error checks for **required arguments**. | ||
|
||
![Required arguments error checks in VS Code](./img/vs_code_03.png) | ||
|
||
And you will also get error checks for **invalid data types**. | ||
|
||
![Invalid data types error checks in VS Code](./img/vs_code_04.png) | ||
|
||
### Configure mypy | ||
|
||
You might also want to configure mypy in VS Code to get mypy error checks inline in your editor (alternatively/additionally to Pyright). | ||
|
||
This would include the errors detected by the [*pydantic* mypy plugin](./mypy_plugin.md), if you configured it. | ||
|
||
To enable mypy in VS Code, do the following: | ||
|
||
* Open the "User Settings" | ||
* Search for `Mypy Enabled` | ||
* You will find an option under `Python › Linting: Mypy Enabled` | ||
* Check the box (by default it's unchecked) | ||
|
||
![mypy enabled in VS Code](./img/vs_code_05.png) | ||
|
||
## Tips and tricks | ||
|
||
Here are some additional tips and tricks to improve your developer experience when using VS Code with *pydantic*. | ||
|
||
### Strict errors | ||
|
||
The way this additional editor support works is that Pylance/Pyright will treat your *pydantic* models as if they were Python's pure `dataclasses`. | ||
|
||
And it will show **strict type error checks** about the data types passed in arguments when creating a new *pydantic* model instance. | ||
|
||
In this example you can see that it shows that a `str` of `'23'` is not a valid `int` for the argument `age`. | ||
|
||
![VS Code strict type errors](./img/vs_code_06.png) | ||
|
||
It would expect `age=23` instead of `age='23'`. | ||
|
||
Nevertheless, the design, and one of the main features of *pydantic*, is that it is very **lenient with data types**. | ||
|
||
It will actually accept the `str` with value `'23'` and will convert it to an `int` with value `23`. | ||
|
||
These strict error checks are **very useful** most of the time and can help you **detect many bugs early**. But there are cases, like with `age='23'`, where they could be inconvenient by reporting a "false positive" error. | ||
|
||
--- | ||
|
||
This example above with `age='23'` is intentionally simple, to show the error and the differences in types. | ||
|
||
But more common cases where these strict errors would be inconvenient would be when using more sophisticated data types, like `int` values for `datetime` fields, or `dict` values for *pydantic* sub-models. | ||
|
||
For example, this is valid for *pydantic*: | ||
|
||
```Python hl_lines="12 17" | ||
from pydantic import BaseModel | ||
|
||
|
||
class Knight(BaseModel): | ||
title: str | ||
age: int | ||
color: str = 'blue' | ||
|
||
|
||
class Quest(BaseModel): | ||
title: str | ||
knight: Knight | ||
|
||
|
||
quest = Quest( | ||
title='To seek the Holy Grail', | ||
knight={'title': 'Sir Lancelot', 'age': 23} | ||
) | ||
``` | ||
|
||
The type of the field `knight` is declared with the class `Knight` (a *pydantic* model) and the code is passing a literal `dict` instead. This is still valid for *pydantic*, and the `dict` would be automatically converted to a `Knight` instance. | ||
|
||
Nevertheless, it would be detected as a type error: | ||
|
||
![VS Code strict type errors with model](./img/vs_code_07.png) | ||
|
||
#### Disable type checks in a line | ||
|
||
In those cases, you can disable the errors for a specific line using a comment of: | ||
|
||
``` | ||
# type: ignore | ||
``` | ||
|
||
coming back to the example with `age='23'`, it would be: | ||
|
||
```Python hl_lines="10" | ||
from pydantic import BaseModel | ||
|
||
|
||
class Knight(BaseModel): | ||
title: str | ||
age: int | ||
color: str = 'blue' | ||
|
||
|
||
lancelot = Knight(title='Sir Lancelot', age='23') # type: ignore | ||
``` | ||
|
||
that way Pylance/Pyright and mypy will ignore errors in that line. | ||
|
||
#### Override the type of a variable | ||
|
||
Alternatively, you can create a variable with the value you want to use, and set an explicit type of `Any`. | ||
|
||
```Python hl_lines="1 11-12" | ||
from typing import Any | ||
from pydantic import BaseModel | ||
|
||
|
||
class Knight(BaseModel): | ||
title: str | ||
age: int | ||
color: str = 'blue' | ||
|
||
|
||
age_str: Any = '23' | ||
lancelot = Knight(title='Sir Lancelot', age=age_str) | ||
``` | ||
|
||
that way Pylance/Pyright and mypy will interpret the variable `age_str` as if they didn't know its type, instead of knowing it has a type of `str` when an `int` was expected (and then showing the corresponding error). | ||
|
||
The advantage of this technique is that you will still see any additional errors for the other arguments. | ||
|
||
The disadvantage is that you have to create a new variable in a new line for each argument with inexact data types. | ||
|
||
### Config in class arguments | ||
|
||
*pydantic* has a rich set of [Model Configurations](./usage/model_config.md) available. | ||
|
||
These configurations can be set in an internal `class Config` on each model: | ||
|
||
```Python hl_lines="9-10" | ||
from pydantic import BaseModel | ||
|
||
|
||
class Knight(BaseModel): | ||
title: str | ||
age: int | ||
color: str = 'blue' | ||
|
||
class Config: | ||
frozen = True | ||
``` | ||
|
||
or passed as keyword arguments when defining the model class: | ||
|
||
```Python hl_lines="4" | ||
from pydantic import BaseModel | ||
|
||
|
||
class Knight(BaseModel, frozen=True): | ||
title: str | ||
age: int | ||
color: str = 'blue' | ||
``` | ||
|
||
The specific configuration **`frozen`** (in beta) has a special meaning. | ||
|
||
It prevents other code from changing a model instance once it's created, keeping it **"frozen"**. | ||
|
||
When using the second version to declare `frozen=True` (with **keyword arguments** in the class definition), Pylance/Pyright can use it to help you check in your code and **detect errors** when something is trying to set values in a model that is "frozen". | ||
|
||
![VS Code strict type errors with model](./img/vs_code_08.png) | ||
|
||
## Technical Details | ||
|
||
!!! warning | ||
As a *pydantic* user, you don't need the details below. Feel free to skip the rest of this section. | ||
|
||
These details are only useful for other library authors, etc. | ||
|
||
This additional editor support works by implementing the proposed draft standard for [Dataclass Transform](https://github.com/microsoft/pyright/blob/master/specs/dataclass_transforms.md). | ||
|
||
The proposed draft standard is written by Eric Traut, from the Microsoft team, the same author of the open source package Pyright (used by Pylance to provide Python support in VS Code). | ||
|
||
The intention of the standard is to provide a way for libraries like *pydantic* and others to tell editors and tools that they (the editors) should treat these libraries (e.g. *pydantic*) as if they were `dataclasses`, providing autocompletion, type checks, etc. | ||
|
||
The draft standard also includes an [Alternate Form](https://github.com/microsoft/pyright/blob/master/specs/dataclass_transforms.md#alternate-form) for early adopters, like *pydantic*, to add support for it right away, even before the new draft standard is finished and approved. | ||
|
||
This new draft standard, with the Alternate Form, is already supported by Pyright, so it can be used via Pylance in VS Code. | ||
|
||
As it is being proposed as an official standard for Python, other editors can also easily add support for it. | ||
|
||
And authors of other libraries similar to *pydantic* can also easily adopt the standard right away (using the "Alternate Form") and get the benefits of these additional editor features. |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -28,7 +28,7 @@ | |
from .class_validators import ValidatorGroup, extract_root_validators, extract_validators, inherit_validators | ||
from .error_wrappers import ErrorWrapper, ValidationError | ||
from .errors import ConfigError, DictError, ExtraError, MissingError | ||
from .fields import MAPPING_LIKE_SHAPES, ModelField, ModelPrivateAttr, PrivateAttr, Undefined | ||
from .fields import MAPPING_LIKE_SHAPES, Field, FieldInfo, ModelField, ModelPrivateAttr, PrivateAttr, Undefined | ||
from .json import custom_pydantic_encoder, pydantic_encoder | ||
from .parse import Protocol, load_file, load_str_bytes | ||
from .schema import default_ref_template, model_schema | ||
|
@@ -102,6 +102,20 @@ def __call__(self, schema: Dict[str, Any], model_class: Type['Model']) -> None: | |
except AttributeError: | ||
compiled = False | ||
|
||
|
||
_T = TypeVar('_T') | ||
|
||
|
||
def __dataclass_transform__( | ||
*, | ||
eq_default: bool = True, | ||
order_default: bool = False, | ||
kw_only_default: bool = False, | ||
samuelcolvin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
field_descriptors: Tuple[Union[type, Callable[..., Any]], ...] = (()), | ||
) -> Callable[[_T], _T]: | ||
return lambda a: a | ||
|
||
|
||
__all__ = 'BaseConfig', 'BaseModel', 'Extra', 'compiled', 'create_model', 'validate_model' | ||
|
||
|
||
|
@@ -223,6 +237,7 @@ def hash_function(self_: Any) -> int: | |
_is_base_model_class_defined = False | ||
|
||
|
||
@__dataclass_transform__(kw_only_default=True, field_descriptors=(Field, FieldInfo)) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there any possibility to add the I can't see how this is possible but @erictraut suggested it should be. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You can pass frozen=True when constructing a class that derives from class CustomerModel(ModelBase, frozen=True): That was based on a suggestion you made in our email discussion — or at least how I interpreted your suggestion. Did I understand you correctly? All classes are assumed to be non-frozen unless otherwise specified. There is a request from the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh, that's awesome. I've just tried it and it's already working! This looks great then, the only problem I can see is the type strictness/laxness which is being discussed on microsoft/pyright#1782 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. EDIT: Dang it, I didn't see all your previous conversation above this message, I guess I had a stale window. Sorry. EDIT 2: discard this message pretty much entirely, already covered above and below. 🤦 😂 Old message below: I didn't see a way to do it from what I read in the spec. If I understand correctly, it would be possible for Pyright (and the spec) to understand if the model was created like: from pydantic import BaseModel
class Fish(BaseModel, frozen=True):
title: str ...which is currently not supported by pydantic (although I think sounds interesting). What is currently supported by pydantic (but I understand not by Pyright nor the spec) is: from pydantic import BaseModel
class Fish(BaseModel):
title: str
class Config:
frozen = True I'll wait for @erictraut's input in case it's possible to support pydantic's internal There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. class Fish(BaseModel, frozen=True):
title: str is actually supported in 1.8 @tiangolo 😉 Config arguments can now be passed as class kwargs thanks to @MrMrRobat's amazing work in #2356 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Dang it again! 🤦 😂 pydantic has evolved so much and I haven't noticed everything! 😅 Thanks for the clarification @PrettyWood, that's awesome! 🎉 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I tried this with v1.9 and Pyright does not seem to recognise # pyright: strict
from typing import Hashable
from pydantic import BaseModel
class Foo(BaseModel, frozen=True):
pass
foo: Hashable = Foo() # Error: Expression of type "Foo" cannot be assigned to declared type "Hashable"
reveal_type(Foo.__hash__) # Type of "Foo.__hash__" is "None"
from dataclasses import dataclass
@dataclass(frozen=True)
class Bar:
pass
bar: Hashable = Bar() # No error
reveal_type(Bar.__hash__) # Type of "Bar.__hash__" is "(self: Bar) -> int" There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Interesting, this probably needs a new issue. I've no idea if this is something pydantic can help with, or an issue with pyright. @erictraut any idea? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
class ModelMetaclass(ABCMeta): | ||
@no_type_check # noqa C901 | ||
def __new__(mcs, name, bases, namespace, **kwargs): # noqa C901 | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, the advantage is that it will not disable all type checking for that line, e.g. it will alert you if you misspell
age
or the name of the class or if you make a syntactical error.# type: ignore
disables all of these checks. A third option would also be to usecast
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. And great idea!
cast
could work quite well. 🚀I'll add the
cast
option and update the docs for that to explain the disadvantages of# type: ignore
as well. And show the 3 alternatives in order of increasing steps/complexity.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cast
should work nicely but as I mentioned in the discussion, the problem is that is technically wrong.cast()
says "this think is actually anint
, promise. If I'm wrong, it's my problem" here that's not the case, the thing really is a string (or whatever) it's just that that type is valid.I'm afraid sadly there's no non-hack here, but I guess
cast()
could be useful.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The non-hack is to be explicit in type conversions. If pydantic contains logic that performs type conversions implicitly, does it expose those type conversion routines so they can be invoked explicitly for users who are interested in static type safety? The use of
Any
orcast
or# type: ignore
are all poor workarounds if you care about static type checking.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, agreed. I updated the docs including another example using
cast(Any, '23')
instead ofcast(int, '23')
, I think that's an acceptable balance, telling the editor "don't check this", which would also work for any other type, without having to do something that is technically wrong likecast(int, '23')
.Agreed, and for cases like the example with a
'23'
it's quite obvious and it would be a lot better to do the conversion manually.But for things like
datetime
s that accept multiple values includingstr
,int
,float
, or for passing literaldict
s in places declared with a pydantic model class, I think that using a singlecast()
could be an acceptable tradeoff for now.I understand that not currently. Maybe that could be a new feature request, and then these docs could be updated accordingly. But I think that with respect to this PR, these docs could be enough for now.