New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: change pydantic dataclass decorator #2557
Changes from all commits
665b5c9
b723e59
914f3bb
8706598
2e15ef5
f3d68ce
95f1d15
f5a5756
9ae2e30
d7b2b27
64dbc56
e86f8bb
4086193
dc717fe
9799d80
e7cbee1
538c7b3
3874661
9174d7a
adda021
bf0f191
c3aa37d
461fa1b
77f4d6b
263859c
066f23c
432ae04
1926e85
e3a8f87
80da159
bde50bb
eee2e26
d101420
04f7ea0
8e24502
84ec7b5
a8237d8
5c5d152
6715d4c
c6d8201
6914c04
30b58f3
0079d93
fb77d0d
1a60ab2
ffecfc2
3a6ddf1
5907f82
571609d
4810457
0c7d578
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
Refactor the whole _pydantic_ `dataclass` decorator to really act like its standard lib equivalent. | ||
It hence keeps `__eq__`, `__hash__`, ... and makes comparison with its non-validated version possible. | ||
It also fixes usage of `frozen` dataclasses in fields and usage of `default_factory` in nested dataclasses. | ||
The support of `Config.extra` has been added. | ||
Finally, config customization directly via a `dict` is now possible. | ||
<br/><br/> | ||
**BREAKING CHANGES** | ||
- The `compiled` boolean (whether _pydantic_ is compiled with cython) has been moved from `main.py` to `version.py` | ||
- Now that `Config.extra` is supported, `dataclass` ignores by default extra arguments (like `BaseModel`) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
from pydantic import ConfigDict | ||
from pydantic.dataclasses import dataclass | ||
|
||
|
||
# Option 1 - use directly a dict | ||
# Note: `mypy` will still raise typo error | ||
@dataclass(config=dict(validate_assignment=True)) | ||
class MyDataclass1: | ||
a: int | ||
|
||
|
||
# Option 2 - use `ConfigDict` | ||
# (same as before at runtime since it's a `TypedDict` but with intellisense) | ||
@dataclass(config=ConfigDict(validate_assignment=True)) | ||
class MyDataclass2: | ||
a: int | ||
|
||
|
||
# Option 3 - use a `Config` class like for a `BaseModel` | ||
class Config: | ||
validate_assignment = True | ||
|
||
|
||
@dataclass(config=Config) | ||
class MyDataclass3: | ||
a: int |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
import dataclasses | ||
|
||
from pydantic import ValidationError | ||
from pydantic.dataclasses import dataclass as pydantic_dataclass, set_validation | ||
|
||
|
||
@dataclasses.dataclass | ||
class User: | ||
id: int | ||
name: str | ||
|
||
|
||
# Enhance stdlib dataclass | ||
pydantic_dataclass(User) | ||
|
||
|
||
user1 = User(id='whatever', name='I want') | ||
|
||
# validate data of `user1` | ||
try: | ||
user1.__pydantic_validate_values__() | ||
except ValidationError as e: | ||
print(e) | ||
|
||
# Enforce validation | ||
try: | ||
with set_validation(User, True): | ||
User(id='whatever', name='I want') | ||
except ValidationError as e: | ||
print(e) |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -18,7 +18,7 @@ You can use all the standard _pydantic_ field types, and the resulting dataclass | |
created by the standard library `dataclass` decorator. | ||
|
||
The underlying model and its schema can be accessed through `__pydantic_model__`. | ||
Also, fields that require a `default_factory` can be specified by a `dataclasses.field`. | ||
Also, fields that require a `default_factory` can be specified by either a `pydantic.Field` or a `dataclasses.field`. | ||
|
||
```py | ||
{!.tmp_examples/dataclasses_default_schema.py!} | ||
|
@@ -34,6 +34,20 @@ keyword argument `config` which has the same meaning as [Config](model_config.md | |
For more information about combining validators with dataclasses, see | ||
[dataclass validators](validators.md#dataclass-validators). | ||
|
||
## Dataclass Config | ||
|
||
If you want to modify the `Config` like you would with a `BaseModel`, you have three options: | ||
|
||
```py | ||
{!.tmp_examples/dataclasses_config.py!} | ||
``` | ||
|
||
!!! warning | ||
After v1.10, _pydantic_ dataclasses support `Config.extra` but some default behaviour of stdlib dataclasses | ||
may prevail. For example, when `print`ing a _pydantic_ dataclass with allowed extra fields, it will still | ||
use the `__str__` method of stdlib dataclass and show only the required fields. | ||
This may be improved further in the future. | ||
|
||
## Nested dataclasses | ||
|
||
Nested dataclasses are supported both in dataclasses and normal models. | ||
|
@@ -51,12 +65,25 @@ Dataclasses attributes can be populated by tuples, dictionaries or instances of | |
|
||
Stdlib dataclasses (nested or not) can be easily converted into _pydantic_ dataclasses by just decorating | ||
them with `pydantic.dataclasses.dataclass`. | ||
_Pydantic_ will enhance the given stdlib dataclass but won't alter the default behaviour (i.e. without validation). | ||
It will instead create a wrapper around it to trigger validation that will act like a plain proxy. | ||
The stdlib dataclass can still be accessed via the `__dataclass__` attribute (see example below). | ||
|
||
```py | ||
{!.tmp_examples/dataclasses_stdlib_to_pydantic.py!} | ||
``` | ||
_(This script is complete, it should run "as is")_ | ||
|
||
### Choose when to trigger validation | ||
|
||
As soon as your stdlib dataclass has been decorated with _pydantic_ dataclass decorator, magic methods have been | ||
added to validate input data. If you want, you can still keep using your dataclass and choose when to trigger it. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is great. I wrote some code which wraps standard lib dataclasses using pydantic dataclasses when I want to call validation to achieve the same thing. Would love to see it implemented in the pydantic lib itself so we can allow users to set up their model however they like (i.e. without having to pass everything to the init func) and parse/validate when they are done. |
||
|
||
```py | ||
{!.tmp_examples/dataclasses_stdlib_run_validation.py!} | ||
``` | ||
_(This script is complete, it should run "as is")_ | ||
|
||
### Inherit from stdlib dataclasses | ||
|
||
Stdlib dataclasses (nested or not) can also be inherited and _pydantic_ will automatically validate | ||
|
@@ -95,6 +122,11 @@ When you initialize a dataclass, it is possible to execute code *after* validati | |
with the help of `__post_init_post_parse__`. This is not the same as `__post_init__`, which executes | ||
code *before* validation. | ||
|
||
!!! tip | ||
If you use a stdlib `dataclass`, you may only have `__post_init__` available and wish the validation to | ||
be done before. In this case you can set `Config.post_init_call = 'after_validation'` | ||
|
||
|
||
```py | ||
{!.tmp_examples/dataclasses_post_init_post_parse.py!} | ||
``` | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,6 +4,7 @@ | |
|
||
from .typing import AnyCallable | ||
from .utils import GetterDict | ||
from .version import compiled | ||
|
||
if TYPE_CHECKING: | ||
from typing import overload | ||
|
@@ -27,7 +28,7 @@ def __call__(self, schema: Dict[str, Any], model_class: Type[BaseModel]) -> None | |
else: | ||
SchemaExtraCallable = Callable[..., None] | ||
|
||
__all__ = 'BaseConfig', 'Extra', 'inherit_config', 'prepare_config' | ||
__all__ = 'BaseConfig', 'ConfigDict', 'get_config', 'Extra', 'inherit_config', 'prepare_config' | ||
|
||
|
||
class Extra(str, Enum): | ||
|
@@ -36,6 +37,46 @@ class Extra(str, Enum): | |
forbid = 'forbid' | ||
|
||
|
||
# https://github.com/cython/cython/issues/4003 | ||
# Will be fixed with Cython 3 but still in alpha right now | ||
if not compiled: | ||
from typing_extensions import Literal, TypedDict | ||
|
||
class ConfigDict(TypedDict, total=False): | ||
title: Optional[str] | ||
anystr_lower: bool | ||
anystr_strip_whitespace: bool | ||
min_anystr_length: int | ||
max_anystr_length: Optional[int] | ||
validate_all: bool | ||
extra: Extra | ||
allow_mutation: bool | ||
frozen: bool | ||
allow_population_by_field_name: bool | ||
use_enum_values: bool | ||
fields: Dict[str, Union[str, Dict[str, str]]] | ||
validate_assignment: bool | ||
error_msg_templates: Dict[str, str] | ||
arbitrary_types_allowed: bool | ||
orm_mode: bool | ||
getter_dict: Type[GetterDict] | ||
alias_generator: Optional[Callable[[str], str]] | ||
keep_untouched: Tuple[type, ...] | ||
schema_extra: Union[Dict[str, Any], 'SchemaExtraCallable'] | ||
json_loads: Callable[[str], Any] | ||
json_dumps: Callable[..., str] | ||
json_encoders: Dict[Type[Any], AnyCallable] | ||
underscore_attrs_are_private: bool | ||
|
||
# whether or not inherited models as fields should be reconstructed as base model | ||
copy_on_model_validation: bool | ||
# whether dataclass `__post_init__` should be run after validation | ||
post_init_call: Literal['before_validation', 'after_validation'] | ||
|
||
else: | ||
ConfigDict = dict # type: ignore | ||
|
||
|
||
class BaseConfig: | ||
title: Optional[str] = None | ||
anystr_lower: bool = False | ||
|
@@ -66,6 +107,8 @@ class BaseConfig: | |
copy_on_model_validation: bool = True | ||
# whether `Union` should check all allowed types before even trying to coerce | ||
smart_union: bool = False | ||
# whether dataclass `__post_init__` should be run before or after validation | ||
post_init_call: Literal['before_validation', 'after_validation'] = 'before_validation' | ||
|
||
@classmethod | ||
def get_field_info(cls, name: str) -> Dict[str, Any]: | ||
|
@@ -100,6 +143,25 @@ def prepare_field(cls, field: 'ModelField') -> None: | |
pass | ||
|
||
|
||
def get_config(config: Union[ConfigDict, Type[BaseConfig], None]) -> Type[BaseConfig]: | ||
if config is None: | ||
return BaseConfig | ||
|
||
else: | ||
config_dict = ( | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. if There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The thing is most people write
without actually setting |
||
config | ||
if isinstance(config, dict) | ||
else {k: getattr(config, k) for k in dir(config) if not k.startswith('__')} | ||
) | ||
|
||
class Config(BaseConfig): | ||
... | ||
|
||
for k, v in config_dict.items(): | ||
setattr(Config, k, v) | ||
return Config | ||
|
||
|
||
def inherit_config(self_config: 'ConfigType', parent_config: 'ConfigType', **namespace: Any) -> 'ConfigType': | ||
if not self_config: | ||
base_classes: Tuple['ConfigType', ...] = (parent_config,) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does this really count as breaking? The import from main will still work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
from pydantic.main import compiled
will fail. If it doesn't matter let's remove those 2 lines