From 72fcb85c3b1083e9614dd173b7f7a2fa10c99e15 Mon Sep 17 00:00:00 2001 From: tpdorsey Date: Wed, 5 Jul 2023 19:47:59 -0400 Subject: [PATCH 1/9] dataclasses and typeadapter --- docs/migration.md | 31 +++++++++++--------- docs/usage/dataclasses.md | 61 ++++++++++++++++++++++++--------------- pydantic/dataclasses.py | 2 +- 3 files changed, 57 insertions(+), 37 deletions(-) diff --git a/docs/migration.md b/docs/migration.md index 5885cca46a..8851b34a97 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -145,23 +145,27 @@ The following properties have been removed from or changed in `Field`: ### Changes to dataclasses +Pydantic [dataclasses](usage/dataclasses.md) continue to be useful for enabling the data validation on standard +dataclasses without having to subclass `BaseModel`. However, we have made some changes to the behavior of Pydantic +dataclasses in V2: + * When used as fields, dataclasses (Pydantic or vanilla) no longer accept tuples as validation inputs; dicts should be used instead. * The `__post_init__` in Pydantic dataclasses will now be called _after_ validation, rather than before. * As a result, the `__post_init_post_parse__` method would have become redundant, so has been removed. * We no longer support `extra='allow'` for Pydantic dataclasses, where extra fields passed to the initializer would be - stored as extra attributes on the dataclass. `extra='ignore'` is still supported for the purpose of ignoring - unexpected fields while parsing data, they just won't be stored on the instance. + stored as extra attributes on the dataclass. `extra='ignore'` is still supported for the purpose of ignoring + unexpected fields while parsing data, they just won't be stored on the instance. * Pydantic dataclasses no longer have an attribute `__pydantic_model__`, and no longer use an underlying `BaseModel` - to perform validation or provide other functionality. + to perform validation or provide other functionality. * To perform validation, generate a JSON schema, or make use of - any other functionality that may have required `__pydantic_model__` in V1, you should now wrap the dataclass - with a `TypeAdapter` (discussed more below) and make use of its methods. - * [TODO: Add link to TypeAdapter documentation. You can find example usage in `tests/test_type_adapter.py`.] + any other functionality that may have required `__pydantic_model__` in V1, you should now wrap the dataclass + with a [`TypeAdapter`](usage/models.md#typeadapter) ([discussed more below](#introduction-of-typeadapter)) and + make use of its methods. * In Pydantic V1, if you used a vanilla (i.e., non-Pydantic) dataclass as a field, the config of the parent type would - be used as though it was the config for the dataclass itself as well. In Pydantic V2, this is no longer the case. - * [TODO: Need to specify how to override the config used for vanilla dataclass; possibly need to add functionality?] - + be used as though it was the config for the dataclass itself as well. In Pydantic V2, this is no longer the case. + * In Pydantic V2, to override the config like you would with a `BaseModel`, you can use the `config` parameter + on the `dataclass` decorator. See [Dataclass Config](usage/dataclasses.md#dataclass-config) for examples. ### Changes to config @@ -589,10 +593,11 @@ Pydantic V1 had weak support for validating or serializing non-`BaseModel` types To work with them, you had to either create a "root" model or use the utility functions in `pydantic.tools` (namely, `parse_obj_as` and `schema_of`). -In Pydantic V2 this is _a lot_ easier: the `TypeAdapter` class lets you create an object with methods for validating, -serializing, and producing JSON schemas for arbitrary types. This serves as a complete replacement for `parse_obj_as` -and `schema_of` (which are now deprecated), and also covers some of the use cases of "root" models -(`RootModel`, discussed above, covers the others). +In Pydantic V2 this is _a lot_ easier: the [`TypeAdapter`](usage/models.md#typeadapter) class lets you create an object +with methods for validating, serializing, and producing JSON schemas for arbitrary types. +This serves as a complete replacement for `parse_obj_as` and `schema_of` (which are now deprecated), +and also covers some of the use cases of "root" models. ([`RootModel`](usage/models.md#rootmodel-and-custom-root-types), +[discussed above](#changes-to-pydanticbasemodel), covers the others.) ```python from typing import List diff --git a/docs/usage/dataclasses.md b/docs/usage/dataclasses.md index 3993aa6a11..269d857df0 100644 --- a/docs/usage/dataclasses.md +++ b/docs/usage/dataclasses.md @@ -25,18 +25,26 @@ User(id=42, name='John Doe', signup_ts=datetime.datetime(2032, 6, 21, 12, 0)) ``` !!! note - Keep in mind that `pydantic.dataclasses.dataclass` is **not** a replacement for `pydantic.BaseModel` (with a small - difference in how [initialization hooks](#initialize-hooks) work). `pydantic.dataclasses.dataclass` provides a - similar functionality to `dataclasses.dataclass` with the addition of Pydantic validation. There are cases where - subclassing `pydantic.BaseModel` is the better choice. + Keep in mind that `pydantic.dataclasses.dataclass` is **not** a replacement for `pydantic.BaseModel`. + `pydantic.dataclasses.dataclass` provides a similar functionality to `dataclasses.dataclass` with the addition of + Pydantic validation. + There are cases where subclassing `pydantic.BaseModel` is the better choice. For more information and discussion see [pydantic/pydantic#710](https://github.com/pydantic/pydantic/issues/710). -You can use all the standard _pydantic_ field types. Note, however, that arguments passed to constructor will be copied in order to perform validation and, where necessary coercion. +Some differences between Pydantic dataclasses and `BaseModel` include: -The schema can be accessed by [TypeAdapter](models.md#typeadapter). -Also, fields that require a `default_factory` can be specified by either a `pydantic.Field` or a `dataclasses.field`. +* How [initialization hooks](#initialization-hooks) work +* [JSON dumping](#json-dumping) + +You can use all the standard Pydantic field types. Note, however, that arguments passed to constructor will be copied in +order to perform validation and, where necessary coercion. + +To perform validation or generate a JSON schema on a Pydantic dataclass, you should now wrap the dataclass +with a [`TypeAdapter`](models.md#typeadapter) and make use of its methods. + +Fields that require a `default_factory` can be specified by either a `pydantic.Field` or a `dataclasses.field`. ```py import dataclasses @@ -101,10 +109,13 @@ keyword argument `config` which has the same meaning as [model_config](model_con For more information about combining validators with dataclasses, see [dataclass validators](validators.md#dataclass-validators). -## Dataclass Config +## Dataclass config If you want to modify the `config` like you would with a `BaseModel`, you have two options: +* Apply config to the dataclass decorator as a dict +* Use `ConfigDict` as the config + ```py from pydantic import ConfigDict from pydantic.dataclasses import dataclass @@ -126,11 +137,12 @@ class MyDataclass2: 1. You can read more about `validate_assignment` in [model_config](model_config.md#validate-assignment). -!!! warning - After v1.10, _pydantic_ dataclasses support `Config.extra` but some default behaviour of stdlib dataclasses - may prevail. For example, when `print`ing a _pydantic_ dataclass with allowed extra fields, it will still - use the `__str__` method of stdlib dataclass and show only the required fields. - This may be improved further in the future. +!!! note + Pydantic dataclasses do not support [`extra='allow'`](model_config/#extra-attributes), where extra fields passed + to the initializer would be stored as extra attributes on the dataclass. + + `extra='ignore'` is still supported for the purpose of ignoring + unexpected fields while parsing data; they just won't be stored on the instance. ## Nested dataclasses @@ -156,13 +168,13 @@ print(navbar) #> Navbar(button=NavbarButton(href=Url('https://example.com/'))) ``` -Dataclasses attributes can be populated by tuples, dictionaries or instances of the dataclass itself. +When used as fields, dataclasses (Pydantic or vanilla) should use dicts as validation inputs. -## Stdlib dataclasses and _pydantic_ dataclasses +## Stdlib dataclasses and Pydantic dataclasses ### Inherit from stdlib dataclasses -Stdlib dataclasses (nested or not) can also be inherited and _pydantic_ will automatically validate +Stdlib dataclasses (nested or not) can also be inherited and Pydantic will automatically validate all the inherited fields. ```py @@ -203,8 +215,8 @@ except pydantic.ValidationError as e: ### Use of stdlib dataclasses with `BaseModel` -Bear in mind that stdlib dataclasses (nested or not) are **automatically converted** into _pydantic_ -dataclasses when mixed with `BaseModel`! Furthermore the generated _pydantic_ dataclass will have +Bear in mind that stdlib dataclasses (nested or not) are **automatically converted** into Pydantic +dataclasses when mixed with `BaseModel`! Furthermore the generated Pydantic dataclass will have the **exact same configuration** (`order`, `frozen`, ...) as the original one. ```py @@ -260,8 +272,8 @@ except dataclasses.FrozenInstanceError as e: ### Use custom types -Since stdlib dataclasses are automatically converted to add validation using -custom types may cause some unexpected behaviour. +Since stdlib dataclasses are automatically converted to add validation, using +custom types may cause some unexpected behavior. In this case you can simply add `arbitrary_types_allowed` in the config! ```py @@ -317,10 +329,10 @@ print(repr(m)) #> Model(dc=DC(a=ArbitraryType(value=3), b='qwe'), other='other') ``` -## Initialize hooks +## Initialization hooks When you initialize a dataclass, it is possible to execute code *before* or *after* validation -with the help of `model_validator` decorator. +with the help of the [`@model_validator`](validators.md#model-validators) decorator `mode` parameter. ```py from typing import Any, Dict @@ -360,6 +372,9 @@ class User: user = User(**{'birth': {'year': 1995, 'month': 3, 'day': 2}}) ``` +The `__post_init__` in Pydantic dataclasses is called _after_ validation, rather than before. + + ```py requires="3.8" from dataclasses import InitVar from pathlib import Path @@ -394,7 +409,7 @@ methods decorated with `model_validator`. ## JSON dumping -_Pydantic_ dataclasses do not feature a `.model_dump_json()` function. To dump them as JSON, you will need to +Pydantic dataclasses do not feature a `.model_dump_json()` function. To dump them as JSON, you will need to make use of the [RootModel](models.md#rootmodel-and-custom-root-types) as follows: ```py output="json" diff --git a/pydantic/dataclasses.py b/pydantic/dataclasses.py index c36f8bc259..9e83d22863 100644 --- a/pydantic/dataclasses.py +++ b/pydantic/dataclasses.py @@ -107,7 +107,7 @@ def dataclass( kw_only: bool = False, slots: bool = False, ) -> Callable[[type[_T]], type[PydanticDataclass]] | type[PydanticDataclass]: - """A decorator used to create a Pydantic-enhanced dataclass, similar to the standard Python `dataclasses`, + """A decorator used to create a Pydantic-enhanced dataclass, similar to standard Python `dataclasses`, but with added validation. This function should be used similarly to `dataclasses.dataclass`. From ec42a00138693d0479c8716a8bd2f390fda4c18f Mon Sep 17 00:00:00 2001 From: tpdorsey Date: Wed, 5 Jul 2023 20:49:58 -0400 Subject: [PATCH 2/9] add changes file --- changes/6470-tpdorsey.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 changes/6470-tpdorsey.md diff --git a/changes/6470-tpdorsey.md b/changes/6470-tpdorsey.md new file mode 100644 index 0000000000..f3fc2acaba --- /dev/null +++ b/changes/6470-tpdorsey.md @@ -0,0 +1 @@ +Docs: document new dataclases features, add links to TypeAdapter documentation. From 9582692fee8b4976809d521e0bba6f854cb821d0 Mon Sep 17 00:00:00 2001 From: Terrence Dorsey Date: Thu, 6 Jul 2023 13:39:06 -0400 Subject: [PATCH 3/9] Apply suggestions from code review Co-authored-by: Yaakov Bressler <40807730+ybressler@users.noreply.github.com> --- docs/migration.md | 5 ++--- pydantic/dataclasses.py | 2 +- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/migration.md b/docs/migration.md index 8851b34a97..fd53dad257 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -146,14 +146,13 @@ The following properties have been removed from or changed in `Field`: ### Changes to dataclasses Pydantic [dataclasses](usage/dataclasses.md) continue to be useful for enabling the data validation on standard -dataclasses without having to subclass `BaseModel`. However, we have made some changes to the behavior of Pydantic -dataclasses in V2: +dataclasses without having to subclass `BaseModel`. Pydantic V2 introduces the following changes to this dataclass behavior: * When used as fields, dataclasses (Pydantic or vanilla) no longer accept tuples as validation inputs; dicts should be used instead. * The `__post_init__` in Pydantic dataclasses will now be called _after_ validation, rather than before. * As a result, the `__post_init_post_parse__` method would have become redundant, so has been removed. -* We no longer support `extra='allow'` for Pydantic dataclasses, where extra fields passed to the initializer would be +* Pydantic no longer supports `extra='allow'` for Pydantic dataclasses, where extra fields passed to the initializer would be stored as extra attributes on the dataclass. `extra='ignore'` is still supported for the purpose of ignoring unexpected fields while parsing data, they just won't be stored on the instance. * Pydantic dataclasses no longer have an attribute `__pydantic_model__`, and no longer use an underlying `BaseModel` diff --git a/pydantic/dataclasses.py b/pydantic/dataclasses.py index 9e83d22863..c36f8bc259 100644 --- a/pydantic/dataclasses.py +++ b/pydantic/dataclasses.py @@ -107,7 +107,7 @@ def dataclass( kw_only: bool = False, slots: bool = False, ) -> Callable[[type[_T]], type[PydanticDataclass]] | type[PydanticDataclass]: - """A decorator used to create a Pydantic-enhanced dataclass, similar to standard Python `dataclasses`, + """A decorator used to create a Pydantic-enhanced dataclass, similar to the standard Python `dataclasses`, but with added validation. This function should be used similarly to `dataclasses.dataclass`. From a9ed29ebdc64b39fb10ed3425f8970023620fc6c Mon Sep 17 00:00:00 2001 From: Terrence Dorsey Date: Thu, 6 Jul 2023 13:40:51 -0400 Subject: [PATCH 4/9] Apply suggestions from code review --- docs/migration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/migration.md b/docs/migration.md index fd53dad257..99919a1fe7 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -163,7 +163,7 @@ dataclasses without having to subclass `BaseModel`. Pydantic V2 introduces the f make use of its methods. * In Pydantic V1, if you used a vanilla (i.e., non-Pydantic) dataclass as a field, the config of the parent type would be used as though it was the config for the dataclass itself as well. In Pydantic V2, this is no longer the case. - * In Pydantic V2, to override the config like you would with a `BaseModel`, you can use the `config` parameter + * In Pydantic V2, to override the `config` (like you would with a `BaseModel`), you can use the `config` parameter on the `dataclass` decorator. See [Dataclass Config](usage/dataclasses.md#dataclass-config) for examples. ### Changes to config From 9c9d398a8656d669750ced36cc4683e6db9f4d78 Mon Sep 17 00:00:00 2001 From: tpdorsey Date: Thu, 6 Jul 2023 14:04:31 -0400 Subject: [PATCH 5/9] remove update warning --- docs/usage/dataclasses.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/usage/dataclasses.md b/docs/usage/dataclasses.md index 269d857df0..225c33a58c 100644 --- a/docs/usage/dataclasses.md +++ b/docs/usage/dataclasses.md @@ -1,7 +1,4 @@ -!!! warning - This page still needs to be updated for v2.0. - -If you don't want to use _pydantic_'s `BaseModel` you can instead get the same data validation on standard +If you don't want to use Pydantic's `BaseModel` you can instead get the same data validation on standard [dataclasses](https://docs.python.org/3/library/dataclasses.html) (introduced in Python 3.7). ```py From 5edc16e96e9eb9f5a86b0c9ae2248fe5dd4e4192 Mon Sep 17 00:00:00 2001 From: tpdorsey Date: Mon, 10 Jul 2023 12:37:03 -0400 Subject: [PATCH 6/9] revisions based on feedback --- docs/migration.md | 5 +++-- pydantic/dataclasses.py | 16 ++++++++-------- 2 files changed, 11 insertions(+), 10 deletions(-) diff --git a/docs/migration.md b/docs/migration.md index 99919a1fe7..3a643d8bf3 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -163,8 +163,9 @@ dataclasses without having to subclass `BaseModel`. Pydantic V2 introduces the f make use of its methods. * In Pydantic V1, if you used a vanilla (i.e., non-Pydantic) dataclass as a field, the config of the parent type would be used as though it was the config for the dataclass itself as well. In Pydantic V2, this is no longer the case. - * In Pydantic V2, to override the `config` (like you would with a `BaseModel`), you can use the `config` parameter - on the `dataclass` decorator. See [Dataclass Config](usage/dataclasses.md#dataclass-config) for examples. + * In Pydantic V2, to override the config (like you would with `model_config` on a `BaseModel`), + you can use the `config` parameter on the `@dataclass` decorator. + See [Dataclass Config](usage/dataclasses.md#dataclass-config) for examples. ### Changes to config diff --git a/pydantic/dataclasses.py b/pydantic/dataclasses.py index c36f8bc259..39d141cb43 100644 --- a/pydantic/dataclasses.py +++ b/pydantic/dataclasses.py @@ -107,31 +107,31 @@ def dataclass( kw_only: bool = False, slots: bool = False, ) -> Callable[[type[_T]], type[PydanticDataclass]] | type[PydanticDataclass]: - """A decorator used to create a Pydantic-enhanced dataclass, similar to the standard Python `dataclasses`, + """A decorator used to create a Pydantic-enhanced dataclass, similar to the standard Python `dataclass`, but with added validation. This function should be used similarly to `dataclasses.dataclass`. Args: - _cls: The target dataclass. + _cls: The target `dataclass`. init: Included for signature compatibility with `dataclasses.dataclass`, and is passed through to `dataclasses.dataclass` when appropriate. If specified, must be set to `False`, as pydantic inserts its own `__init__` function. - repr: A boolean indicating whether or not to include the field in the __repr__ output. + repr: A boolean indicating whether or not to include the field in the `__repr__` output. eq: Determines if a `__eq__` should be generated for the class. order: Determines if comparison magic methods should be generated, such as` __lt__`, but not `__eq__`. unsafe_hash: Determines if an unsafe hashing function should be included in the class. - frozen: Determines if the generated class should be a 'frozen' dataclass, which does not allow its + frozen: Determines if the generated class should be a 'frozen' `dataclass`, which does not allow its attributes to be modified from its constructor. - config: A configuration for the dataclass generation. - validate_on_init: A deprecated parameter included for backwards compatibility; in V2, all pydantic dataclasses + config: A configuration for the `dataclass` generation. + validate_on_init: A deprecated parameter included for backwards compatibility; in V2, all Pydantic `dataclass`es are validated on init. kw_only: Determines if `__init__` method parameters must be specified by keyword only. Defaults to `False`. - slots: Determines if the generated class should be a 'slots' dataclass, which does not allow the addition of + slots: Determines if the generated class should be a 'slots' `dataclass`, which does not allow the addition of new attributes after instantiation. Returns: - A decorator that accepts a class as its argument and returns a Pydantic dataclass. + A decorator that accepts a class as its argument and returns a Pydantic `dataclass`. Raises: AssertionError: Raised if `init` is not `False` or `validate_on_init` is `False`. From cd28ad02325bb78ab3e2034ae2f7418b6c09c585 Mon Sep 17 00:00:00 2001 From: tpdorsey Date: Mon, 10 Jul 2023 12:37:55 -0400 Subject: [PATCH 7/9] fix typo --- pydantic/dataclasses.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pydantic/dataclasses.py b/pydantic/dataclasses.py index 39d141cb43..cb2cbd22f7 100644 --- a/pydantic/dataclasses.py +++ b/pydantic/dataclasses.py @@ -119,7 +119,7 @@ def dataclass( own `__init__` function. repr: A boolean indicating whether or not to include the field in the `__repr__` output. eq: Determines if a `__eq__` should be generated for the class. - order: Determines if comparison magic methods should be generated, such as` __lt__`, but not `__eq__`. + order: Determines if comparison magic methods should be generated, such as `__lt__`, but not `__eq__`. unsafe_hash: Determines if an unsafe hashing function should be included in the class. frozen: Determines if the generated class should be a 'frozen' `dataclass`, which does not allow its attributes to be modified from its constructor. From 1d41a2c5b858d7e1c1a9a80669351e51247d22cc Mon Sep 17 00:00:00 2001 From: Terrence Dorsey Date: Mon, 10 Jul 2023 13:55:32 -0400 Subject: [PATCH 8/9] Apply suggestions from code review Co-authored-by: Yaakov Bressler <40807730+ybressler@users.noreply.github.com> --- pydantic/dataclasses.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pydantic/dataclasses.py b/pydantic/dataclasses.py index cb2cbd22f7..3fdeff59e5 100644 --- a/pydantic/dataclasses.py +++ b/pydantic/dataclasses.py @@ -124,7 +124,7 @@ def dataclass( frozen: Determines if the generated class should be a 'frozen' `dataclass`, which does not allow its attributes to be modified from its constructor. config: A configuration for the `dataclass` generation. - validate_on_init: A deprecated parameter included for backwards compatibility; in V2, all Pydantic `dataclass`es +validate_on_init: A deprecated parameter included for backwards compatibility; in V2, all Pydantic dataclasses are validated on init. kw_only: Determines if `__init__` method parameters must be specified by keyword only. Defaults to `False`. slots: Determines if the generated class should be a 'slots' `dataclass`, which does not allow the addition of From c1766c0c6c0688f38ce6e7613a479878b2d1e223 Mon Sep 17 00:00:00 2001 From: tpdorsey Date: Mon, 10 Jul 2023 14:37:39 -0400 Subject: [PATCH 9/9] format fixed --- pydantic/dataclasses.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pydantic/dataclasses.py b/pydantic/dataclasses.py index 3fdeff59e5..ea58d71419 100644 --- a/pydantic/dataclasses.py +++ b/pydantic/dataclasses.py @@ -124,7 +124,7 @@ def dataclass( frozen: Determines if the generated class should be a 'frozen' `dataclass`, which does not allow its attributes to be modified from its constructor. config: A configuration for the `dataclass` generation. -validate_on_init: A deprecated parameter included for backwards compatibility; in V2, all Pydantic dataclasses + validate_on_init: A deprecated parameter included for backwards compatibility; in V2, all Pydantic dataclasses are validated on init. kw_only: Determines if `__init__` method parameters must be specified by keyword only. Defaults to `False`. slots: Determines if the generated class should be a 'slots' `dataclass`, which does not allow the addition of