New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature Request] Provide a discriminated union type (OpenAPI 3) #619
Comments
The validation component of this can already by accomplished via the This allows you to add a field which is constrained to one or multiple values then use that field to discriminate between to models. However at the moment this doesn't extend to a discriminator object in the schema. Perhaps it would be possible to either:
? |
Guessing what the discriminator field may be based on const or literal values might lead to unexpected behavior, using Config properties or schema parameters would probably be preferable. A union field could have a discriminator parameter on its schema object to indicate which field to match against, which would then have to be const/literal values on each of the variants. Each literal value could then be rendered as a separate key/value pair in the mapping dictionary, which would raise an error in case of collision. This would leave the default case (first key in the discriminator mapping is considered as the default option is nothing matches, IIRC) undefined, though... I'll see if I can try coming up with something that can be easily validated against while not being too cumbersome to work with. |
I've taken another look at the OpenAPI spec on this and tried a number of potential syntaxes. The from pydantic import BaseModel, Schema
from typing import Literal
class Pet(BaseModel):
petType: str = Schema(...)
# One or the other
__discriminator__ = 'petType'
class Config:
discriminator = 'petType'
class Cat(Pet):
petType: Literal['cat', 'robot_cat'] # Should render as a string enum
name: str = Schema(...)
class Dog(Pet):
petType: Literal['dog', 'robot_dog']
bark: str = Schema(...) There would however be the problem of dealing with versions of Python without support for The OpenAPI spec also mentions that the schema names can act as implicit discriminators, whether or not an explicit mapping is present (so 'Dog' and 'Cat' could technically be valid if there is no other constraint), however I don't really believe supporting that use case is of much concern. |
Literal is supported in python 3.7 (and 3.6 if I recall correctly) by importing from the package |
Ah, well, that's sure to make things a lot simpler, then.
EDIT: All I might try filing a PR for this in the coming days if I have time. |
I will propose a PR to include additional schema (JSON Schema) data for models. This will allow you to create the validation required using, e.g. And then you can describe it in the extra schema data (as JSON Schema/OpenAPI schema) as an OpenAPI discriminator, etc. purely for documentation. I think adding discriminator support directly to Pydantic wouldn't be convenient, as the discriminator ideas are very specific to OpenAPI, are a bit constrained (not that generalizable), and have conflicts with some ideas in JSON Schema ( But combining these things I described above (with the extra schema I'll PR) you should be able to achieve what you need, with the OpenAPI documentation you expect @sm-Fifteen . |
@sm-Fifteen I think you can now perform the validation as you need in a validator and generate the schema the way you want it using |
Thinking about this more (and running into a similar problem myself), I think some kind of discriminator field is required:
Personally I think this should be done by adding a |
So Usage would be something like class Foo(BaseModel):
model_type: Literal['foo']
class Bar(BaseModel):
model_type: Literal['bar']
class MyModel(BaseModel):
foobar: Union[Foo, Bar] = Field(..., descriminator='model_type') |
@tiangolo: I don't know if we could have a solution that would work for both OpenAPI and JSONSchema without losing the benefits of mypy validation. I don't even know if JSON Schema's fully conditional validation system can cleanly be mapped to a type system at all. Being able to specify fields unknown to Pydantic in your generated schema is nice, but discriminators affect validation logic and not being able to get mypy to tell appart the subtypes would be unfortunate.
@samuelcolvin: I like the idea and proposed syntax, though I see the "running into a similar problem" issue was closed after you posted your reply, so I'm not sure if you still thing the addition is warranted? |
I still definitely want |
but it might need to be a function or a field name. |
maybe (I've just spent some time looking for this issue as I looked for "determinant" rather than "descriminator") |
Considering this would mainly be there to map with OpenAPI's
I figured I should restate that part, since it would probably affect the resulting API design. |
My interest in using descriminators is not related to openAPI or JSONSchema, I want a way of using Unions, where:
I'm fine with
Can't say I entirely understand what this means, but sounds good to me. |
Nevermind that, I thought the spec essentially said something along the lines of "Only schemas that are in The spec actually says something more like "The discriminator can only map towards schemas that have IDs (i.e: the subclasses must appear in MyResponseType:
discriminator:
propertyName: petType
mapping:
# Notice how there's no $ref, it's just a direct reference to the target type
dog: '#/components/schemas/Dog'
monster: 'https://gigantic-server.com/schemas/Monster/schema.json'
oneOf:
- $ref: '#/components/schemas/Cat'
- $ref: '#/components/schemas/Dog'
- $ref: '#/components/schemas/Lizard'
- $ref: 'https://gigantic-server.com/schemas/Monster/schema.json' |
This feature is highly desired in my teams implementation :) |
I don't get how |
class ActionModel(BaseModel):
class Config:
fields = {"action": dict(const=True)}
extra = "forbid"
class Something(ActionModel):
action = "something"
ACTIONS = {
"something": Something,
}
class Action:
@classmethod
def __get_validators__(cls):
yield cls.return_action
@classmethod
def return_action(cls, values):
try:
action = values["action"]
except KeyError:
raise MalformedAction(
f"Missing required 'action' field for action: {values}"
)
try:
return ACTIONS[action](**values)
except KeyError:
raise MalformedAction(f"Incorrect action: {action}")
class Flow(BaseModel):
actions: List[Action] |
@samuelcolvin @PrettyWood Any updates on this? I'm in the progress of switching away from GraphQL (for a lot of reasons, unrelated) but having proper support for tagged unions is kinda an issue. Also that tagged unions aren't properly exposed to OpenAPI. |
Not sure if there are any bounty programs that pydantic supports, but I'd be happy to put up some $ for this |
@ghostbody Inspired our solution. We used type @unique
class SelectFieldTypes(str, Enum):
date = "date"
multiple_choice = "multiple-choice"
multiple_select = "multiple-select"
class DateModel(BaseModel):
id: str
type: Literal[SelectFieldTypes.date]
text: str
description: str
required: bool
value: Optional[str]
class MultipleChoice(BaseModel):
id: str
type: Literal[SelectFieldTypes.multiple_choice]
text: str
description: str
required: bool
options: List[Option]
value: Optional[str]
class MultipleSelect(BaseModel):
id: str
type: Literal[SelectFieldTypes.multiple_select]
text: str
description: str
required: bool
options: List[Option]
value: Optional[str]
class SelectTemplate(BaseModel):
id: str
text: str
fields_: List[
Union[
DateModel,
MultipleChoice,
MultipleSelect,
]
] = Field(..., alias="fields") |
Hi everyone 😃
If that's all we want I guess my PR should be a good first draft 👍 |
@PrettyWood That PR looks great. I'll give it a try later! |
@PrettyWood: That looks like a pretty good way of handling things, having types with literal fields that could be validated just fine without the use of discriminators, and then specifying a discriminator field on some union of those types use on the parent union, in a way that builds on top of regular validation. This is actually similar to what's has been proposed as a potential replacement for I'm not seeing any logic or tests for handling such discriminated unions as the root element, though. Is this supported by this PR? |
class Cat(BaseModel):
pet_type: Literal['cat']
name: str
class Dog(BaseModel):
pet_type: Literal['dog']
name: str
class Pet(BaseModel):
__root__: Union[Cat, Dog] = Field(..., discriminator='pet_type')
my_dog = Pet.parse_obj({'pet_type': 'dog', 'name': 'woof'}).__root__
assert isinstance(my_dog, Dog) works. Not sure if that's what you had in mind |
Btw @sm-Fifteen in fact it should be even easier with #2147. Pet = Annotated[Union[Cat, Dog], Field(discriminator='pet_type')]
try:
parse_obj_as(Pet, {'pet_type': 'dog', 'nam': 'woof'})
except ValidationError as e:
print(e)
# 1 validation error for ParsingModel[Annotated[Union[__main__.Cat, __main__.Dog], FieldInfo(default=Ellipsis, extra={'discriminator': 'pet_type'})]]
# __root__ (Dog) -> name
# field required (type=value_error.missing)
print(schema_json(Pet, title='Pet', indent=2))
# {
# "title": "Pet",
# "discriminator": {
# "propertyName": "pet_type",
# "mapping": {
# "cat": "#/definitions/Cat",
# "dog": "#/definitions/Dog"
# }
# },
# "anyOf": [
# {
# "$ref": "#/definitions/Cat"
# },
# {
# "$ref": "#/definitions/Dog"
# }
# ],
# "definitions": {
# ... |
I wasn't super convinced by the other root union example, although I figured it was at least serviceable, but using PEP 593 |
Thank you so much for that PR. Testing it out, is seems like you can call class Cat(BaseModel):
pet_type: Literal['cat'] = 'cat'
name: str
class Dog(BaseModel):
pet_type: Literal['dog'] = 'dog'
name: str
Pet = Annotated[Union[Cat, Dog], Field(discriminator='pet_type')]
# parse_obj_as(Pet, dict(name='Felix')) # fails as expected
parse_obj_as(Pet, dict(pet_type='cat', name='Felix'))
Cat(name='Felix') Is this an expected use case or are there problems with this pattern? |
@KevOrr Yep the discriminator is used for the schema and to improve validation (faster and more explicit) |
I've been using this branch for weeks to add a discriminator to several types and it seems like it would be a handy compliment to |
#619 (comment) However, with this code, it seems as if (1) (even non-pre) root validators on the non-matching classes are still executed, though, and (2) they don't get passed the discriminator field value: class Cat(BaseModel):
pet_type: Literal['cat']
name: str
age: int
@root_validator
def height_constraints(cls, values: Dict[str, Any]) -> Dict[str, Any]:
print(values)
assert str(values["age"]) in values["name"] # silly check
return values
class Dog(BaseModel):
pet_type: Literal['dog']
name: str
class Pet(BaseModel):
__root__: Union[Cat, Dog] = Field(..., discriminator='pet_type')
my_dog = Pet.parse_obj({'pet_type': 'dog', 'name': 'woof'}).__root__
assert isinstance(my_dog, Dog) Running this code prints:
Does anyone know if that is being addressed in #2336? |
Hi @tgpfeiffer |
Feature Request
Pydantic currently has a decent support for union types through the
typing.Union
type from PEP484, but it does not currently cover all the cases covered by the JSONSchema and OpenAPI specifications, most likely because the two specifications diverge on those points.OpenAPI supports something similar to tagged unions where a certain field is designated to serve as a "discriminator", which is then matched against literal values to determine which of multiple schemas to use for payload validation. In order to allow Pydantic to support those, I suppose there would have to be a specific type similar to
typing.Union
in order to specify what discriminator field to use and how to match it. Such a type would then be rendered into a schema object (oneOf
) with an OpenAPI discriminator object built into it, as well as correctly validate incoming JSON into the correct type based on the value or the discriminator field. This change would only impact OpenAPI, as JSON schema (draft 7 onwards) uses conditional types instead, which would probably need to be the topic of a different feature request, as both methods appear mutually incompatible.Implementation ideas
I'd imagine the final result to be something like this.
Python doesn't have a feature like TypeScript to let you statically ensure that
discriminator
exists as a field for all variants of that union, though that shouldn't be a problem since this is going to be raised during validation regardless.discriminator
andmapping
could also simply be added toSchema
, though I'm not sure about whether it's a good idea to add OpenAPI-specific extensions there.PEP 593 would also have been a nice alternative, since it would hypothetically allow tagged unions to be implemented as a regular union with annotations specific to Pydantic for that purpose, however it is only still a draft and most likely won't make it until Python 3.9 (if at all).
The text was updated successfully, but these errors were encountered: