Parse error when nested objects include a possible Union

[mgberg]

Checks

• I added a descriptive title to this issue
• I have searched (google, github) for similar issues and couldn't find anything
• I have read and followed the docs and still think this is a bug

Bug

I'm using pydantic version 1.8.2 with Python 3.9.2 on linux.

I'm trying to wrap an existing API with Pydantic models and I ran into a situation where a Field may take either an object that conforms to one of two Models or a list of objects that conform to either of those models, plus some other metadata. In certain situations, the object does not get parsed properly. Here's a mockup/simplification of the Models associated with this issue:

from pydantic import BaseModel, Field
from typing import Union, Optional


class MetadataModel(BaseModel):
    label: Optional[str] = Field()
    description: Optional[str] = Field()


class ModelA(MetadataModel):
    type: str = Field(...)
    value: str = Field(...)


class ModelB(MetadataModel):
    type: str = Field(...)
    attributes: Optional[dict] = Field()


class MainModel(BaseModel):
    children: Union[ModelA, ModelB, list[Union[ModelA, ModelB]]] = Field()

If I run the following six test cases, the last two produce something odd.

# Case 1: One child of type ModelA works
print(
    repr(
        MainModel.parse_obj(
            {
                "children": {
                    "type": "A",
                    "label": "Object of type ModelA",
                    "value": "some string",
                }
            }
        )
    ),
)
# MainModel(children=ModelA(label='Object of type ModelA', description=None, type='A', value='some string'))

# Case 2: A list of one child of type ModelA works
print(
    repr(
        MainModel.parse_obj(
            {
                "children": [
                    {
                        "type": "A",
                        "label": "Object of type ModelA",
                        "value": "some string",
                    }
                ]
            }
        )
    ),
)
# MainModel(children=[ModelA(label='Object of type ModelA', description=None, type='A', value='some string')])

# Case 3: A list of two children of type ModelA works
print(
    repr(
        MainModel.parse_obj(
            {
                "children": [
                    {
                        "type": "A",
                        "label": "Object of type ModelA",
                        "value": "some string",
                    },
                    {
                        "type": "A",
                        "label": "Another object of type ModelA",
                        "value": "some string",
                    },
                ]
            }
        )
    ),
)
# MainModel(children=[ModelA(label='Object of type ModelA', description=None, type='A', value='some string'), ModelA(label='Another object of type ModelA', description=None, type='A', value='some string')])

# Case 4: One child of type ModelB works
print(
    repr(
        MainModel.parse_obj(
            {"children": {"type": "B", "label": "Object of type ModelB"}}
        )
    ),
)
# MainModel(children=ModelB(label='Object of type ModelB', description=None, type='B', attributes=None))

# Case 5: A list of one child of type ModelB DOES NOT work
print(
    repr(
        MainModel.parse_obj(
            {"children": [{"type": "B", "label": "Object of type ModelB"}]}
        )
    ),
)
# The list is misinterpreted and the "label" key became a value
# MainModel(children=ModelB(label=None, description=None, type='label', attributes=None))


# Case 6: A list of two children of type ModelB DOES NOT work
print(
    repr(
        MainModel.parse_obj(
            {
                "children": [
                    {"type": "B", "label": "Object of type ModelB"},
                    {"type": "B", "label": "Another object of type ModelB"},
                ]
            }
        )
    ),
)
# Same error as above
# MainModel(children=ModelB(label=None, description=None, type='label', attributes=None))

I found that if I were to reorder the type annotations, the issue (at least this specific issue) goes away. If I redefine MainModel like this:

class MainModel(BaseModel):
    children: Union[list[Union[ModelA, ModelB]], ModelA, ModelB] = Field()

Then run the test cases above again, the first four work correctly still and the second two work and produce the following output:

# Case 5: Correct
# MainModel(children=[ModelB(label='Object of type ModelB', description=None, type='B', attributes=None)])

# Case 6: Correct
# MainModel(children=[ModelB(label='Object of type ModelB', description=None, type='B', attributes=None), ModelB(label='Another object of type ModelB', description=None, type='B', attributes=None)])

I scanned through the documentation and I didn't find anything that seemed to directly impact this scenario. This issue seems problematic and reordering the type annotations seems like a fragile fix. I'm new to pydantic, so perhaps there are best practices that I am not aware of that would prevent this problem from happening or perhaps this is a actually a feature needed for something else.

[PrettyWood]

Hi @mgberg
#2092 will solve your issue. It should be available in v1.9.
Once it's released, you'll need to set Config.smart_union = True and you'll be good to go.
Note that it will probably be the default behaviour in v2