generated dataclasses not compatible with fastapi/pydantic dataclass bridge

[cmungall]

When using fastapi to serialize linkml objects we get an error

  File "/Users/cjm/.local/share/virtualenvs/linkml-6dchTQmP/lib/python3.9/site-packages/linkml_runtime/utils/dataclass_extensions_376.py", line 24, in dataclasses_init_fn_with_kwargs
    raise TypeError(f'non-default argument {f.name!r} '
TypeError: non-default argument 'mixins' follows default argument

note: this even happens when we use the bridge layer between dataclasses and pydantic: https://fastapi.tiangolo.com/advanced/dataclasses/

here is the code that reproduces this (I will also make a PR)

main.py:

#https://fastapi.tiangolo.com/advanced/dataclasses/
from dataclasses import dataclass
from enum import Enum
from typing import Optional

from linkml_runtime.linkml_model import SlotDefinition
from pydantic import BaseModel
import pydantic
from pydantic.dataclasses import dataclass

from fastapi import FastAPI, Query

@dataclass
class SimpleSlotDefinition:
    name: str
    description: str = None
    # ...

app = FastAPI()


# This section does NOT work
@app.get("/linkml_slot/", response_model=SlotDefinition)
async def linkml_slot(name: str):
    slot = SlotDefinition(name, description='foo')
    return slot

# This section DOES work
@app.get("/simple_slot/", response_model=SimpleSlotDefinition)
async def simple_slot(name: str):
    slot = SimpleSlotDefinition(name, description='foo')
    return slot

#@app.get("/wrapped_slot/", response_model=SimpleSlotDefinition)
#async def wrapped_slot(name: str):
#    slot = SlotDefinition(name, description='foo')
#    slot = SimpleSlotDefinition(**slot)
#    return slot

to test:

pipenv run uvicorn main:app --reload

full stack trace:

pipenv run uvicorn main:app --reload
INFO:     Will watch for changes in these directories: ['/Users/cjm/repos/linkml']
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [31309] using statreload
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/cjm/.local/share/virtualenvs/linkml-6dchTQmP/lib/python3.9/site-packages/uvicorn/subprocess.py", line 76, in subprocess_started
    target(sockets=sockets)
  File "/Users/cjm/.local/share/virtualenvs/linkml-6dchTQmP/lib/python3.9/site-packages/uvicorn/server.py", line 68, in run
    return asyncio.run(self.serve(sockets=sockets))
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/Users/cjm/.local/share/virtualenvs/linkml-6dchTQmP/lib/python3.9/site-packages/uvicorn/server.py", line 76, in serve
    config.load()
  File "/Users/cjm/.local/share/virtualenvs/linkml-6dchTQmP/lib/python3.9/site-packages/uvicorn/config.py", line 448, in load
    self.loaded_app = import_from_string(self.app)
  File "/Users/cjm/.local/share/virtualenvs/linkml-6dchTQmP/lib/python3.9/site-packages/uvicorn/importer.py", line 21, in import_from_string
    module = importlib.import_module(module_str)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 855, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/Users/cjm/repos/linkml/./main.py", line 24, in <module>
    async def linkml_slot(name: str):
  File "/Users/cjm/.local/share/virtualenvs/linkml-6dchTQmP/lib/python3.9/site-packages/fastapi/routing.py", line 582, in decorator
    self.add_api_route(
  File "/Users/cjm/.local/share/virtualenvs/linkml-6dchTQmP/lib/python3.9/site-packages/fastapi/routing.py", line 525, in add_api_route
    route = route_class(
  File "/Users/cjm/.local/share/virtualenvs/linkml-6dchTQmP/lib/python3.9/site-packages/fastapi/routing.py", line 351, in __init__
    self.response_field = create_response_field(
  File "/Users/cjm/.local/share/virtualenvs/linkml-6dchTQmP/lib/python3.9/site-packages/fastapi/utils.py", line 65, in create_response_field
    return response_field(field_info=field_info)
  File "pydantic/fields.py", line 342, in pydantic.fields.ModelField.__init__
  File "pydantic/fields.py", line 456, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 670, in pydantic.fields.ModelField.populate_validators
  File "pydantic/validators.py", line 675, in find_validators
  File "pydantic/dataclasses.py", line 266, in make_dataclass_validator
  File "pydantic/dataclasses.py", line 255, in pydantic.dataclasses.dataclass
  File "pydantic/dataclasses.py", line 250, in pydantic.dataclasses.dataclass.wrap
  File "pydantic/dataclasses.py", line 158, in pydantic.dataclasses._process_class
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/dataclasses.py", line 1021, in dataclass
    return wrap(cls)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/dataclasses.py", line 1013, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash, frozen)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/dataclasses.py", line 927, in _process_class
    _init_fn(flds,
  File "/Users/cjm/.local/share/virtualenvs/linkml-6dchTQmP/lib/python3.9/site-packages/linkml_runtime/utils/dataclass_extensions_376.py", line 24, in dataclasses_init_fn_with_kwargs
    raise TypeError(f'non-default argument {f.name!r} '
TypeError: non-default argument 'mixins' follows default argument

this is the code here: https://github.com/linkml/linkml-runtime/blob/main/linkml_runtime/utils/dataclass_extensions_376.py

Related? linkml/linkml-runtime#73

[deepakunni3]

Very fortuitous to see this bug already reported.

I am encountering something similar for when I am trying to use the dataclasses from LinkML in FastAPI.

[cmungall]

@hsolbrig you were working on this code yesterday to add 3.10 support linkml/linkml-runtime#75

any chance you could take a look at this?

in the interim @deepakunni3 you may be interested in pydanticgen see #172 (comment)

[deepakunni3]

Thanks @cmungall ! Will take a look at pydanticgen

[cmungall]

We have two PRs demoing the problem:

• uses linkml dataclasses, and fails due to the bridge: DO NOT MERGE. example showing fastapi incompatibilities with dataclass_extensions_376 for #486 #487
• uses pydantic, and succeeds: DNM: Demo that fastapi works with pydantic. See also #486 #575

[cmungall]

Hoping this will fix things: linkml/linkml-runtime#168

[cmungall]

OK, I think the problem here is not with dataclass_extensions_376.py -- the above PR shifts the problem to

File "/Users/cjm/repos/linkml/tests/test_issues/test_linkml_issue_486.py", line 17, in <module>
    async def linkml_slot(name: str):
  File "/Users/cjm/Library/Caches/pypoetry/virtualenvs/linkml-lavaHNw6-py3.9/lib/python3.9/site-packages/fastapi/routing.py", line 617, in decorator
    self.add_api_route(
  File "/Users/cjm/Library/Caches/pypoetry/virtualenvs/linkml-lavaHNw6-py3.9/lib/python3.9/site-packages/fastapi/routing.py", line 556, in add_api_route
    route = route_class(
  File "/Users/cjm/Library/Caches/pypoetry/virtualenvs/linkml-lavaHNw6-py3.9/lib/python3.9/site-packages/fastapi/routing.py", line 384, in __init__
    self.response_field = create_response_field(
  File "/Users/cjm/Library/Caches/pypoetry/virtualenvs/linkml-lavaHNw6-py3.9/lib/python3.9/site-packages/fastapi/utils.py", line 69, in create_response_field
    return response_field(field_info=field_info)
  File "pydantic/fields.py", line 419, in pydantic.fields.ModelField.__init__
  File "pydantic/fields.py", line 539, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 801, in pydantic.fields.ModelField.populate_validators
  File "pydantic/validators.py", line 683, in find_validators
  File "pydantic/dataclasses.py", line 266, in make_dataclass_validator
    f'repr={self.repr!r},'
  File "pydantic/dataclasses.py", line 255, in pydantic.dataclasses.dataclass
    if metadata is None else
  File "pydantic/dataclasses.py", line 250, in pydantic.dataclasses.dataclass.wrap
    self.init = init
  File "pydantic/dataclasses.py", line 159, in pydantic.dataclasses._process_class
    # factory will be used.  This is given a nice repr() which will appear
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/dataclasses.py", line 1024, in dataclass
    return wrap(cls)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/dataclasses.py", line 1016, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash, frozen)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/dataclasses.py", line 930, in _process_class
    _init_fn(flds,
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/dataclasses.py", line 506, in _init_fn
    raise TypeError(f'non-default argument {f.name!r} '
TypeError: non-default argument 'mixins' follows default argument 'mixin'

[cmungall]

OK, the problem is that the generated code like:

mixins: Optional[Union[Union[str, SlotDefinitionName], List[Union[str, SlotDefinitionName]]]] = empty_list()

tricks the pydantic dataclass wrapper, it doesn't recognize this as a default field

this is true even if we replace for a more explicit = field(default_factory=list)

Others seem to have the same issue:
pydantic/pydantic#2555

[nlharris]

Is this still relevant?