Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: resolve forward refs for inherited dataclasses #2220

Merged

Conversation

PrettyWood
Copy link
Member

@PrettyWood PrettyWood commented Dec 24, 2020

Change Summary

When a stdlib dataclass is used inside a BaseModel, it is converted into a pydantic dataclass. But __annotations__ passed down when converted were not resolved properly so if we used 'Literal...' or postponed annotations by default with from __future__ import annotations, it failed.
The main problem is that we don't set __module__ = _cls.__module__ in the pydantic dataclass but rather __module__ = __name__. This is done to allow pickling. So the resolve_annotations done in the metaclass fails.
To support both I suggest we simply call resolve_annotations when creating the new pydantic dataclass

Related issue number

Checklist

  • Unit tests for the changes exist
  • Tests pass on CI and coverage remains at 100%
  • Documentation reflects the changes where applicable
  • changes/<pull request or issue id>-<github username>.md file added describing change
    (see changes/README.md for details)

@codecov
Copy link

codecov bot commented Dec 24, 2020

Codecov Report

Merging #2220 (6550d0a) into master (13a5c7d) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master     #2220   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           21        21           
  Lines         4199      4200    +1     
  Branches       854       854           
=========================================
+ Hits          4199      4200    +1     
Impacted Files Coverage Δ
pydantic/dataclasses.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 13a5c7d...6550d0a. Read the comment docs.

@andriilahuta
Copy link
Contributor

@PrettyWood thanks for the PR. But I believe it doesn't resolve the original issue as I was using pydantic dataclasses, not stdlib ones.
I was able to workaround it though by calling update_forward_refs manually in __post_init__.

@PrettyWood
Copy link
Member Author

PrettyWood commented Dec 26, 2020

Ah yes thanks for double checking @laevilgenius you're absolutely right!
I caught another issue but yours is still there.
So in your example with a.py and b.py, the issue lies in __annotations__ of dataclass
B.__annotations__ is empty with BaseModel but with pydantic dataclassewe have

# for class A
namespace == {'__annotations__': {'uuid': 'UUID'}, '__module__': 'a', 'uuid': Ellipsis}

# for class B
namespace == {'__annotations__': {'uuid': 'UUID'}, '__module__': '__main__', 'uuid': Ellipsis}

And with postponed evaluation of annotation, we don't have the right module to resolve UUID when calling resolve_annotations in the ModelMetaclass hence the crash...
I don't think we have a way to know if the field 'uuid' is actually the one of the parent or if it has been (re)defined in the class.

Even though this PR still fixes one problem, I would like to tackle the whole "postponed evaluation" thing properly and I'm not convinced of my solution. I hence put the "deferred" label.
Feedback / idea / new PR on this whole topic more than welcome ! 😉

@PrettyWood PrettyWood added the deferred deferred until future release or until something else gets done label Dec 26, 2020
As it doesn't solve the target issue, let's change the PR number
@samuelcolvin
Copy link
Member

@PrettyWood don't we have the same problem on normal models inheriting from BaseModel?

Frankly postponed annotations are a big head ache, I'm not all all clear if we can ever have a completely watertight solution, but since they'll become the default in 3.10 I guess we'll have to do our best.

@PrettyWood
Copy link
Member Author

PrettyWood commented Jan 2, 2021

@samuelcolvin yes we do have the same problem hence the

Even though this PR still fixes one problem, I would like to tackle the whole "postponed evaluation" thing properly

I agree it's a mess but it's something we need to start thinking about. Not directly for v2 but to keep in mind! This PR can probably be closed TBH. But it was a good way for me to start playing with it and have a look at the PEP

@samuelcolvin
Copy link
Member

makes sense, but it looks like this is still an improvement?

If you think it would be better to merge it, I can easily resolve conflicts and merge?

@PrettyWood
Copy link
Member Author

PrettyWood commented Jan 2, 2021

Sorry for the wrong reply I was on my phone yesterday and answered too fast. We don't have the same issue with BaseModel. The explanation as for the why can be seen above in fact.

This PR fixes a problem and should probably be merged yes!
If one day builtin dataclasses support properly postponed annotations, this extra piece of code will probably be useless or a bit different. But it's probably better to wait for stdlib dataclasses to work on it first before trying anything on our side...

@PrettyWood PrettyWood removed the deferred deferred until future release or until something else gets done label Jan 2, 2021
@samuelcolvin samuelcolvin merged commit 7bef40b into pydantic:master Feb 13, 2021
@PrettyWood PrettyWood deleted the dataclass-resolve-forward-refs branch February 13, 2021 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants