Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with BaseModel if used with importlib #2363

Closed
3 tasks done
vfdev-5 opened this issue Feb 14, 2021 · 2 comments · Fixed by #2366
Closed
3 tasks done

Issue with BaseModel if used with importlib #2363

vfdev-5 opened this issue Feb 14, 2021 · 2 comments · Fixed by #2366
Labels
bug V1 Bug related to Pydantic V1.X

Comments

@vfdev-5
Copy link

vfdev-5 commented Feb 14, 2021

Checks

  • I added a descriptive title to this issue
  • I have searched (google, github) for similar issues and couldn't find anything
  • I have read and followed the docs and still think this is a bug

Bug

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

             pydantic version: 1.7.3
            pydantic compiled: True
                 install path: /opt/conda/lib/python3.8/site-packages/pydantic
               python version: 3.8.2 (default, Mar 25 2020, 17:03:02)  [GCC 7.3.0]
                     platform: Linux-4.15.0-130-generic-x86_64-with-glibc2.10
     optional deps. installed: ['typing-extensions']

If I'm trying to load a module using importlib where a BaseModel is used then execution is crashed with the following traceback:

Traceback (most recent call last):
  File "loader.py", line 13, in <module>
    m = load_module("module_to_load.py")
  File "loader.py", line 7, in load_module
    spec.loader.exec_module(module)  # type: ignore[union-attr]
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "module_to_load.py", line 4, in <module>
    class User(BaseModel):
  File "pydantic/main.py", line 274, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/typing.py", line 293, in pydantic.typing.resolve_annotations
    def __copy__(self):

How to repro:

  1. create loader.py file with the following content
import importlib.util

def load_module(fpath):
    spec = importlib.util.spec_from_file_location("mymodule", fpath)
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)  # type: ignore[union-attr]
    return module

if __name__ == "__main__":    
    m = load_module("module_to_load.py")
    print(m)
    m.foo()
  1. create module_to_load.py :
from pydantic import BaseModel

class User(BaseModel):
    id: int
    name = 'Jane Doe'

def foo():
    print(User(id=12, name="test"))
  1. Run python loader.py

As use-case may seem strange, I'm a bit hestiating between a bug and wrong usage, but in the latter case, error message is unclear at all. Important part is that I have to import the module in this way and not in any other more standard python way in which pydantic indeed work without any issue.
Thanks

EDIT:

Indeed, if we add sys.modules["mymodule"] = module to load_module as suggested here, there is no issue, anymore.
I have to figure out what would be the impact of doing that for my goal.

@samuelcolvin
Copy link
Member

samuelcolvin commented Feb 15, 2021

I've created a PR to make resolve_annotations more lenient, see #2366.

But actually the fix for this is to use a simplified approach to loading the module, just

from importlib.machinery import SourceFileLoader

def load_module(fpath):
    return SourceFileLoader('mymodule', fpath).load_module()

...

Works fine.

@vfdev-5
Copy link
Author

vfdev-5 commented Feb 15, 2021

Thanks for the suggestion and for the code update !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug V1 Bug related to Pydantic V1.X
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants