Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Code wich uses DeclarativeMeta crashes mypy #9102

Closed
Weyaaron opened this issue Jan 16, 2023 · 8 comments
Closed

Code wich uses DeclarativeMeta crashes mypy #9102

Weyaaron opened this issue Jan 16, 2023 · 8 comments
Labels
bug Something isn't working near-term release addition to the milestone which indicates this should be in a near-term release SQLA mypy plugin mypy plugin issues only. general pep-484 issues should be "typing"
Milestone
1.4.x

Comments

@Weyaaron
Copy link

Describe the bug
The provided code snippet crashes mypy.

Expected behavior
The code snippet should be parsed without a crash.

To Reproduce
Install the provided versions of sql alchemy, this plugin, and mypy.
Create a file test.py with the following content:

from dataclasses import field, dataclass
from typing import Optional

from sqlalchemy import Table, Column, String, Integer
from sqlalchemy.orm import registry
from sqlalchemy.orm.decl_api import DeclarativeMeta


class BackendMeta(metaclass=DeclarativeMeta):
    __abstract__ = True
    mapped_registry = registry()
    metadata = mapped_registry.metadata


@BackendMeta.mapped_registry.mapped
@dataclass
class User:
    __table__ = Table(
        "user",
        BackendMeta.metadata,
        Column("id", Integer, primary_key=True),
        Column("name", String(50)),
        Column("fullname", String(50)),
        Column("nickname", String(12)),
    )
    id: int = field(init=False)
    name: Optional[str] = None
    fullname: Optional[str] = None
    nickname: Optional[str] = None

Error without traceback

test.py:19: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
If this issue continues with mypy master, please report a bug at https://github.com/python/mypy/issues
version: 1.0.0+dev.f957a39f4338e6532dd7381230d07cf51c72f265
test.py:19: : note: please use --show-traceback to print a traceback when reporting a bug

Error with traceback

test.py:19: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.0.0+dev.f957a39f4338e6532dd7381230d07cf51c72f265
Traceback (most recent call last):
  File "/home/aaron/.conda/envs/qse/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/main.py", line 95, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/main.py", line 174, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/build.py", line 194, in build
    result = _build(
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/build.py", line 277, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/build.py", line 2920, in dispatch
    process_graph(graph, manager)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/build.py", line 3317, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/build.py", line 3412, in process_stale_scc
    mypy.semanal_main.semantic_analysis_for_scc(graph, scc, manager.errors)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/semanal_main.py", line 84, in semantic_analysis_for_scc
    process_top_levels(graph, scc, patches)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/semanal_main.py", line 211, in process_top_levels
    deferred, incomplete, progress = semantic_analyze_target(
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/semanal_main.py", line 339, in semantic_analyze_target
    analyzer.refresh_partial(
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/semanal.py", line 596, in refresh_partial
    self.refresh_top_level(node)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/semanal.py", line 607, in refresh_top_level
    self.accept(d)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/semanal.py", line 6173, in accept
    node.accept(self)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/nodes.py", line 1125, in accept
    return visitor.visit_class_def(self)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/semanal.py", line 1534, in visit_class_def
    self.analyze_class(defn)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/semanal.py", line 1604, in analyze_class
    self.configure_base_classes(defn, base_types)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/semanal.py", line 2120, in configure_base_classes
    self.calculate_class_mro(defn, self.object_type)
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/mypy/semanal.py", line 2168, in calculate_class_mro
    hook(ClassDefContext(defn, FakeExpression(), self))
  File "/home/aaron/.conda/envs/qse/lib/python3.9/site-packages/sqlalchemy/ext/mypy/plugin.py", line 187, in _fill_in_decorators
    assert isinstance(target.expr, NameExpr)
AssertionError: 
test.py:19: : note: use --pdb to drop into pdb

Versions.

  • OS: Manjaro
  • Python: 3.9.12
  • SQLAlchemy: 1.4.46
  • mypy: 1.0.0+dev.f957a39f4338e6532dd7381230d07cf51c72f265
  • SQLAlchemy2-stubs: 0.0.2a31

Additional context
This code snippet has been obtained by striping away most of the details from code that has been running fine for years.
We are trying to setup mypy and ran into trouble during that.

Have a nice day!
@Weyaaron Weyaaron added the requires triage New issue that requires categorization label Jan 16, 2023
@CaselIT CaselIT transferred this issue from sqlalchemy/sqlalchemy2-stubs Jan 16, 2023
@CaselIT CaselIT added bug Something isn't working SQLA mypy plugin mypy plugin issues only. general pep-484 issues should be "typing" and removed requires triage New issue that requires categorization labels Jan 16, 2023
@CaselIT
Copy link
Member

CaselIT commented Jan 16, 2023

Hi,

Thanks for reporting. Moving to sqlalchemy repo since the mypy plugin is in sqlalchemy

@zzzeek
Copy link
Member

zzzeek commented Jan 16, 2023

if you are only using the dev version of mypy, mypy just made a change here at python/mypy#6617 (comment) which may or may not be related.

would need to know what versions of mypy reproduce the issue.

@zzzeek zzzeek added this to the 1.4.x milestone Jan 16, 2023
@Weyaaron
Copy link
Author

Weyaaron commented Jan 17, 2023 •
edited

if you are only using the dev version of mypy, mypy just made a change here at python/mypy#6617 (comment) which may or may not be related.

would need to know what versions of mypy reproduce the issue.

Thanks for the quick answer :) This bug happens on the latest release of mypy as well. We noticed it on the release version(0.991)

@zzzeek
Copy link
Member

zzzeek commented Jan 17, 2023

oh ok, so not a regression, just a new issue beginning to use mypy.

Can I perhaps talk you into upgrading to SQLAlchemy 2.0 first before starting efforts with mypy? The approach to typing has changed dramatically and you will get vastly better (and much better supported by me) results in SQLAlchemy 2.0. First release is before the end of the month.

@Weyaaron
Copy link
Author

I agree that this is the best way forward. The project this code is taken from is already fully compliant with 2.0. The code runs just fine, but the type hints are a mess. SQL Alchemy causes ~100 errors across our codebase. These are the remaining errors on our way to 100% mypy compliancy. The matter is not particularly urgent, but our attempts up until today have been futile.

@zzzeek
Copy link
Member

zzzeek commented Jan 17, 2023

oh, forget it, typing in 1.4 was a hack to get something out the door. 2.0 is where I changed the foundational assumptions of everything and introduced an all-new declarative system to get typing to work. You'd have to use 2.0 patterns in your ORM models to get full typing support, though.

overall if things aren't "urgent", let 2.0 come out and then see if you can get onto that release in the next few months, then do the typing.

@zzzeek zzzeek added the near-term release addition to the milestone which indicates this should be in a near-term release label Jan 17, 2023
@sqla-tester
Copy link
Collaborator

Mike Bayer has proposed a fix for this issue in the rel_1_4 branch:

mypy plugin fixes https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/4365

@sqla-tester
Copy link
Collaborator

Mike Bayer has proposed a fix for this issue in the main branch:

mypy plugin fixes https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/4366

sqlalchemy-bot pushed a commit that referenced this issue Jan 18, 2023
@zzzeek
mypy plugin fixes
2a53f70 
Adjustments made to the mypy plugin to accommodate for some potential
changes being made for issue #236 sqlalchemy2-stubs when using SQLAlchemy
1.4. These changes are being kept in sync within SQLAlchemy 2.0.
The changes are also backwards compatible with older versions of
sqlalchemy2-stubs.

Fixed crash in mypy plugin which could occur on both 1.4 and 2.0 versions
if a decorator for the :func:`_orm.registry.mapped` decorator were used
that was referenced in an expression with more than two components (e.g.
``@Backend.mapper_registry.mapped``). This scenario is now ignored; when
using the plugin, the decorator expression needs to be two components (i.e.
``@reg.mapped``).

References: sqlalchemy/sqlalchemy2-stubs#236
Fixes: #9102
Change-Id: Ieb1bf7bf8184645bcd43253e57f1c267b2640537
(cherry picked from commit cf64582f61b15716228302f669322d7efa1003c1)
(cherry picked from commit 36285760238314f70eed4532ca2c2c0c2d684609)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working near-term release addition to the milestone which indicates this should be in a near-term release SQLA mypy plugin mypy plugin issues only. general pep-484 issues should be "typing"
Projects
None yet
Milestone
1.4.x
Development

No branches or pull requests
4 participants
@zzzeek @sqla-tester @CaselIT @Weyaaron