1.4.10: test suite is failing

[kloczek]

1.4.9 was OK.

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sqlalchemy-1.4.10-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sqlalchemy-1.4.10-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/python3 -Bm pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.9, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/tkloczko/rpmbuild/BUILD/SQLAlchemy-1.4.10/.hypothesis/examples')
rootdir: /home/tkloczko/rpmbuild/BUILD/SQLAlchemy-1.4.10, configfile: setup.cfg
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, asyncio-0.14.0, expect-1.1.0, cov-2.11.1, mock-3.5.1, httpbin-1.0.0, xdist-2.2.1, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, pyfakefs-4.4.0, freezegun-0.4.2, flaky-3.7.0, cases-3.4.6, hypothesis-6.10.0, relaxed-1.1.5
collected 0 items / 1 error

================================================================================== ERRORS ==================================================================================
______________________________________________________________________ ERROR collecting test session _______________________________________________________________________
Direct construction of SpecModule has been deprecated, please use SpecModule.from_parent.
See https://docs.pytest.org/en/stable/deprecations.html#node-construction-changed-to-node-from-parent for more details.
========================================================================= short test summary info ==========================================================================
ERROR
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================= 1 error in 0.21s =============================================================================

[zzzeek]

I'm guessing this might be something on your platform, assuming you're building for Debian (duh, rpms you're building for fedora :) ) or something like that? Using a pure Python environment built from pypi directly without any packges I can't reproduce this issue, and also SQLAlchemy does not refer to the SpecModule class, so that seems like it might be within some dependencies here:

[classic@photon3 SQLAlchemy-1.4.10:master]$ find lib/sqlalchemy/ -name "*.py" -exec grep -H SpecModule {} \;
[classic@photon3 SQLAlchemy-1.4.10:master]$ 

Here's the start of a straight tox run, that installs the latest pytest and dependencies:

cd SQLAlchemy-1.4.10/
[classic@photon3 SQLAlchemy-1.4.10:master]$ tox
GLOB sdist-make: /home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.4.10/setup.py
py create: /home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.4.10/.tox/py
py installdeps: pytest>=4.6.11,<5.0; python_version < '3', pytest>=6.2; python_version >= '3', pytest-xdist, greenlet != 0.4.17, mock; python_version < '3.3', importlib_metadata; python_version < '3.8'
py inst: /home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.4.10/.tox/.tmp/package/1/SQLAlchemy-1.4.10.zip
py installed: apipkg==1.5,attrs==20.3.0,execnet==1.8.0,greenlet==1.0.0,iniconfig==1.1.1,packaging==20.9,pluggy==0.13.1,py==1.10.0,pyparsing==2.4.7,pytest==6.2.3,pytest-forked==1.3.0,pytest-xdist==2.2.1,SQLAlchemy @ file:///home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.4.10/.tox/.tmp/package/1/SQLAlchemy-1.4.10.zip,toml==0.10.2
py run-test-pre: PYTHONHASHSEED='3210241069'
py run-test: commands[0] | python -m pytest --rootdir /home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.4.10 --log-info=sqlalchemy.testing -n4 --max-worker-restart=5 --nomemory
INFO:sqlalchemy.testing.plugin_base:Adding database URL: sqlite+pysqlite:///:memory:
=================================================== test session starts ===================================================
platform linux -- Python 3.9.2, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.4.10/.tox/py/bin/python
cachedir: .tox/py/.pytest_cache
rootdir: /home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.4.10, configfile: setup.cfg
plugins: xdist-2.2.1, forked-1.3.0
[gw0] linux Python 3.9.2 cwd: /home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.4.10
gw0 C / gw1 I / gw2 I / gw3 IINFO:sqlalchemy.testing.provision:CREATE database test_93745af56d5e, URI sqlite+pysqlite:///:memory:
[gw1] linux Python 3.9.2 cwd: /home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.4.10
gw0 C / gw1 C / gw2 I / gw3 IINFO:sqlalchemy.testing.provision:CREATE database test_fa2cd3209321, URI sqlite+pysqlite:///:memory:
[gw2] linux Python 3.9.2 cwd: /home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.4.10
gw0 C / gw1 C / gw2 C / gw3 IINFO:sqlalchemy.testing.provision:CREATE database test_0c0cb3871082, URI sqlite+pysqlite:///:memory:
[gw3] linux Python 3.9.2 cwd: /home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.4.10
gw0 C / gw1 C / gw2 C / gw3 CINFO:sqlalchemy.testing.provision:CREATE database test_abe136a6a2b3, URI sqlite+pysqlite:///:memory:
[gw0] Python 3.9.2 (default, Feb 20 2021, 00:00:00)  -- [GCC 10.2.1 20201125 (Red Hat 10.2.1-9)]
gw0 ok / gw1 C / gw2 C / gw3 CINFO:sqlalchemy.testing.plugin_base:Adding database URL: sqlite+pysqlite:///:memory:
[gw1] Python 3.9.2 (default, Feb 20 2021, 00:00:00)  -- [GCC 10.2.1 20201125 (Red Hat 10.2.1-9)]
gw0 ok / gw1 ok / gw2 C / gw3 CINFO:sqlalchemy.testing.plugin_base:Adding database URL: sqlite+pysqlite:///:memory:
[gw2] Python 3.9.2 (default, Feb 20 2021, 00:00:00)  -- [GCC 10.2.1 20201125 (Red Hat 10.2.1-9)]
gw0 ok / gw1 ok / gw2 ok / gw3 CINFO:sqlalchemy.testing.plugin_base:Adding database URL: sqlite+pysqlite:///:memory:
[gw3] Python 3.9.2 (default, Feb 20 2021, 00:00:00)  -- [GCC 10.2.1 20201125 (Red Hat 10.2.1-9)]
gw0 ok / gw1 ok / gw2 ok / gw3 okINFO:sqlalchemy.testing.plugin_base:Adding database URL: sqlite+pysqlite:///:memory:
gw0 [14957] / gw1 [14957] / gw2 [14957] / gw3 [14957]
scheduling tests via LoadScheduling

test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_incremental[ticket_6147] 
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[as_declarative.py] 
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[abstract_one.py] 
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_incremental[stubs_14] 
[gw2] [  0%] SKIPPED test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[abstract_one.py] 
[gw0] [  0%] SKIPPED test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_incremental[stubs_14] 
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[as_declarative_base.py] 
[gw0] [  0%] SKIPPED test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[as_declarative_base.py] 
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[complete_orm_no_plugin.py] 
[gw0] [  0%] SKIPPED test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[complete_orm_no_plugin.py] 
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[decl_attrs_one.py] 
[gw0] [  0%] SKIPPED test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[decl_attrs_one.py] 
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[declarative_base_dynamic.py] 
[gw1] [  0%] SKIPPED test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_incremental[ticket_6147] 
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[boolean_col.py] 
[gw1] [  0%] SKIPPED test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[boolean_col.py] 
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[composite_props.py] 
[gw3] [  0%] SKIPPED test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[as_declarative.py] 


... etc

Here's the same command you are running, after I upgraded my pytest to the latest:

[classic@photon3 SQLAlchemy-1.4.10:master]$ python3 -Bm pytest -ra
=================================================== test session starts ===================================================
platform linux -- Python 3.9.2, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /home/classic/.venv3/bin/python3
cachedir: .pytest_cache
rootdir: /home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.4.10, configfile: setup.cfg
plugins: xdist-2.2.1, forked-1.3.0
collected 15019 items                                                                                                     

test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_incremental[stubs_14] PASSED                           [  0%]
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_incremental[ticket_6147] PASSED                        [  0%]
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[abstract_one.py] PASSED                           [  0%]
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[as_declarative.py] PASSED                         [  0%]
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[as_declarative_base.py] PASSED                    [  0%]
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[boolean_col.py] PASSED                            [  0%]
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[cols_noninferred_plain_nonopt.py] PASSED          [  0%]
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[cols_notype_on_fk_col.py] PASSED                  [  0%]
test/ext/mypy/test_mypy_plugin_py3k.py::MypyPluginTest::test_mypy[complete_orm_no_plugin.py] ^C

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
/home/classic/dev/mypy/mypy/semanal.py:4001: KeyboardInterrupt
(to show a full traceback on KeyboardInterrupt use --full-trace)
============================================== 8 passed in 78.05s (0:01:18) ===============================================

etc

nothing has changed in the test suite between 1.4.9 and 1.4.10. it's a very short release.

[zzzeek]

also this is not someting I know about:

hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/tkloczko/rpmbuild/BUILD/SQLAlchemy-1.4.10/.hypothesis/examples')

maybe something in there?

[zzzeek]

this is a lot of plugins I don't use:

plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, asyncio-0.14.0, expect-1.1.0, cov-2.11.1, mock-3.5.1, httpbin-1.0.0, xdist-2.2.1, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, pyfakefs-4.4.0, freezegun-0.4.2, flaky-3.7.0, cases-3.4.6, hypothesis-6.10.0, relaxed-1.1.5

the issue is mentioned here in the relaxed plugin: bitprophet/pytest-relaxed#12

[kloczek]

I'm guessing this might be something on your platform, assuming you're building for Debian (duh, rpms you're building for fedora :)

No, I'm usig my own distribution in which I have many pytjon modules much freshen than in Fedora rawbide.

the issue is mentioned here in the relaxed plugin: bitprophet/pytest-relaxed#12

Kiijs like it may be that ..

[tkloczko@barrel SQLAlchemy-1.4.10]$ grep -r SpecModule /usr/lib/python3.8/site-packages/
grep: /usr/lib/python3.8/site-packages/pytest_relaxed/__pycache__/classes.cpython-38.opt-1.pyc: binary file matches
grep: /usr/lib/python3.8/site-packages/pytest_relaxed/__pycache__/classes.cpython-38.opt-2.pyc: binary file matches
grep: /usr/lib/python3.8/site-packages/pytest_relaxed/__pycache__/classes.cpython-38.pyc: binary file matches
grep: /usr/lib/python3.8/site-packages/pytest_relaxed/__pycache__/plugin.cpython-38.opt-1.pyc: binary file matches
grep: /usr/lib/python3.8/site-packages/pytest_relaxed/__pycache__/plugin.cpython-38.opt-2.pyc: binary file matches
grep: /usr/lib/python3.8/site-packages/pytest_relaxed/__pycache__/plugin.cpython-38.pyc: binary file matches
/usr/lib/python3.8/site-packages/pytest_relaxed/classes.py:    # SpecModule's override.
/usr/lib/python3.8/site-packages/pytest_relaxed/classes.py:class SpecModule(RelaxedMixin, Module):
/usr/lib/python3.8/site-packages/pytest_relaxed/classes.py:        good_name = getattr(super(SpecModule, self), test_func)(obj, name)
/usr/lib/python3.8/site-packages/pytest_relaxed/classes.py:        items = super(SpecModule, self).collect()
/usr/lib/python3.8/site-packages/pytest_relaxed/classes.py:        # TODO: can we unify this with SpecModule's same bits?
/usr/lib/python3.8/site-packages/pytest_relaxed/plugin.py:from .classes import SpecModule
/usr/lib/python3.8/site-packages/pytest_relaxed/plugin.py:        return SpecModule(path, parent)

And this is why it is worth to have implemented in all possible packages fire included test suites because sometimes it is possible expose some issue not in build package but in used tooling :)

[zzzeek]

can you please disable this plugin and ensure test suite passes after that, thanks

[kloczek]

Just for the test removed rpm package with pytest-relaxed module and test suite is OK :)
I'm closing that ticket because that issue has nothing to do with sqlalchemy.

Thank you for your help :)