Tox 4.0 compatibility

[maxnoe]

Description

This pull request is to address compatibility issues with tox 4.0.x released on 2022-12-07.

Fixes #14143

Checklist for package maintainer(s)

This checklist is meant to remind the package maintainer(s) who will review this pull request of some common things to look for. This list is not exhaustive.

• Do the proposed changes actually accomplish desired goals?
• Do the proposed changes follow the Astropy coding guidelines?
• Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
• Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
• Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see "When to rebase and squash commits".
• Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the Extra CI label. Codestyle issues can be fixed by the bot.
• Is a change log needed? If yes, did the change log check pass? If no, add the no-changelog-entry-needed label. If this is a manual backport, use the skip-changelog-checks label unless special changelog handling is necessary.
• Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
• Is a milestone set? Milestone must be set but astropy-bot check might be missing; do not let the green checkmark fool you.
• At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate backport-X.Y.x label(s) before merge.

[pllim]

OK, now the failure is at least consistent with the other repo that is reported at tox-dev/tox#2612

[pllim]

Doesn't look like it is the .tmp vs .temp. Maybe we pin to tox < 4 for now?

[maxnoe]

Unfortunately, we cannot just pin tox, since it comes in via https://github.com/OpenAstronomy/github-actions-workflows/blob/main/.github/workflows/tox.yml

[mhvk]

Grumpy side comment: I wonder where this idea came from that it is good to upgrade tox for every CI run... But mostly: thanks for trying to fix this!!

[Cadair]

I am pretty sure there is a better option than this by using https://github.com/OpenAstronomy/github-actions-workflows/#toxdeps

I can take a swing tomorrow if nobody else beats me.

[Cadair]

Grumpy side comment: I wonder where this idea came from that it is good to upgrade tox for every CI run... But mostly: thanks for trying to fix this!!

We aren't really "upgrading" it we are installing it. The upgrade is really for the other user provided packages.

[maxnoe]

I am pretty sure there is a better option than this by using https://github.com/OpenAstronomy/github-actions-workflows
/#toxdeps

I mean now this is about getting it to run with tox 4, if you can pin it to 3 to get the ci running again, great!

[pllim]

Trying out the pinning at #14142

Different objectives of the PR.

[zacharyburnett]

just chiming in to say the JWST pipeline had a similar issue spacetelescope/jwst#7384

since tox is now using PEP517 to build, the C extensions are not built / linked correctly, but that's another issue; is it possible to force the legacy build without using use_develop?

[gaborbernat]

This PR should remove the pin her ehttps://github.com/maxnoe/astropy/blob/remove_pypi_filter/.github/workflows/ci_workflows.yml#L53, because otherwise you're not testing against tox 4 :-)

[maxnoe]

The remaining failures now are the 32 bit parallel test which fails due to xdist not getting the same tests in each worker process (which I suspect somehow is a tox issue since it passes on tox < 4) and the allowed failures, where there is a scipy issue about a missing optional dependency pooch, which I think is not tox' fault.

[pllim]

Re: 32-bit job -- On our side here, it is not unreasonable to keep the tox pin for that one job because it is going to die soon anyway. However, potential incompatibility with pytest-xdist might still be real but isn't something we can fix here.

[gaborbernat]

Re: 32-bit job -- On our side here, it is not unreasonable to keep the tox pin for that one job because it is going to die soon anyway. However, potential incompatibility with pytest-xdist might still be real but isn't something we can fix here.

Likely this is probably related to some env-var missing, or random seeding, would be great if someone could look into finding out what 😊

[dhomeier]

The remaining failures now are the 32 bit parallel test which fails due to xdist not getting the same tests in each worker process (which I suspect somehow is a tox issue since it passes on tox < 4) and the allowed failures, where there is a scipy issue about a missing optional dependency pooch, which I think is not tox' fault.

Yes, that's likely just a newly pulled in dependency in scipy-dev that needs to be added.

In #14123 I am still getting a 'utf-8' decode error in the Windows job; that's the only place I found where the allowlist_externals might make a difference, though I could not spot any actual difference in the installed packages in the tox run.

[pllim]

Hold your horses... Let numpy release first and make sure nothing breaks with it. That is more important. Thanks!

[dhomeier]

Is numpy 1.24 involved here? It is already released btw.

[pllim]

Yes, we want to make sure CI is green with the new release first without having to also worry about tox upgrade. Thanks for your patience.

[dhomeier]

Will need another rebase to get the numpy 1.24 fix from #14193 in; this should also run with tox 4.0.15 then to resolve tox-dev/tox#2747.

[maxnoe]

The dev dependency tests fail with an error related to downloading a scipy test image and the link check docs build fails also, but I cannot really make out why.

[pllim]

OK CI should be green now in main. Please rebase. Thanks!

[mhvk]

Wonderful that this now works!

[dhomeier]

Looks great now; +1 for opening an issue on PYTHONHASHSEED, and also to keep track of progress on
astrofrog/tox-pypi-filter#8

[maxnoe]

The tests are running, just not in parallel due to fixtures and parametrized Tests using sets with string values, which order depends on the sees and then the pytest xdist test collection fails.

[maxnoe]

@dhomeier @mhvk @pllim merge?

[maxnoe]

Ok, with that the only change in tox.ini is the change from spaces to commas and the removal of pypi filter

[dhomeier]

I've just re-run the CI to check against the current tox 4.3.1, and could not see any no tox-related failures.

[dhomeier]

OK, I have not seen any concerns against removing pypi_filter at this point, so if the last commit passes, let's get this in!

[dhomeier]

same Weekly cron failure on main; thanks for the fix!

[pllim]

I think this patch is incomplete. Two CircleCI jobs that don't run on PR branches now fail, see:

• TST: CircleCI baseline image jobs fail with No module named 'extension_helpers' #14276

[dhomeier]

Maybe should have rebased again... or simply missed

astropy/.circleci/config.yml

Line 61 in b1e6ff8

pip install pip "tox<4"

as it is only running on main? The failed jobs are running tox-3.28.0.

[pllim]

Good catch, @dhomeier . Thanks!