Tests failing with pytest 7.2.0

[dstansby]

Describe the bug

With pytest 7.2.0 the following tests are failing:

 FAILED ../../.tox/py310/lib/python3.10/site-packages/sunpy/map/tests/test_map_factory.py::test_data_filenames
FAILED ../../.tox/py310/lib/python3.10/site-packages/sunpy/io/tests/test_fits.py::test_data_filenames
FAILED ../../.tox/py310/lib/python3.10/site-packages/sunpy/map/sources/tests/test_eit_source.py::test_data_filenames
FAILED ../../.tox/py310/lib/python3.10/site-packages/sunpy/physics/tests/test_differential_rotation.py::test_differential_rotation
FAILED ../../.tox/py310/lib/python3.10/site-packages/sunpy/timeseries/tests/test_timeseries_factory.py::test_data_filenames

with messages like:

which will be an error in a future version of pytest.  Did you mean to use `assert` instead of `return`?

I'm guessing these tests are unintentionally returning something?

I'm going to fix this by pinning pytest<7.2 on the 4.0 branch to get 4.0.6 out, but when a fix is merged to main and backported we should remember to remove that pytest pin on the 4.0 branch.

To Reproduce

import sunpy

sunpy.map.Map(...)
etc

Screenshots

No response

System Details

No response

Installation method

No response

[dstansby]

We're now getting this mysterious warning with the new release of pytest-xdist (pytest-dev/pytest-xdist#828 is the PR where the warning came in I think):

INTERNALERROR>     warnings.warn(warning, stacklevel=stacklevel)
INTERNALERROR> DeprecationWarning: The --rsyncdir command line argument and rsyncdirs config variable are deprecated.
INTERNALERROR> The rsync feature will be removed in pytest-xdist 4.0.

[ConorMacBride]

The test_data_filenames errors have been fixed by #6507.

test_differential_rotation needs astropy/pytest-arraydiff#36 to be released, however, I don't think we are actually running pytest-arraydiff. It needs to be enabled with --arraydiff. Do we want to enable it? When I do that the test fails with:

Not equal to tolerance rtol=1e-07, atol=0

Mismatched elements: 10 / 16384 (0.061%)
Max absolute difference: 0.001
Max relative difference: 5.55484577e-06

[nabobalis]

Oh did we mark the test and then never run it properly?

We should enable it or refactor the test.

[ayshih]

Heh, test_differential_rotation traces back to #3245 (Jun 2019), and then I updated the differential-rotation code in #4979 (Feb 2021). Sometime between those two PRs, we must have stopped enabling pytest-arraydiff in the CI, because the test wasn't updated accordingly in #4979.

[dstansby]

I can't work out where, but at some point I think this was fixed - with pytest 7.2.1 the tests I mentioned in the original issue seem to work fine for me now locally.