Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use new raise syntax in one case #5481

Merged
merged 1 commit into from Jun 24, 2019
Merged

Use new raise syntax in one case #5481

merged 1 commit into from Jun 24, 2019

Conversation

asottile
Copy link
Member

No description provided.

Copy link
Member

@nicoddemus nicoddemus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@codecov
Copy link

codecov bot commented Jun 24, 2019

Codecov Report

Merging #5481 into master will increase coverage by 0.2%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #5481     +/-   ##
=========================================
+ Coverage   95.86%   96.07%   +0.2%     
=========================================
  Files         115      115             
  Lines       25458    25455      -3     
  Branches     2468     2467      -1     
=========================================
+ Hits        24405    24455     +50     
+ Misses        746      697     -49     
+ Partials      307      303      -4
Impacted Files Coverage Δ
src/_pytest/outcomes.py 95.38% <100%> (-0.21%) ⬇️
src/_pytest/terminal.py 93.02% <0%> (+0.42%) ⬆️
testing/test_assertrewrite.py 84.03% <0%> (+1.5%) ⬆️
src/_pytest/pytester.py 90.96% <0%> (+5.92%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 61dcb84...380ca8f. Read the comment docs.

@asottile asottile merged commit a24933b into pytest-dev:master Jun 24, 2019
@asottile asottile deleted the minor_py3_cleanup branch June 24, 2019 19:07
@bsipocz
Copy link

bsipocz commented Jul 2, 2019

This PR has introduced regression in astropy testing, and now we see errors like:

______________________________ ERROR collecting astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py _______________________________
ImportError while importing test module '/private/var/folders/dc/hsm7tqpx2d57n7vb3k1l81xw0000gq/T/astropy-test-mc9tylff/lib/python3.7/site-packages/astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py:7: in <module>
    from asdf.tests.helpers import assert_roundtrip_tree
E   ModuleNotFoundError: No module named 'asdf.tests.helpers'

example file: https://github.com/astropy/astropy/blob/master/astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py

I've double checked and reverting this change on top of current pytest master everything works as it used to.

I would be also interested in suggested ways to go around this issue, but I got stack as it seems that the importorskip is not prepared for from x import y imports.

@blueyed
Copy link
Contributor

blueyed commented Jul 2, 2019

Does it work with raise Skipped(reason, allow_module_level=True) (removing from None)?

@bsipocz
Copy link

bsipocz commented Jul 2, 2019

Indeed, it does. Thanks for the quick diagnostic.

@nicoddemus
Copy link
Member

@bsipocz just to check: reverting the change causes the whole module to be skipped, is that it?

@bsipocz
Copy link

bsipocz commented Jul 2, 2019

yes, indeed the module is not listed in the collections list, which is unfortunate. Though having the error means that nothing is collected either.

@nicoddemus
Copy link
Member

I'm baffled, I get the correct behavior on Windows:

====================== short test summary info ======================
SKIPPED [1] x:\astropy\astropy\io\misc\asdf\tags\coordinates\tests\test_earthlocation.py:7: could not import 'asdf': No module named 'asdf'
=============== 1 skipped, 1 warnings in 0.14 seconds ===============

🤔

@blueyed
Copy link
Contributor

blueyed commented Jul 2, 2019

@bsipocz @nicoddemus
Which Python version do you use?

@nicoddemus
Copy link
Member

I've tested with both 3.6 and 3.7 (based that @bsipocz is using 3.7 from the traceback)

@bsipocz
Copy link

bsipocz commented Jul 3, 2019

Here is a full header for the run with the patched pytest:

============================================================== test session starts ==============================================================
platform darwin -- Python 3.7.3, pytest-5.0.1.dev17+g50b846e9d.d20190702, py-1.8.0, pluggy-0.12.0

Running tests with Astropy version 4.0.dev25135.
Running tests in astropy/io/misc.

Date: 2019-07-02T16:27:23

Platform: Darwin-17.7.0-x86_64-i386-64bit

Executable: /usr/local/opt/python/bin/python3.7

Full Python Version: 
3.7.3 (default, Mar 27 2019, 09:23:39) 
[Clang 10.0.0 (clang-1000.11.45.5)]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Numpy: 1.16.4
Scipy: 1.3.0
Matplotlib: 3.1.0.post1209+g6ba265188
h5py: 2.8.0
Pandas: 0.24.2
Cython: 0.29.10
Scikit-image: 0.15.0
astropy_helpers: 3.2.1
Using Astropy options: remote_data: none.

rootdir: /private/var/folders/dc/hsm7tqpx2d57n7vb3k1l81xw0000gq/T/astropy-test-xwcwduux/lib/python3.7/site-packages, inifile: setup.cfg
plugins: forked-1.0.2, arraydiff-0.3, cov-2.6.0, xonsh-0.8.3, dependency-0.4.0, doctestplus-0.3.0, requests-mock-1.6.0, remotedata-0.3.1, openfiles-0.4.0.dev0
collected 114 items / 28 skipped / 86 selected                                                                                                  

astropy/io/misc/yaml.py .                                                                                                                 [  0%]
astropy/io/misc/tests/test_hdf5.py ........................................................ssss...........ss.                             [ 65%]
astropy/io/misc/tests/test_pandas.py .....                                                                                                [ 70%]
astropy/io/misc/tests/test_pickle_helpers.py ....                                                                                         [ 73%]
astropy/io/misc/tests/test_yaml.py ..............................   

@bsipocz
Copy link

bsipocz commented Jul 3, 2019

And this is the failing one:

============================================================== test session starts ==============================================================
platform darwin -- Python 3.7.3, pytest-5.0.0, py-1.8.0, pluggy-0.12.0

Running tests with Astropy version 4.0.dev25135.
Running tests in astropy/io/misc.

Date: 2019-07-02T15:07:07

Platform: Darwin-17.7.0-x86_64-i386-64bit

Executable: /usr/local/opt/python/bin/python3.7

Full Python Version: 
3.7.3 (default, Mar 27 2019, 09:23:39) 
[Clang 10.0.0 (clang-1000.11.45.5)]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Numpy: 1.16.4
Scipy: 1.3.0
Matplotlib: 3.1.0.post1209+g6ba265188
h5py: 2.8.0
Pandas: 0.24.2
Cython: 0.29.10
Scikit-image: 0.15.0
astropy_helpers: 3.2.1
Using Astropy options: remote_data: none.

rootdir: /private/var/folders/dc/hsm7tqpx2d57n7vb3k1l81xw0000gq/T/astropy-test-mc9tylff/lib/python3.7/site-packages, inifile: setup.cfg
plugins: openfiles-0.3.2, forked-1.0.2, arraydiff-0.3, cov-2.6.0, xonsh-0.8.3, dependency-0.4.0, doctestplus-0.3.0, remotedata-0.3.1, requests-mock-1.6.0
collected 114 items / 3 errors / 22 skipped / 89 selected                                                                                       

==================================================================== ERRORS =====================================================================
______________________________ ERROR collecting astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py _______________________________
ImportError while importing test module '/private/var/folders/dc/hsm7tqpx2d57n7vb3k1l81xw0000gq/T/astropy-test-mc9tylff/lib/python3.7/site-packages/astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py:7: in <module>
    from asdf.tests.helpers import assert_roundtrip_tree
E   ModuleNotFoundError: No module named 'asdf.tests.helpers'
_________________________________ ERROR collecting astropy/io/misc/asdf/tags/coordinates/tests/test_skycoord.py _________________________________
ImportError while importing test module '/private/var/folders/dc/hsm7tqpx2d57n7vb3k1l81xw0000gq/T/astropy-test-mc9tylff/lib/python3.7/site-packages/astropy/io/misc/asdf/tags/coordinates/tests/test_skycoord.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
astropy/io/misc/asdf/tags/coordinates/tests/test_skycoord.py:12: in <module>
    from asdf.tests.helpers import assert_roundtrip_tree
E   ModuleNotFoundError: No module named 'asdf.tests.helpers'
____________________________________ ERROR collecting astropy/io/misc/asdf/tags/time/tests/test_timedelta.py ____________________________________
ImportError while importing test module '/private/var/folders/dc/hsm7tqpx2d57n7vb3k1l81xw0000gq/T/astropy-test-mc9tylff/lib/python3.7/site-packages/astropy/io/misc/asdf/tags/time/tests/test_timedelta.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
astropy/io/misc/asdf/tags/time/tests/test_timedelta.py:7: in <module>
    from asdf.tests.helpers import assert_roundtrip_tree
E   ModuleNotFoundError: No module named 'asdf.tests.helpers'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 3 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
====================================================== 22 skipped, 3 error in 4.86 seconds ======================================================

@nicoddemus
Copy link
Member

nicoddemus commented Jul 3, 2019

@bsipocz Could you please also post the full output of your failed run? Thanks!

My header:

======================== test session starts ========================
platform win32 -- Python 3.7.0, pytest-5.0.0, py-1.8.0, pluggy-0.12.0

I will clock out now, I need to unwind a bit before bed. I will continue this tomorrow. 👍

Thanks for the report @bsipocz, and sorry for the trouble!

@bsipocz
Copy link

bsipocz commented Jul 3, 2019

Thanks for looking into this. And no trouble at all, we have other issues with the new release, too fixes are needed on our side, too. :)

@asottile
Copy link
Member Author

asottile commented Jul 3, 2019

weird, works fine for me:

$ pytest astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py 
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-5.0.0, py-1.8.0, pluggy-0.12.0

Running tests with Astropy version 4.0.dev1.
Running tests in astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py.

Date: 2019-07-02T22:50:21

Platform: Linux-4.15.0-54-generic-x86_64-with-Ubuntu-18.04-bionic

Executable: /tmp/astropy2/venv/bin/python3

Full Python Version: 
3.6.8 (default, Jan 14 2019, 11:02:34) 
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Numpy: 1.16.4
Scipy: not available
Matplotlib: not available
h5py: not available
Pandas: not available
Cython: not available
Scikit-image: not available
astropy_helpers: 3.2.1

rootdir: /tmp/astropy2, inifile: setup.cfg
collected 0 items / 1 skipped                                                  

=============================== warnings summary ===============================
venv/lib/python3.6/site-packages/_pytest/config/__init__.py:513
  /tmp/astropy2/venv/lib/python3.6/site-packages/_pytest/config/__init__.py:513: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: astropy.tests.plugins.display
    self.import_plugin(import_spec)

-- Docs: https://docs.pytest.org/en/latest/warnings.html
==================== 1 skipped, 1 warnings in 0.10 seconds =====================
$ pip freeze --all
-e git+git@github.com:astropy/astropy@a668c168eb1f136829547866aa45240c4152cfd9#egg=astropy
atomicwrites==1.3.0
attrs==19.1.0
importlib-metadata==0.18
more-itertools==7.1.0
numpy==1.16.4
packaging==19.0
pip==19.1.1
pluggy==0.12.0
py==1.8.0
pyparsing==2.4.0
pytest==5.0.0
setuptools==41.0.1
six==1.12.0
wcwidth==0.1.7
wheel==0.33.4
zipp==0.5.1

@asottile
Copy link
Member Author

asottile commented Jul 3, 2019

The only way I can reproduce this is if I have a busted local copy of asdf:

$ tree asdf/
asdf/
├── __init__.py
└── tests

1 directory, 1 file
$ pytest astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py  -rs
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-5.0.0, py-1.8.0, pluggy-0.12.0

Running tests with Astropy version 4.0.dev1.
Running tests in astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py.

Date: 2019-07-02T22:57:07

Platform: Linux-4.15.0-54-generic-x86_64-with-Ubuntu-18.04-bionic

Executable: /tmp/astropy2/venv/bin/python3

Full Python Version: 
3.6.8 (default, Jan 14 2019, 11:02:34) 
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Numpy: 1.16.4
Scipy: not available
Matplotlib: not available
h5py: not available
Pandas: not available
Cython: not available
Scikit-image: not available
astropy_helpers: 3.2.1

rootdir: /tmp/astropy2, inifile: setup.cfg
collected 0 items / 1 errors                                                   

==================================== ERRORS ====================================
_ ERROR collecting astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py _
ImportError while importing test module '/tmp/astropy2/astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
astropy/io/misc/asdf/tags/coordinates/tests/test_earthlocation.py:7: in <module>
    from asdf.tests.helpers import assert_roundtrip_tree
E   ModuleNotFoundError: No module named 'asdf.tests.helpers'
=============================== warnings summary ===============================
venv/lib/python3.6/site-packages/_pytest/config/__init__.py:513
  /tmp/astropy2/venv/lib/python3.6/site-packages/_pytest/config/__init__.py:513: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: astropy.tests.plugins.display
    self.import_plugin(import_spec)

-- Docs: https://docs.pytest.org/en/latest/warnings.html
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
===================== 1 warnings, 1 error in 0.18 seconds ======================

@bsipocz could you also provide ls -al and if there isn't anything there, python -c 'import asdf; print(asdf)'

@asottile
Copy link
Member Author

asottile commented Jul 3, 2019

(and even then, changing the raise syntax makes no difference)

@blueyed
Copy link
Contributor

blueyed commented Jul 3, 2019

changing the raise syntax makes no difference

Yeah - I would not really have expected it to make a difference either.

@bsipocz
Might indicate that there's another issue, e.g. stale .pyc files (try find -name "*.pyc" -delete, but create a backup maybe for investigating. Also tox etc might be helpful to make this more reproducible).

@nicoddemus
Copy link
Member

Yeah - I would not really have expected it to make a difference either.

Yeah, looking at the code I can't figure out how that change could have affected importorskip at all.

@bsipocz
Copy link

bsipocz commented Jul 3, 2019

This is now getting very weird as I cannot reproduce the original issue any more (with pytest v5.0.0) after making sure the repo is git cleaned.
However it still true that when importorskip-ed then no files are listed only the total number of tests. Edit: never mind, this was yet another misunderstanding in my local testing. The downside of doing tests with a huge library.

@nicoddemus
Copy link
Member

However it still true that when importorskip-ed then no files are listed only the total number of tests.

What do you mean? Can you also please post the full output you are getting?

@bsipocz
Copy link

bsipocz commented Jul 3, 2019

Let me open a PR in astropy to remove the version limitations, so we'll have clean VMs and multiple setups as now I don't trust my local any more.

@bsipocz
Copy link

bsipocz commented Jul 3, 2019

OK, so here is a run with asdf installed: https://travis-ci.org/astropy/astropy/jobs/553924069#L1446
and here is one without: https://travis-ci.org/astropy/astropy/jobs/553951083#L1710

It would be useful if the asdf files would be listed in the second case filled with 's' flags.

@asottile
Copy link
Member Author

asottile commented Jul 3, 2019

ok but that's not a regression, and certainly not caused by this 😆

I believe the reason they don't show up is they're excluded at discovery time instead of at runtime -- there's no "test" to skip (because the whole module is skipped)

@bsipocz
Copy link

bsipocz commented Jul 3, 2019

Yes, the regression I run into was something weird stuff locally and can't reproduce now (while it was totally consistent yesterday.). I'm sorry for the noise and false alarm. Switching to use pytest 5 at the end wasn't as painful as we first thought 😅

@asottile
Copy link
Member Author

asottile commented Jul 3, 2019

🎉 🎉 🎉

it might be possible to include skipped tests in the output one way or another -- perhaps open a separate issue for that?

@nicoddemus
Copy link
Member

I'm sorry for the noise and false alarm.

No problem at all, glad it was not a real problem in the end! 😁

Switching to use pytest 5 at the end wasn't as painful as we first thought 😅

Great to hear. Indeed this release should not cause big surprises other than dropping py27/py34 and deprecated features. 👍

@bsipocz
Copy link

bsipocz commented Jul 3, 2019

Originally we had 90+ failures, but at the end all distilled down to the changes due to #5412.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants