Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detailed xfail output sometimes distracts from unexpected failures #1891

Open
EliahKagan opened this issue Mar 31, 2024 · 0 comments
Open

Detailed xfail output sometimes distracts from unexpected failures #1891

EliahKagan opened this issue Mar 31, 2024 · 0 comments

Comments

@EliahKagan
Copy link
Contributor

EliahKagan commented Mar 31, 2024

This issue is to track the problem noticed in #1853 and #1862 (see comments in both) where the specific details of new test failures are made less obvious in reading pytest output, in CI checks, due to the otherwise useful pytest 8 feature to show more such details.

From #1853 (comment):

pytest output now contains details of tests with xfail (expected failure) status, which are unrelated, and that may have distracted from the related test's failure. This changed in major version 8 of pytest (which GitPython is using automatically except on Python 3.7 which doesn't support it). Overall, for most projects, it seems to me that this is an improvement, since it's useful to be able to see the specific details of xfailing tests without extra effort (and check if they really match up with the messages written in the xfail decorators). However, it may be that for GitPython's tests it makes common tasks more cumbersome. If so, then the pytest configuration can be changed (this does not require downgrading pytest).

I'm unsure what the best solution is, but if necessary the -ra in

addopts = "--cov=git --cov-report=term --disable-warnings -ra"

can be changed to something that does not imply x (xfail). The "full list of available characters" in the Producing a detailed summary report section of the documentation documents how to specify which test statuses to emit detailed output for.

I'm unsure of what should be done and I don't know of an easy way to eliminate the distracting effects while still having the benefit of seeing specific output of xfailing tests.

@EliahKagan EliahKagan changed the title Detailed xfail tracebacks sometimes distract from unexpected failures Detailed xfail output sometimes distracts from unexpected failures Mar 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

2 participants