Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--report-type='' implies no --cov-fail-under #199

Conversation

terencehonles
Copy link
Contributor

Specifying --cov-report= will not output any report information and will not check minimum coverage which may be specified with --cov-fail-under=MIN or through the config. This change is to allow using --cov-append in future runs without reporting failure too early.

With this change, on the final build you will need to generate at least --cov-report=term or run coverage report directly


I made this change because I want to declare fail_under in my coverage config (for me it lives in setup.cfg which passes some other common options by using addopts (like mentioned in #195)), but my current test setup requires at least 2 passes running pytest which initially the sequence would be:

ENV_VAR=value pytest --cov-report=
ENV_VAR=other_value pytest --cov-append

However, without the proposed PR the first test fails and I would have to ignore the exit code (which is not appropriate because the test suite may have actually failed)

With this change I can leave fail_under as part of my coverage config instead of having to hardcode it in the CI runner or add an additional file/configuration option.

@terencehonles terencehonles force-pushed the report-less-run-allows-cov-append branch from 4645a0e to 4411c8e Compare April 14, 2018 21:48
@terencehonles terencehonles force-pushed the report-less-run-allows-cov-append branch from 4411c8e to 4e4e525 Compare April 14, 2018 22:17
@ionelmc
Copy link
Member

ionelmc commented Sep 3, 2018

Hmm.. shouldn't we have a negative option, like --cov-disable-reporting ?

I wonder if users will trip up on this empty value (eg: they forget the = - --cov-report tests.py will have undesirable result).

@terencehonles
Copy link
Contributor Author

@ionelmc I'm up for whatever. I've actually re-worked our system to not need two test passes, but I'm willing to alter this to help whoever may need this in the future.

@ionelmc
Copy link
Member

ionelmc commented Mar 25, 2019

@terencehonles agree to just close this? It doesn't sound like you or anyone needs this anymore ...

@graingert
Copy link
Member

graingert commented Sep 2, 2019

@terencehonles you can actually just set:

ENV_VAR=value pytest --cov-fail-under=0
ENV_VAR=other_value pytest --cov-append

@graingert graingert closed this Sep 2, 2019
@terencehonles
Copy link
Contributor Author

@graingert are you sure that works? IIRC I tried that and the value in setup.cfg took precedence to that.

@terencehonles
Copy link
Contributor Author

Well it does seem like the command line flag is working now, so not sure if something changed or I didn't actually try this. Either way, if that works this PR is unnecessary

@terencehonles terencehonles deleted the report-less-run-allows-cov-append branch September 9, 2019 06:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants