Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

configure pytest to be stricter #1040

Merged
merged 9 commits into from Sep 11, 2020

Conversation

graingert
Copy link
Member

@graingert graingert commented Aug 31, 2020

configure pytest to be stricter and fix all the problems that the increased strictness detects,

eg tests/test_datastructures::TestUploadFile PytestCollectionWarning

scripts/test Outdated Show resolved Hide resolved
Copy link
Member

@JayH5 JayH5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey 👋 thanks for the PR, but I've gone in a slightly different direction to keep things consistent with other encode projects. I believe most of the things in this PR have now been fixed, but to summarise:

Thanks again, and I'd be open to this PR just fixing the first issue in this list, and maybe a separate PR for the second point if that's something you can give a stronger reason for.

@graingert graingert force-pushed the fix-collection-warning-stricter-pytest branch 2 times, most recently from 1ce19ad to 396c018 Compare September 6, 2020 12:01
@graingert
Copy link
Member Author

graingert commented Sep 6, 2020

"Stricter" pytest: you'd need to explain why this is desirable/what problem it solves for you.

It's very possible to accidentally skip tests by adding an __init__ method to the suite

secondly it's good to find and explicitly list the warnings that you're ignoring, rather than just having them fruitlessly print out at the end of the suite.

thirdly you had a SyntaxWarning that until run with a stricter pytest was not noticed.

@graingert
Copy link
Member Author

Filtering deprecation warnings in pytest: I'm not sure we actually want to do this. We probably should keep tabs on these warnings, even if they're in 3rd party libraries.

You were previously filtering all of them, in this PR I'm fixing it so you can keep tabs on them

setup.cfg Outdated Show resolved Hide resolved
setup.cfg Outdated Show resolved Hide resolved
setup.cfg Show resolved Hide resolved
Copy link
Member

@JayH5 JayH5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

secondly it's good to find and explicitly list the warnings that you're ignoring, rather than just having them fruitlessly print out at the end of the suite.

Fair enough, although "it's good to" is not a tangible benefit. I do see a benefit to having an explicit list as it means we know what warnings we expect and when a new one shows up we're more aware of it. On the other hand, somebody then needs to maintain the list of known warnings and it can get stale. I could go either way 🤷‍♂️

thirdly you had a SyntaxWarning that until run with a stricter pytest was not noticed.

Where was that?


This PR looks like a good change, thanks. IMO, parts of Starlette could do with some TLC, and using our test/lint tools to highlight those areas and keep things neat and tidy is a good idea.

Please try to keep it to a minimal set of changes, though. If the PR does exactly what it says in the title then I'm happy.

tests/test_datastructures.py Outdated Show resolved Hide resolved
setup.cfg Outdated Show resolved Hide resolved
setup.cfg Show resolved Hide resolved
starlette/middleware/sessions.py Outdated Show resolved Hide resolved
@graingert
Copy link
Member Author

Where was that?

The missing r"

@graingert graingert changed the title fix tests/test_datastructures::TestUploadFile PytestCollectionWarning and make pytest stricter fix tests/test_datastructures::TestUploadFile PytestCollectionWarning, apply pyupgrade and make pytest stricter Sep 6, 2020
@graingert
Copy link
Member Author

On the other hand, somebody then needs to maintain the list of known warnings and it can get stale.

Yeah it's not so bad as you get notified when you need to add new ones, I've got a ticket for xwarns decorator in pytest too

@JayH5
Copy link
Member

JayH5 commented Sep 6, 2020

If the PR does exactly what it says in the title then I'm happy.

😰 let me be clearer: that doesn't mean add more stuff to the title and then the PR is ok, it means this PR has too much stuff in it. Please try to do one thing at a time in a PR. If a PR has an "and" in the title then it's a good sign that it is doing more than one thing. pyupgrade has nothing to do with pytest and I don't see why it should be included here.

I understand that this is more work for you, but squeezing extra stuff into PRs makes more work for the reviewer. We also squash commits when we merge, so PRs that change multiple unrelated things make it harder to track down the reasons for changes in git history.

@graingert
Copy link
Member Author

@JayH5 originally these are all related to the increase in strictness of pytest

enhance pytest strictness to catch warnings that would have prevented
thest TestUploadFile file warning

restore cov-report and require 100% coverage
@graingert graingert force-pushed the fix-collection-warning-stricter-pytest branch from a3c7f2e to c923a96 Compare September 6, 2020 21:44
@graingert graingert changed the title fix tests/test_datastructures::TestUploadFile PytestCollectionWarning, apply pyupgrade and make pytest stricter make pytest stricter and fix all warnings, eg tests/test_datastructures::TestUploadFile PytestCollectionWarning Sep 6, 2020
@graingert graingert changed the title make pytest stricter and fix all warnings, eg tests/test_datastructures::TestUploadFile PytestCollectionWarning configure pytest to be stricter Sep 6, 2020
@graingert
Copy link
Member Author

@JayH5 I've split out the black and pyupgrade changes: #1049 #1048

starlette/testclient.py Outdated Show resolved Hide resolved
starlette/testclient.py Outdated Show resolved Hide resolved
@graingert graingert force-pushed the fix-collection-warning-stricter-pytest branch from e6fa6e6 to c923a96 Compare September 7, 2020 00:31
@graingert
Copy link
Member Author

to keep things consistent with other encode projects

I'd recommend using a custom pytest-encode plugin that configures all your settings and depends on any other pytest plugins you need across your ecosystem

@graingert graingert requested a review from JayH5 September 7, 2020 00:49
Copy link
Member

@JayH5 JayH5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend using a custom pytest-encode plugin that configures all your settings and depends on any other pytest plugins you need across your ecosystem

That might be nice to have, but unless someone has interest in building/maintaining such a thing it is unlikely to happen for now.

tests/test_datastructures.py Outdated Show resolved Hide resolved
tests/test_datastructures.py Outdated Show resolved Hide resolved
setup.cfg Outdated Show resolved Hide resolved
Co-authored-by: Jamie Hewland <jhewland@gmail.com>
setup.cfg Outdated Show resolved Hide resolved
@graingert graingert requested a review from JayH5 September 9, 2020 23:18
Copy link
Member

@JayH5 JayH5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I've suggested a couple of comments so we can hopefully better keep track of the deprecation warnings. It looks like the context/variable ones we can maybe fix in Starlette, but for a different issue/PR.

I'm not super clear on where comments can go in setup.cfg files so relying on my syntax highlighter to say those are ok.

setup.cfg Show resolved Hide resolved
setup.cfg Show resolved Hide resolved
graingert and others added 2 commits September 10, 2020 21:38
Co-authored-by: Jamie Hewland <jhewland@gmail.com>
Co-authored-by: Jamie Hewland <jhewland@gmail.com>
setup.cfg Outdated Show resolved Hide resolved
Co-authored-by: Jamie Hewland <jhewland@gmail.com>
Copy link
Member

@JayH5 JayH5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We got there eventually 🎉 thanks for your patience

@JayH5 JayH5 merged commit b043fe5 into encode:master Sep 11, 2020
@graingert graingert deleted the fix-collection-warning-stricter-pytest branch September 11, 2020 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants