feat(test): run tests with pytest 8.x and 7.x

[zaufi]

Changes in this PR

The recently released pytest 8.0 brings some breaking changes, deprecations, and new features. For future development, it'll be nice to test the plugin with 7.x and 8.x versions of pytest (while not many projects have been ported to 8.x ;-)

Here are the changes to the hatch configuration for this purpose…

[zaufi]

Ping @xymaxim ;-)

[xymaxim]

@zaufi Oh, wow, the new pytest release, and the test are passing!

Wasn't aware of the changes and took a quick look at the changelog.

1. Features and Improvements / Improved Diffs

#11520 The very verbose (-vv) diff output is now colored as a diff instead of a big chunk of red.

Hmm, for some reason, I can’t get pytest to produce a colored diff output for a simple test. Anyway, it’s worth taking a closer look when implementing issue #13 for consistency.

2. Bug Fixes

#11456 Parametrized tests now really do ensure that the ids given to each input are unique - for example, a, a, a0 now results in a1, a2, a0 instead of the previous (buggy) a0, a1, a0. This necessarily means changing nodeids where these were previously colliding, and for readability adds an underscore when non-unique ids end in a number.

This could affect pattern files, so our users potentially need to re-store them after switching to 8.x.

3. “New-style” Hook Wrappers

#11122: pytest now uses “new-style” hook wrappers internally, available since pluggy 1.2.0. See pluggy’s 1.2.0 changelog and the updated docs for details.

Plugins which want to use new-style wrappers can do so if they require pytest>=8.

We don’t use such hook wrappers yet, so just note it for the future.

4. What’s worth looking at is the Collection Changes section.

I plan on messing with this feature this weekend (or next week), if you’re not ahead of me. Right now we have the flat testing layout in tests, and it would be nice to test more tricky cases—haven't figured out how the new changes might affect it.

[zaufi]

My attention was caught by

• #11387: Added the new verbosity_assertions configuration option for fine-grained control of failed assertions verbosity.

Also, I wanted to add type= to options…

[xymaxim]

The recently released pytest 8.0 brings some breaking changes, deprecations, and new features. For future development, it'll be nice to test the plugin with 7.x and 8.x versions of pytest (while not many projects have been ported to 8.x ;-)

Sorry to be late to respond, but I was busy these days. Yeah, it sounds reasonable for me to test two versions for now—already saw issues with migrating to 8.x on GitHub—to support not yet ported projects or projects with pinned dependencies.

Here are the changes to the hatch configuration for this purpose…

Looks good to me, and like how Hatch matrices fit here (no need for tox).

[xymaxim]

My attention was caught by

• #11387: Added the new verbosity_assertions configuration option for fine-grained control of failed assertions verbosity.

This would definitely be nice to use! Sometimes the full output may help to localize/preview non-matched lines in context faster, while other test noise is just a distraction.