[wip] add sphinx version to test matrix

[bollwyvl]

This takes a swing at #446, adding the sphinx version to the CI matrix.

Have started noting some issues on e.g. conda-forge/pydata-sphinx-theme-feedstock#24

[bollwyvl]

Some options:

• add sphinx major version to fixture path
• skip fixture tests for older versions

[choldgraf]

My feeling is that we should support the latest 2 versions of Sphinx (maybe with a 6 month buffer for when new major versions are released). To that extent, I'd say we remove the Sphinx v2 tests from this. What do others think about that?

[choldgraf]

This is generally looking pretty good to me! Thoughts on only running the latest two major versions of sphinx? I think that will be easier to sustain...

[jorisvandenbossche]

Yes, I agree that testing for the last two major versions (so now 3.x and 4.x) certainly feels sufficient.

We still need a solution for the fixture tests, though (where the generated output differs depending on the sphinx version)

[choldgraf]

In my opinion, we should only test fixtures on the latest version of Sphinx (maybe making a switch 3 months after a new major version is released). Otherwise, to update fixtures you would need to run pytest twice with two different versions of Sphinx, which feels very cumbersome to me.

So I guess what I'm suggesting is that we have strong support for the latest version of Sphinx, and softer but still decent support for the 2nd-to-last, and no guaranteed support beyond that

[jorisvandenbossche]

Yes, sounds good

[choldgraf]

@jorisvandenbossche I wrote up a short proposal in this issue, so we have something tangible to iterate on: #567

If you're generally +1 I can turn that into a quick PR to our contributing docs, and then follow up after with #547 and #532