Remove xdist from requirements and pytest config

[Drulex]

Checklist

GitHub

• I've given this PR a concise, self-descriptive, and meaningful title
• I've linked relevant issues in the PR body
• I've applied the relevant labels to this PR
• I've assigned a reviewer

PR contents

• I've consulted SCT's internal developer documentation to ensure my contribution is in line with any relevant design decisions
• I've added relevant tests for my contribution
• I've updated the relevant documentation for my changes, including argparse descriptions, docstrings, and ReadTheDocs tutorial pages

Description

This PR removes the default usage of pytest-xdist when running the unit tests. See minutes from dev meeting:

• Motivation: we have hit some limitations/issues with regards to concurrency issues (Session-Scoped Fixtures are not Session-Scoped with Pytest-Xdist pytest-dev/pytest-xdist#271 (comment), Fix test download concurrency bug. #3152). This complicates the ongoing effort to port CI to github actions (Port CI to Github Actions #3125, Fix test download concurrency bug. #3152)

• Other annoyances such as xdist disabling -s (--capture=no) thus preventing the user from viewing the output of stdout which is confusing for users. (Relevant dev Wiki page about this issue.)

  • This issue specifically is causing Julien to disable xdist for local testing, for example.

• Doesn’t add value in CI since almost all VMs are single-core.

• Alternatives:

  • pytest-parallel (to be evaluated)
  • Disabled by default config. People who know what they are doing can enable it (or it can be enabled in ci.sh only).
  • Strategies in Speed up CI #2674, particularly split tests across jobs; but we also need to consider that each job needs to spend installation time (there is a sweet spot in terms of number of tests per job)

• CI time on GH actions: total ~15min; on Travis: ~1hr

  • Largely dominated by waiting for the macOS VM to boot

In a nutshell: there is not a lot of value added (at the moment) in using xdist and working around the issues mentioned above. Tests can happily run sequentially and it won't make a difference for 99% of the people. People who do care about parallelization can still use xdist locally.

[Drulex]

TODO update wiki

[joshuacwnewton]

So, by removing pytest-xdist, we wouldn't need the locking fix from #3152, right? (cc'ing @kousu)

[Drulex]

So, by removing pytest-xdist, we wouldn't need the locking fix from #3152, right? (cc'ing @kousu)

Indeed. @kousu may still contribute the fix upstream if he wants.

[joshuacwnewton]

I've updated the Wiki here: https://github.com/neuropoly/spinalcordtoolbox/wiki/Contributing:-Testing-your-contribution#a-note-about-parallelization

[Drulex]

I've updated the Wiki here: https://github.com/neuropoly/spinalcordtoolbox/wiki/Contributing:-Testing-your-contribution#a-note-about-parallelization

Thanks a lot!