Reorganize test requirements

[ssbarnea]

• Move test requirements from tox.ini to a test extra
• Removes unused test requirements
• Bumps test requirements to decrease change of running outdated
  version while developing. Avoids New release needed to work with latest pytest-xdist pytest-dev/pytest-cov#422

This enables running of tests w/o tox, with just pip install -e .[test] or similar, something that was not possible before.

Update: Please check https://discuss.python.org/t/is-it-preferable-to-add-test-dependensies-as-an-extra-require-instead-of-tox-ini/4984 thread if you have doubts if this approach may not be good.

[webknjaz]

This env pins shouldn't be in extras. They should be in pip-compile lockfiles instead.

Unactionable

[ssbarnea]

recheck

[odyssey4me]

LGTM

[greg-hellings]

I'm in favor of not being permanently lashed to one particular tool

[webknjaz]

@greg-hellings we are not talking about any tools here. @ssbarnea is trying to shove tests execution virtualenv deps into the metadata of a pip-installable distribution that is completely unrelated to setting up the test environment.

[webknjaz]

Test deps should be (1) installable w/o the dist itself. They should be pinned (2). And those pins should be auto-updatable by tools like dependabot and similar. I don't see how adding them to dist metadata would help here. I'd say that it's harmful to add them to runtime deps. Besides, influencing setuptools in the virtualenv is not a good idea either.

[ssbarnea]

Test deps should be (1) installable w/o the dist itself. They should be pinned (2). And those pins should be auto-updatable by tools like dependabot and similar. I don't see how adding them to dist metadata would help here. I'd say that it's harmful to add them to runtime deps. Besides, influencing setuptools in the virtualenv is not a good idea either.

I already considered that after you introduced them to docs. Still I have some concerns that too tight control over them would translate into few additional issues:

• conflicts where a pinned set of reqs fails on specific platform (one not covered by CI)
• extra noise from dependabot updates -- i really do not want to see dependabot updating setuptools every other day. Now I want to use a pinned version of setuptools, as it would prevent us from discovering bugs with newer ones while testing.
• two extra files in repository root, a place already crowded

Note that I am not strongly against dependabot approach. I only stated my concerns and why I prefer using extras with range instead of pinning. Still, I am willing to accommodate other preferences. Lets see what others think, adding feedback-needed label.

[ssbarnea]

Dropped in favour of #1011 alternative