Update Testing Docs regarding Primer

[clavedeluna]

Bug description

Primer testing docs mention batch_one and batch_two:

at the very least, the suggested command with batch_two doesn't run any tests.

but I don't find any batch_two in the repo. Should this documentation be updated?

Configuration

No response

Command used

n/a

Pylint output

n/a

Expected behavior

n/a

Pylint version

n/a

OS / Environment

n/a

Additional dependencies

No response

[jacobtylerwalls]

Yes, this is very out of date. Not only is there no longer a batch two, we aren't even running batch one anymore on CI the batch one packages are only checked for crashes, they don't get their messages diffed. The new primer that diffs messages is in tests/primer/__main__.py

[mbyrnepr2]

These are the latest version of the docs: https://pylint.pycqa.org/en/latest/development_guide/contributor_guide/tests/launching_test.html#primer-tests

[clavedeluna]

Ah! No I think I actually made a mistake. The link I provided
https://pylint.pycqa.org/en/v2.12.2/development_guide/testing.html#primer-tests
is clearly for an old version. Google just took me to that and I didn't notice.

Given https://pylint.pycqa.org/en/v2.15.5/development_guide/contributor_guide/tests/launching_test.html exists and has the correct docs, I'll close this issue.

Thanks for clarifying!

[jacobtylerwalls]

That documentation is still out of date. It only refers to the old primer.

[clavedeluna]

See #7685 (comment) for how to run primer

[mbyrnepr2]

Would it be an idea to add the primer commands to tox and document how to run the primer using tox?

[Pierre-Sassoulas]

That could work but primer are slow and tox also make things slower. (We'd have to check that the primed repoes are not downloaded by tox each time). I guess new contributor would appreciate to have a tox -e primer command and not having to read the full doc, and maintainer will know the faster command.

Regardless of what we do to make it easier to launch primer, we can close this issue once the primer are documented more clearly in the doc.

[clavedeluna]

I would be all for documenting the full commands

python tests/primer/__main__.py prepare --clone
python tests/primer/__main__.py run --type=pr

while also adding to tox and indicating both can be used.

As a frequent contributor, it took me waaay to long to figure out how to run primer locally...

[Pierre-Sassoulas]

Also it should probably be launchable with a single command and no argument (python tests/primer/__main__.py).

[DanielNoord]

It was a conscious decision to separate the commands to be able to easily reuse currently cloned project.

We could add an full-run parser but good documentation would probably be more effective and easier.

[Pierre-Sassoulas]

It was a conscious decision to separate the commands to be able to easily reuse currently cloned project.

Sure and it's a good thing, but we can auto-detect that the repositories were already cloned.