JUnit formatter incorrectly reports the total number of tests run

[AlexHowansky]

Bug report

Using v1.4.8

When logging via the JUnit formatter, the tests attribute in the testsuite element gets set to the number of failures. It should get set to the total number of tests performed.

Code snippet that reproduces the problem

vendor/bin/phpstan --error-format=junit

This is the log for a run with 50 tests, 1 of which fails:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="1" name="phpstan" tests="1" xmlns:xsi="...">

Expected output

Via this JUnit doc page:

Child elements: <testsuite>
Attributes:
  id: The ID of the scan.
  name: The label of the scan.
  tests: The total number of rules that were applied.
  failures: The total number of rule violations.
  time The time that was required to process all the rules.

Note the number of tests is known to be 50 because of the progress meter:

50/50 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

The expected output is thus:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="1" name="phpstan" tests="50" xmlns:xsi="...">

The main side effect of this issue is that when all tests pass, the log contains tests="0" and a CI/CD pipeline scanning that log reports that nothing was done. For example, here is BitBucket showing the "Static Analysis" step which should have "50 tests passed":

Did PHPStan help you today? Did it make you happy in any way?

[mergeable]

This bug report is missing a link to reproduction on phpstan.org.

It will most likely be closed after manual review.

[ondrejmirtes]

What number would you expect there? I can't think of anything that would make sense.

[AlexHowansky]

The number that the progress bar displays seemed the obvious choice to me. What does that normally represent, the number of files processed?

[mcaskill]

mglaman/phpstan-junit and Psalm both report the suite as having 1 test if there are no failures/errors (links point to their respective code snippets).

Below is a screen capture demonstrating Psalm's "1 tests passed".

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.