When a test appears in multiple suites, Infection errors out

[andrewmy]

Question	Answer
Infection version	0.26.19
Test Framework version	PHPUnit 10.0.14
PHP version	8.1.5 and 8.2.3
Platform	Alpine, Ubuntu
Github Repo	https://github.com/andrewmy/infection-multi-testsuite-issue/

A test may appear in multiple suites as a method of grouping, e.g. Unit + Functional + "Everything". When this happens, Infection errors out, and I think this was introduced in #1172

Reproducer:

• Two different suites, all green: Add one test suite andrewmy/infection-multi-testsuite-issue#2
• Two suites + "All", red: Add a suite and another one for "All" andrewmy/infection-multi-testsuite-issue#3 — see build output in the "cat junit.xml" step and the infection step. Didn't use the infection's built-in initial test run functionality to be able to dump the exact junit.xml contents.

Output with issue

In Assert.php line 2074:
                                          
 Expected a value identical to 1. Got: 2

No stacktrace here, but it pointed to https://github.com/infection/infection/blob/0.26.19/src/TestFramework/Coverage/JUnit/JUnitTestFileDataProvider.php#L84 when it printed it.

One could argue that this is a bad method for grouping tests but I'd say that it should work anyway. Thank you.

[andreaskienast]

I can confirm this issue. In our case we had multiple testsuites configured sharing the same directories. I think it would help to print some additional output, e.g. the used $queryString and $placeholder values.

[maks-rafalko]

what do you mean by $queryString and $placeholder?

[andreaskienast]

The issue happens within src/TestFramework/Coverage/JUnit/JUnitTestFileDataProvider.php in the method getTestFileInfo(). The method iterates the results of self::testCaseMapGenerator(), leading to $queryString and $placeholder variables used to build the DOMNodeList that is checked later on.

[andreaskienast]

Perhaps it even makes more sense to move Assert::same($nodes->length, 1); into the loop, something like

foreach (self::testCaseMapGenerator($fullyQualifiedClassName) as $queryString => $placeholder) {
    $nodes = $xPath->query(sprintf($queryString, $placeholder));

    if ($nodes->length !== 0) {
        Assert::same($nodes->length, 1, sprintf(
            'Found %d occurrences while expecting 1 when querying %s in %s',
                $nodes->length,
                sprintf($queryString, $placeholder),
                $xPath->document->documentURI
            ));
        break;
    }
}