Fail Infection execution when at least 1 ignore source code regex wasn't matched

[maks-rafalko]

Implements #1604

Will be useful to remove stale/non-matching ignoreSourceCodeByRegex settings inside infection.json and will help debugging them.

TODO

• Do not throw an exception if --git-diff-filter or --filter are used, because mutated files can have no lines for ignore source code regexes (this is why the build is failing atm https://github.com/infection/infection/runs/4650832909?check_suite_focus=true#step:7:29)

[maks-rafalko]

@sanmai I need your help here.

I feel like sanmain/pipeline needs one more method - tap (similar to this one https://lodash.com/docs/#tap)

What I want to do is after Mutants are filtered and we completed building $notMatchedSourceCodeRegexes - I would like to check it, but there is no possibility to do it since we have only filter() method on the Standard class, which iterates over all the items again.

So, desired logic:

take($mutations)
    ->filter(fn (Mutant $mutant): bool {
        // filter ignored mutants out and build $notMatchedSourceCodeRegexes
    })
    // (!) new
    ->tap(fn (array $mutants): array { 
        // check $notMatchedSourceCodeRegexes and throw an exception if needed
        return $mutants;
    })
    ->filter() // continue existing logic - filter not covered by tests mutants

do you have any recommendations here? is it the right way to go?

So, the following diff will reduce the number of iterations from N to 1:

- ->filter(static function (Mutant $mutant) use ($notMatchedSourceCodeRegexes): bool {
+ ->tap(static function (array $mutants) use ($notMatchedSourceCodeRegexes): array {
    if ($notMatchedSourceCodeRegexes !== []) {
        hrow NotMatchedIgnoreSourceCodeRegexFound::forRegexes(array_keys($notMatchedSourceCodeRegexes));
    }

-    return true;
+    return $mutants; 
 })

[sanmai]

The problem here is that there's no array $mutants; there are code paths when we produce them only as required. In other words, worst case there's interable $mutants with who knows how many values in it.

If you want to drop arbitrary parts of a sequence (say, first five, or two after ten), there's a handy slice() method, but I don't think that's what you're looking for here.

[maks-rafalko]

I'm looking for a place (method) where I can do some job after the first filter() but before the second filter()

what are your recommendations?

I can leave it as is, in the filter() method. But from I don't like that we need to iterate over all the values again just to do 1 check and throw an exception if needed

[sanmai]

Since you can't know if you're looking at the very last mutant, you can't throw an exception from any pipeline method here. Neither you should require to iterate over mutants all over again (all memory savings we get will go out of window). Therefore I'd like to suggest to track regex usages (in a dedicated filter() stage always returning true), and only throw at some later stage when we know this regex had no uses. With that said I would much more prefer to see a notice: breaking non-PR builds can be a real pain. (And what about mutants that only appear in PHP 8, will PHP 7 builds fail because of this?)

There's another option: you can inject a fake mutant at the end of the stream, and then catch it somewhere else. I can elaborate further, but this will be incredibly scuffed and very opaque. Simple counting should work better.

[sanmai]

Unmatched regexes should not lead to a non-zero exit status by default as there could be version-dependent mutants which do not always appear and trigger a regex. Showing a notice should be fine, though.

[sanmai]

Could be:

$matchedSourceCodeRegexes[$sourceCodeRegex] += 1;

[sanmai]

This is not going to work because this gets called for every mutant, even for the very first one.