Should syntax errors always be shown in the console output?

[romm]

Since release 0.25.0 (PR #1555), the syntax errors are detected and listed in the console output.

From what I understand this feature will essentially be useful for an upcoming feature that will allow users to write their own Mutators, so they can know when invalid mutants are generated. No doubt about that: being able to list the generated syntax errors will be of great help in this kind of situation.

Meanwhile, I'm not sure to understand the benefit of seeing them in a console output:

That's a lot of red X in the console that harden readability and do not bring useful information to me as a user, because those syntax errors were generated by Infection itself and don't prove that my code nor my test suites show some weakness (or maybe I'm wrong?).

I guess hiding them from the console output would be nice, but there are probably things I miss. What are your thoughts, anyway? 🙂

---

PS: I'm familiar with the tool, but not how you manage this repository, if I'm not in the correct section I'm sorry, please tell me!

PS2: a big thanks to you Maks and all contributors for this wonderful tool!

[maks-rafalko]

Hi,

That's a lot of red X in the console that harden readability and do not bring useful information to me as a user, because those syntax errors were generated by Infection itself and don't prove that my code nor my test suites show some weakness (or maybe I'm wrong?).

as soon as you see X in the terminal, it should mean something is wrong either with your custom Mutator (in the future) or with Infection itself.

Probably, what you see is Infection bug, because Infection must not generate syntax errors due to our tests with linting the code

infection/tests/phpunit/Mutator/BaseMutatorTestCase.php

Lines 178 to 182 in 77b684a

	exec(
	sprintf('echo %s | php -l', escapeshellarg($realMutatedCode)),
	$output,
	$returnCode
	);

Could you please check the logs and past results here? I would like to know what Mutators with what code produce syntax errors, or if the repository is public, could you post a link please?

I guess hiding them from the console output would be nice, but there are probably things I miss. What are your thoughts, anyway? slightly_smiling_face

So, when everything is ok: Infection has no bug and your custom Mutator (in the future) does not generate syntax errors - you won't see any X in the output.

Syntax errors should not be ignored, that's why they are like "red flags" in your terminal - they should be immediately investigated because producing syntax errors leads only to wasting of time: mutation is generated and tests are executed against generated Mutants. And always killed.

PS: Thanks for you kind words!

[romm]

After investigating I understood what happened: all syntax errors are actually coming from services that compile PHP code. Some mutant will cause those compilers to write invalid PHP code, that is then eval'd in the tests, and then crash it because of invalid syntax.

After wrapping these tests like below, everything is fine and the mutants get killed normally.

try {
    eval($phpCode);
} catch (\Error $error) {
    self::fail();
}

So in the end: my bad, I really thought some internal Infection mutators would generate invalid mutants but that's not true at all — good news 😄

Anyway thanks for the explanations, now I understand better how it works.

Have a nice evening. 🙂

[romm]

I'm not used to GitHub's discussion feature: should I close this thread somehow, as this was an error by my side? Can't find where to do it though…

[maks-rafalko]

I see only "delete" or "lock" options, so let's leave it as is

Glad now it works for you 👍