Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark Mutant as killed if Test Framework returns non-zero exit code #1621

Merged
merged 1 commit into from Dec 22, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/Mutant/MutantExecutionResultFactory.php
Expand Up @@ -106,7 +106,7 @@ private function retrieveDetectionStatus(MutantProcess $mutantProcess): string

$output = $this->retrieveProcessOutput($process);

if ($this->testFrameworkAdapter->testsPass($output)) {
if ($process->getExitCode() === 0 && $this->testFrameworkAdapter->testsPass($output)) {
return DetectionStatus::ESCAPED;
}

Expand Down
96 changes: 94 additions & 2 deletions tests/phpunit/Mutant/MutantExecutionResultFactoryTest.php
Expand Up @@ -316,7 +316,7 @@ public function test_it_can_crate_a_result_from_an_escaped_mutant_process(): voi
->willReturn('Tests passed!')
;
$processMock
->expects($this->once())
->expects($this->exactly(2))
->method('getExitCode')
->willReturn(0)
;
Expand Down Expand Up @@ -399,7 +399,7 @@ public function test_it_can_crate_a_result_from_a_killed_mutant_process(): void
->willReturn('Tests failed!')
;
$processMock
->expects($this->once())
->expects($this->exactly(2))
->method('getExitCode')
->willReturn(0)
;
Expand Down Expand Up @@ -463,4 +463,96 @@ public function test_it_can_crate_a_result_from_a_killed_mutant_process(): void
$originalStartingLine
);
}

/**
* PHPUnit can return "Tests passed! OK (10 tests, 32 assertions)" output, however
* return code will be non-zero.
*
* This happens when, for example, symfony/phpunit-bridge is used, and it detects
* outstanding deprecations which fails PHPUnit execution, while all the tests are passing
*
* See https://github.com/infection/infection/issues/1620#issuecomment-999073728
*/
public function test_it_marks_mutant_as_killed_when_tests_pass_from_output_but_exit_code_is_non_zero(): void
{
$processMock = $this->createMock(Process::class);
$processMock
->method('getCommandLine')
->willReturn(
$processCommandLine = 'bin/phpunit --configuration infection-tmp-phpunit.xml --filter "tests/Acme/FooTest.php"'
)
;
$processMock
->method('isTerminated')
->willReturn(true)
;
$processMock
->method('getOutput')
->willReturn('Tests passed! OK (10 tests, 32 assertions)')
;
$processMock
->expects($this->exactly(2))
->method('getExitCode')
->willReturn(1) // PHPUnit says tests passed, but return code is non-zero
;

$this->testFrameworkAdapterMock
->expects($this->never())
->method('testsPass')
->with('Tests passed! OK (10 tests, 32 assertions)')
->willReturn(true)
;

$mutantProcess = new MutantProcess(
$processMock,
MutantBuilder::build(
'/path/to/mutant',
new Mutation(
$originalFilePath = 'path/to/Foo.php',
[],
$mutatorName = MutatorName::getName(For_::class),
[
'startLine' => $originalStartingLine = 10,
'endLine' => 15,
'startTokenPos' => 0,
'endTokenPos' => 8,
'startFilePos' => 2,
'endFilePos' => 4,
],
'Unknown',
MutatedNode::wrap(new Nop()),
0,
[
new TestLocation(
'FooTest::test_it_can_instantiate',
'/path/to/acme/FooTest.php',
0.01
),
]
),
'killed#0',
$mutantDiff = <<<'DIFF'
--- Original
+++ New
@@ @@

- echo 'original';
+ echo 'killed#0';

DIFF,
'<?php $a = 1;'
)
);

$this->assertResultStateIs(
$this->resultFactory->createFromProcess($mutantProcess),
$processCommandLine,
'Tests passed! OK (10 tests, 32 assertions)',
DetectionStatus::KILLED,
$mutantDiff,
$mutatorName,
$originalFilePath,
$originalStartingLine
);
}
}