Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set XDEBUG_MODE for processes with coverage #1518

Merged
merged 43 commits into from Nov 10, 2021

Conversation

sanmai
Copy link
Member

@sanmai sanmai commented May 4, 2021

This PR:

Testing steps:

  • Note how build failed on aa1e089 when we disabled Xdebug

Implementation notes:

  • PHPUnit 9.4.2 doesn't really work well with Xdebug 3 (even with XDEBUG_MODE=coverage it emits a warning "xdebug.mode=coverage has to be set in php.ini" and fails to generate coverage). Upgrading to 9.5.10 fixes the problem.

*/
public function start(?callable $callback = null, ?array $env = null): void
{
if (!extension_loaded('pcov')) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we also support phpdbg

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can add a check for it, but they are mutually exclusive. You can't use Xdebug with PHPDBG, IIRC, so whenever we export or not this variable does not matter.

On the contrary, you can have both Xdebug and PCOV. And that's why we're making sure of not breaking stuff.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm voting to add add phpdbg check to this condition.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. Added.

.github/workflows/ci.yaml Outdated Show resolved Hide resolved
@sanmai
Copy link
Member Author

sanmai commented May 4, 2021

I think I messed up something here. Let me take a day to check everything again.

@sanmai sanmai marked this pull request as draft May 4, 2021 16:17
@acasademont
Copy link

Would love to have this handled automatically, it's not a PITA to add the XDEBUG_MODE=coverage in front of the infection call, but it's a bit annoying having to add that variable for all infection when's something only needed for the intial test runs.

@acasademont
Copy link

And another thing we realized is that everything runs much slower if we set the XDEBUG_MODE=covefage on the infection call.

Previously we used a the --initial-tests-php-options="dxdebug.mode=coverage" but that causes some phpunit tests that need to run in a separate process to not have coverage enabled and fail.

So this patch would probably be the best solution.

XdebugHandler::getSkippedVersion() !== '' ||
// Any other value but false means Xdebug 3 is loaded. Xdebug 2 didn't have
// it too, but it has coverage enabled at all times.
ini_get_unsafe('xdebug.mode') !== false
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't make sense to check for an exact value of xdebug.mode here because it is not affected by XDEBUG_MODE.

devTools/xdebug-coverage.ini Outdated Show resolved Hide resolved
tests/e2e/Skip_Initial_Tests/run_tests.bash Show resolved Hide resolved
$env = array_merge($env ?? [], [
'XDEBUG_MODE' => 'coverage',
]);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure I understand this block.

  1. Why is it needed to pass XDEBUG_MODE=coverage when pcov is loaded, for example?
  2. In the end the user is still in control: they can provide XDEBUG_MODE=coverage on their own. - is it true? If user passes XDEBUG_MODE=debug and Xdebug is loaded (with, let's say, off mode), then this new code will override it by XDEBUG_MODE=coverage, no?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. This was a mistake, now corrected.
  2. I meant that if a user wants to use Xdebug to collect coverage, they can provide XDEBUG_MODE=coverage even if they have PCOV loaded. We can't guarantee other tools will pick this up, but at least we're not standing in the way.
  3. XDEBUG_MODE=debug is no use for us, we have to override it with XDEBUG_MODE=coverage

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, all should be much more clear in shallExtendEnvironmentWithXdebugMode below.

// - Xdebug wasn't ever there
// - or Xdebug isn't version 3+
if (
!extension_loaded('pcov') ||
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be && but not || between all the conditions.

Otherwise, if the current case is phpdbg, the this first condition will result to true and XDEBUG_MODE=coverage will be added

extension_loaded('pcov') ||
PHP_SAPI === 'phpdbg' ||
XdebugHandler::getSkippedVersion() !== '' ||
ini_get_unsafe('xdebug.mode') !== false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as in #1518 (comment), seems like here || also probably needs to be replaced with &&

Copy link
Member Author

@sanmai sanmai Oct 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we need to properly DI test this, without this extension_loaded nonsense.

Copy link
Member Author

@sanmai sanmai Nov 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three weeks later DI stuff isn't here, but it looks like I was able to figure it out: here we assume that
XDEBUG_MODE=coverage is set unless we're running with PCOV, or under PHPDBG, or with Xdebug <3. Checking for XdebugHandler here is a foul business: we never invoke it for tests.

@sanmai sanmai marked this pull request as draft November 4, 2021 00:44
auto-merge was automatically disabled November 4, 2021 00:44

Pull request was converted to draft

@sanmai sanmai removed the request for review from sidz November 4, 2021 00:45
Copy link
Member Author

@sanmai sanmai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤞

Copy link
Member

@maks-rafalko maks-rafalko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏 looks more clear now

@maks-rafalko maks-rafalko marked this pull request as ready for review November 10, 2021 09:25
@sanmai sanmai merged commit 2da5bcc into infection:master Nov 10, 2021
@sanmai sanmai deleted the pr/2020-05/autodetect-XDEBUG_MODE branch November 10, 2021 10:47
@maks-rafalko maks-rafalko added this to the next milestone Nov 10, 2021
@maks-rafalko maks-rafalko modified the milestones: next, 0.26.0 Jan 11, 2022
@alfredbez
Copy link

alfredbez commented Mar 22, 2022

I'm getting an error since this change was merged and released:

Infection - PHP Mutation Testing Framework version 0.25.4.0


Fatal error: Uncaught Error: Call to undefined method Composer\XdebugHandler\XdebugHandler::isXdebugActive() in phar:///usr/local/bin/infection/src/TestFramework/Coverage/CoverageChecker.php:99
Stack trace:
#0 phar:///usr/local/bin/infection/src/TestFramework/Coverage/CoverageChecker.php(47): _HumbugBox75a5b98e45f1\Infection\TestFramework\Coverage\CoverageChecker->hasCoverageGeneratorEnabled()
#1 phar:///usr/local/bin/infection/src/Command/RunCommand.php(140): _HumbugBox75a5b98e45f1\Infection\TestFramework\Coverage\CoverageChecker->checkCoverageRequirements()
#2 phar:///usr/local/bin/infection/src/Command/RunCommand.php(72): _HumbugBox75a5b98e45f1\Infection\Command\RunCommand->startUp(Object(_HumbugBox75a5b98e45f1\Infection\Container), Object(_HumbugBox75a5b98e45f1\Infection\Console\ConsoleOutput), Object(_HumbugBox75a5b98e45f1\Infection\Logger\ConsoleLogger), Object(_HumbugBox75a5b98e45f1\Infection\Console\IO))
#3 phar:///usr/local/bin/infection/src/Command/BaseCommand.php(28): _HumbugBox75a5b98e45f1\Infection\Command\Ru in phar:///usr/local/bin/infection/src/TestFramework/Coverage/CoverageChecker.php on line 99

My Dockerfile looks like this:

FROM php:7.4-alpine

RUN apk --no-cache add pcre-dev ${PHPIZE_DEPS} \
  && pecl install xdebug \
  && docker-php-ext-enable xdebug \
  && apk del pcre-dev ${PHPIZE_DEPS}

RUN wget https://github.com/infection/infection/releases/download/0.25.4/infection.phar \
    && wget https://github.com/infection/infection/releases/download/0.25.4/infection.phar.asc \
    && chmod +x infection.phar \
    && mv infection.phar /usr/local/bin/infection

The same behaviour occurs in all future releases after 0.25.4.


I've extracted the phar and checked how that looks inside:

/tmp/usr/local/bin/infection $ php -r '$v = require_once("vendor/composer/installed.php"); echo $v["versions"]["composer/xdebug-handler"]["version"] . PHP_EOL;'
2.0.0.0
/tmp/usr/local/bin/infection $ grep isXdebugActive vendor/composer/xdebug-handler/src/XdebugHandler.php
    public static function isXdebugActive()
/tmp/usr/local/bin/infection $

Am I doing something wrong?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

For recent Xdebug coverage collection needs XDEBUG_MODE=coverage or xdebug.mode=coverage
5 participants