Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

For Mutant's phpunit.xml, set executionOrder="default" to prevent random ordering of the tests since we need them to be sorted (fastest - first) #1547

Merged
merged 1 commit into from
Aug 7, 2021
Merged

For Mutant's phpunit.xml, set executionOrder="default" to prevent random ordering of the tests since we need them to be sorted (fastest - first) #1547

merged 1 commit into from
Aug 7, 2021

Commits on Aug 7, 2021

  1. For Mutant's phpunit.xml, set executionOrder="default" to prevent r… 

    …andom ordering of the tests since we need them to be sorted (fastest - first)

When Infection build Mutant's `phpunit.xml` file, we add `<file>` tags, sorted by tests execution time, so that the fastest tests runs first.

But when the original `phpunit.xml` contains `executionOrder="default"`, it was inherited to Mutant's `phpunit.xml` and broke the order, running the tests in a random order each time `infection` was executed.

This also explains why I got different execution time of Infection for the project with *functional* tests.

When Mutant is covered by 1000 functional tests, and tests are executed by random, it can be killed by the first test in run #1 and can be killed by 1000-th test in run #2, if the killing test is executed the last.

With this update, all the tests will be executed in the same orders (fastest - first) in all the infection runs.
    @maks-rafalko
    maks-rafalko committed Aug 7, 2021
    Configuration menu
    Copy the full SHA
    9fca23b View commit details
    Browse the repository at this point in the history