Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(ci): Review the mutation testing workflow #1791

Merged
merged 9 commits into from Dec 22, 2022

Conversation

theofidry
Copy link
Member

@theofidry theofidry commented Dec 21, 2022

My understanding of that build was, from what I understand in #1365, to have a dedicated pipeline mimicking what a typical user pipeline may look like and testing our mutation badge feature. Over time this was slightly tweaked:

I think the build is still useful, but we now have a more comprehensive and diversified test suite to test the different OS, dependencies, code coverage reports and co in ci.yaml. Hence this build can be more scoped and from that simplified.

  • Document the above in the build
  • Bump the PHP version to 8.2
  • Use the locked version of the dependencies
  • Use the latest Composer version available

Tried to switch to Xdebug. I was curious if there is a big time difference but in general I also know Xdebug is more accurate than pcov, so we may prefer this to enforce a specific MSI score.

Comparing with this build, we had:

pcov Xdebug delta
time taken (unit tests) 1m7 1m34 +40%
time taken (infection) 8m7 13m41 +70%
MSI 73% 75%

At least from those results it doesn't look worth it.

⚠️ Requires to update the project settings afterwards as the build name changes

@theofidry theofidry changed the title Review the mutation testing workflow refactor(ci): Review the mutation testing workflow Dec 21, 2022
@theofidry theofidry marked this pull request as draft December 21, 2022 22:52
@theofidry theofidry marked this pull request as ready for review December 21, 2022 23:14
.github/workflows/mt.yaml Outdated Show resolved Hide resolved

- name: Collect coverage report
run: |
php vendor/phpunit/phpunit/phpunit --stop-on-failure \
SYMFONY_DEPRECATIONS_HELPER=disabled php vendor/phpunit/phpunit/phpunit --stop-on-failure \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is it needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because depending on the deprecations it can fail this build. It's not necessarily bad, but we already check for the deprecations in ci.yaml. So having it here is redundant, and disabling it guaranteeing we get to the interesting part of that build which is running infection with coverage generated and with the badge API key

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lemme know if that's a satisfying answer for you or you prefer to keep it

@theofidry theofidry merged commit a222e75 into infection:master Dec 22, 2022
@theofidry theofidry deleted the feature/mutation-testing-build branch December 22, 2022 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants