New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A way to baseline / "grandfather" existing code in #691
Comments
I like this feature in tools like psalm/phpstan etc. However, i don't think it is a good fit for infection. For static analysis tools, its all or nothing, the checks either pass or don't. And if 1 check doesn't pass, then the process 'fails'. Infection doesn't require everything to pass. In fact, under default circumstances, literally every mutation could fail, and infeciton would still 'pass'. We already have the ability to set a 'baseline' so to speak, by using |
My worry with this is that there are legit cases where the min-msi could be reduced without adding bad code or bad tests, just by virtue of the code being susceptible to mutations that dont change its logic as outlined here: https://medium.com/@maks_rafalko/infection-mutation-testing-framework-c9ccf02eefd1#ecda |
Related: #669 (comment) |
Apologies in advance if advertising other packages is verboten. There is a base-lining tool available from here: https://github.com/DaveLiddament/sarb It doesn't currently support base-lining for Infection, but according to the maintainer, it should be possible to add that. The reason I'm mentioning it is that it seems to me that the rules surrounding a 'base-lining'/'grandfathering' existing code is something that's going to need to work across all of the tools that analyze code for quality. Adding this piecemeal to each code quality project sounds like it could produce a huge duplication of effort. Adding it to a specific baselining tool could avoid a large amount of work. |
@BackEndTea my 2 cents on this topic: I found way easier and quicker to set minimum score to 100% and fill Because mathematically, below 100% you are automatically allowing (i.e. go unnoticed) new escaped mutant on new line-of-codes proportionally to the missing percentage. |
Hi, with the landing of |
Agree, I have the same thoughts: do we really need baselining when we have |
Is your feature request related to a problem? Please describe.
When trying to include infection into an existing project, the amount of issues might be too overwhelming to deal with right from the get go. Makes it hard/impossible to still allow sane min values be set for the ci pipeline to pass.
Describe the solution you'd like
it would be beneficial to add a baseline feature similar to what psalm offers, that ignores issues from old code and only takes new code into account. Instead of adding all files to an ignorelist or adding a feature that only considers files starting at a certain timestamp, a generated ignorelist that references concrete code parts/mutations could be of great help.
Describe alternatives you've considered
Alternatives that come to mind that would work with the current featureset:
Additional context
See psalm documentation for the related feature: https://psalm.dev/docs/dealing_with_code_issues/#using-a-baseline-file
The text was updated successfully, but these errors were encountered: