Run PHPStan programmatically and get analysis results #10966
Replies: 2 comments 11 replies
-
I really like this idea, it'd make mutation testing a lot more useful! As for how to do this, you could certainly run PHPStan programatically inside other PHP application, but you'd lose a lot of benefits. If you simply run PHPStan on the command line as an external program, you get a lot for free and it's certainly easier. You get parallelism out of the box, and also the result cache (which is very important for this). So in order to achieve your goal, I'd add more CLI options to the main We'd introduce two new options:
Yes, you need to analyse the whole project, otherwise you miss out on errors. Fortunately the result cache can help a lot with the performance. As for other considerations, I think you need to make sure that PHPStan passes with 0 errors before trying out temporary changes in the mutated file. I think this can be simply a prerequisite for Infection to work, similarly to the test suite being green before you run Infection. The baseline would certainly make that easy. Also: you need to ask the user how to run PHPStan. So these things would have to be added to Infection config I think:
One more important question: Is Infection interested in actual error messages reported by PHPStan, or not? If not, we could also add an option Let me know if I'm right and all of this would be sufficient to adopt PHPStan in Infection. |
Beta Was this translation helpful? Give feedback.
-
Just commenting to let you know that I'm interested on giving the solution a try, whenever it is ready for early testing. 😊 Thank you for making PHP better everyday! ❤️ |
Beta Was this translation helpful? Give feedback.
-
Hello.
My question/request is quite specific because we need to run PHPStan under the hood of @infection - Mutation Testing library - to check if modified files still pass SA.
Why we need optional Static Analysis for killing Mutants is well explained here: https://github.com/Roave/infection-static-analysis-plugin?tab=readme-ov-file#background - this is a "brutal" implementation of Psalm integration with Infection as a standalone library.
What I want to achieve is basically the same, but with PHPStan, and not as a hacky solution but natively in Infection.
So, keeping Infection's implementation/integration out of this topic, here is what we really need:
$a + b
to$a - $b
insrc/Calculator.php
- and creates a tempory file with this change, e.g./tmp/infection/Calculator.s0m3h4sh.php
and runs tests as a separate process (in parallel with other "Mutants")/tmp/infection/Calculator.s0m3h4sh.php
is still valid / passes PHPStan checks.If new code (
/tmp/infection/Calculator.s0m3h4sh.php
) does not pass PHPStan check, we need to know result error messages (likeParameter $date of method HelloWorld::sayHello() has invalid typehint type DateTimeImutable.
)This is how it's currently implemented in Roave's infection static analysis plugin for Psalm, where Psalm is executed programmatically from the php code:
https://github.com/Roave/infection-static-analysis-plugin/blob/1.36.x/src/Roave/InfectionStaticAnalysis/Psalm/RunStaticAnalysisAgainstMutant.php#L28-L58
Questions that I have before playing with this:
src/Calculator.php
is replaced with/tmp/infection/Calculator.s0m3h4sh.php
? I'm pretty sure that we need to run PHPStan against all the files, because there can be such mutations (changes) that produce error only when modifedCalculator
class is used outside in other classes (like changingpublic
toprivate
for methods)For example, if we do the following mutation:
then PHPStan will not find an issue if we run it against only this file. But it will find an issue if we run PHPStan against all the files in the project.
Beta Was this translation helpful? Give feedback.
All reactions