New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add github logger to be able to use Annotations on GitHub Actions #1368
Add github logger to be able to use Annotations on GitHub Actions #1368
Conversation
Just a quick note that there’s hardly any reason why Infection can’t output GitHub compatible annotations straight away. |
I mean we can skip this XML and cs2pr altogether by outputting the annotations right from Infection. This will also solve the problem I’ll mention in the review comments in a moment. |
infection.json.dist
Outdated
@@ -9,7 +9,8 @@ | |||
"text": "infection.log", | |||
"badge": { | |||
"branch": "master" | |||
} | |||
}, | |||
"checkstyle": "build/checkstyle.xml" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just like it so it could be:
"github": "/dev/stdout" // or /dev/stderr, whichever works best
And the it could output the annotations directly:
https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-commands-for-github-actions#setting-a-warning-message
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you look in the source of cs2pr, these annotations are pretty damn simple:
It is not to say we should scrap this new checkstyle report, but it won't exactly work here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it can be even better with "github" progress reporter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it can be even better with "github" progress reporter.
what is it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Like ProgressFormatter or DotFormatter. Set with --formatter
option.
If I understood correctly, PHPStan enables this reporter automatically (?) on GitHub CI. Their implementation: https://github.com/phpstan/phpstan-src/blob/c99b7f8c09fe6e319624ef3b0b66f43d30056fe4/src/Command/ErrorFormatter/GithubErrorFormatter.php |
seems so, but I'm not sure it's a good choice TBH for Infection. Without proper filtering (as we do it right now), this will overload the diff with thousands of warnings by default (which MT can produce with even not very big projects) |
If Infection can detect it is running on GitHub (provided it was asked to do that), then it can fetch and run against changed files just as well. |
For example, specifying I agree it's not the best idea to enable this behavior by default. |
my opinion:
So, the current state of PR IMO is ideal - we give a developer new logger that can display GH Annotations to a file / What I think we should add (but in a separate PR), is an ability to set logger by command line option, like Why? To be able to not use checkstyle report for all the Infection runs in all currently available GH Actions, but to specify it only for |
That's what I was thinking when I proposed |
Infection is already very opinionated in a similar way. For example, by choosing to add a whitelist (or what's it called now) to Imagine you're writing a blog post. What would be better to write, that Infection will now automatically mutate only changed files if run with (I'd take it if you say that Infection shouldn't add |
I'm ok with a couple of lines to be added to CI config. This tool is for developers |
If there's |
Sounds good. I will give it a try, thanks! |
464def3
to
9d38e0f
Compare
…s and Errors) on GitHub Actions Try adding markdown syntax to highlight diffs in GitHub annotations Get infection flags Revert changes in ci.yml Remove commented out code Move functions into workflow folder Use relative URL from the root of the repository Add more outputs, test branch names Use HEAD_REF for branch name Test commands (git diff) Test commands (git diff) 2 Test commands (git diff) 3 Test commands (git diff) 4 Test commands (git diff) 5 Move Infection Code Review into its own github action Normilize trailing spaces for diff in tests Fix autoreview tests / cs Move github funcions out of workflow folder, create checkstyle report in build folder
…to `github` Automatically mutate only changed/added files when `--logger-github` is used
… to `--logger-github`
…ix escaped mutant
9b066a3
to
fb507b0
Compare
Thank you all for the great review and ideas! |
There's a Symfony's special reporting component now: |
{ | ||
return (string) shell_exec( | ||
sprintf( | ||
'git diff %s --diff-filter=%s --name-only | grep src/ | paste -sd ","', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hardcoded src/
is likely to be wrong, it should be picked by config.
PR coming soon
This PR:
--git-diff-filter
,--git-diff-base
,--logger-github
tions site#183Fixes #1322
Description
This feature allows adding errors/warnings for particular commits/PRs - GitHub Annotations
Basically, this feature adds escaped mutants information (diff) to particular lines where modified source code wasn't detected by tests:
Things to note
<details>
tag to be able to show/hide huge text.Goal
This will greatly improve review process as it will be immediately clear what parts of the code are not covered by tests properly (thus escaped mutants)
Suggestions are appreciated.