Job summary on PR

[samueljsmalmeida]

Could have an option to publish the job summary to the PR please.

[MikeMcC399]

@samueljsmalmeida

Could have an option to publish the job summary to the PR please.

I understand that you are asking for a feature enhancement to the action. Could you explain a little more what you would like and how it would work?

Please also describe why this is needed and what benefit it would give.

[samueljsmalmeida]

Many thanks for the very quick response.

The general idea is to have an option to publish the test summary to the respective PR as a comment.

If the parameter publish-summary-to-pr is set to true, and if the github.event_name == 'pull_request', then add the test summary MD as a comment to the respective PR.

The benefit of said feature is to deliver the option to centralize relevant info on the PR and avoid having to go to the Actions tab.

Many thanks

[MikeMcC399]

@samueljsmalmeida

It's an interesting idea. I guess we have to wait to see what the Cypress team thinks and if anybody is prepared to submit a PR to implement it.

The best alternative at the moment is to view the results in the PR, for example #1059 which I submitted today shows:

however clicking on any of the relevant Details link like https://github.com/cypress-io/github-action/actions/runs/6559572703/job/17815434926?pr=1059 does not show the Job Summary

Only if you click on the Summary over on the left do you then get to the desired overview page

[samueljsmalmeida]

Exactly, my idea is to avoid having to navigate to the Action summary.
Thanks

[albertodeago]

Would love to see this too

[ghiscoding]

I also wish that this action could add the summary into the PR, that is exactly what Codecov is doing and it is super convenient to get all the info directly in the PR (see below).

Another approach could be to use this other GitHub Action EnricoMi/publish-unit-test-result-action that collects any test reports and displays the summary into multiple places (as a PR comment, as a Job Summary, ...) but that requires us to run Cypress with reporters as JUnit or JSON which is then passed to that publish-unit-test-result-action action. I tried to give it a go but at the end it requires so much setup and external dependencies that I gave up (it shows below, as the last summary, but with 0 tests), however I do have it working with Playwright in another project.

At the end of the day, Cypress already have all the info displayed in the Job Summary, it's just a matter of sending it to the PR as well. Note that it might need extra permissions to write to the PR but that's just a setup that user could accept to opt-in or not (e.g. publish-unit-test-result-action requires these permissions checks: write and pull-requests: write). If Codecov can do it (which is always does by default, no config required), I'm sure Cypress can do it too

[krainboltgreene]

Honestly just adding the grid to the output would be a huge boon, then people could do the commenting bit themselves.

[lumattr]

If you could have the job summaries of all the jobs displayed at the bottom of the PR, that could negate the need for having to comment at all. My issue at the moment is that we have seperate workflows that are trying to publish information to the PR but we end up with comment spam because, whilst they each use a comment maintain action, theres still 5 comments for each PR. Job summaries would be good, but with the details link going to the job page rather than summary, i have to guide the devs to the output data.