Action all the sudden reporting job status using "(merge)" suffix

[mellis481]

I have a number of workflow jobs that use cypress-io/github-action which I have tied to branch protection status checks. Here is an example of a job:

- name: Cypress end-to-end tests
  uses: cypress-io/github-action@v4
  with:
    wait-on: 'https://localhost:9000/index.js'
    start: npm run start:local
    config-file: cypress/config/e2e.ts
    record: true
    install: false
    group: 'E2E'
    ci-build-id: ${{ needs.prepare.outputs.uuid }}
    tag: 'pull-request'

The name of the status check that I previously set for this job is "cypress: E2E" which uses the group property of the action. This has been working fine. All of the sudden, however, the previous status checks are not reporting. Instead, I'm seeing status checks being reported that have my expected checks, but with a "(merge)" suffix. eg:

Why is this happening?

[mellis481]

I manually re-ran the workflow and this time it reported the status checks without the "(merge)" suffix. I still very much want the answer to why this is happening seemingly all of the sudden. And it's, of course, necessary to be able to use a single status check regardless if a workflow was manually re-run or not.

[MikeMcC399]

@mellis481

Was this just a one-time occurrence or are you having repeatable issues? You would need to examine the runner workflow logs for the run which hung in order to see what might have gone wrong. If you find a need to re-run your workflow with debug logging enabled, then you can add:

env:
  DEBUG: '@cypress/github-action, cypress:*'

Your post shows that you are using cypress-io/github-action@v4 which is effectively v4.2.2 published on Nov 28, 2022 and unchanged since then.

To take advantage of on-going improvements and bug-fixes you should consider migrating to cypress-io/github-action@v5 which is the latest version available.

Which version of Cypress are you using?

[mellis481]

v4.

This happened multiple times. I pushed an empty commit to the PR after seeing the problem initially and the corresponding workflow run again had the "(merge)" suffix. It wasn't until I manually re-ran the workflow where the statuses were reported without the "(merge)" suffix.

[MikeMcC399]

@mellis481

Is your screenshot from the pull request status on GitHub? Are you using a GitHub hosted runner or self-hosted? What do you have in your workflow event trigger section on:? I suspect that merge is a GitHub PR status. In open PRs in this repository I see (pull_request) where you see (merge):

v4.

I was actually asking about the version of Cypress in use, like 9.7.0 or 12.6.0 for instance. That is just for background information.

[mellis481]

Is your screenshot from the pull request status on GitHub?

Yes.

Are you using a GitHub hosted runner or self-hosted?

Self-hosted.

What do you have in your workflow event trigger section on:? I suspect that merge is a GitHub PR status. In open PRs in this repository I see (pull_request) where you see (merge):

on:
  pull_request:
    branches:
      - master

I was actually asking about the version of Cypress in use, like 9.7.0 or 12.6.0 for instance. That is just for background information.

10.4.0.

[MikeMcC399]

@mellis481

Thanks for the additional information! It doesn't look like this is an issue caused by cypress-io/github-action. Again, I think you would need to dig into your log files to gain more insight.

[MikeMcC399]

@mellis481

One more thought: when I hover over the GitHub icon in my screenshot it says "@github-actions generated this status".

In your screenshot I can see only Cypress icons, so there is a difference.

Your issue has now been assigned to a Cypress engineer, so hopefully he will be able to chip in with more expertise than I have!

[mellis481]

@MikeMcC399 Thanks! Yes, when I hover over the Cypress icons the popover says "@cypress generated status".

I'm not sure if it was clear from my workflow code, but our workflow job is parallelized and integrated with Cypress Dashboard.

[flotwig]

@mellis481 I think I concur with @MikeMcC399 that this isn't related to the GitHub Action, since these status checks come directly from Cypress Cloud itself.

I spoke with @davidr-cy and he showed me that the relevant code paths that generate these status checks haven't changed in almost 2 years. (merge) is added if it's detected that the commit message is generated from a merge commit, and our logic around that hasn't changed.

So now we are wondering if GitHub is sending us different metadata all of a sudden, causing the change in behavior you're seeing. Could you please share your Cloud organization ID so we can debug further? You can find it under "Organization ID" inside of the Cypress Cloud.

[mellis481]

@flotwig c5365dbf-9c5b-4769-a95d-130d46f70241

[flotwig]

@mellis481 can you try switching to on: push instead of on: pull_request and seeing if it resolves the issue? It's what the docs recommend, for reasons explained there: https://github.com/cypress-io/github-action#record-test-results-on-cypress-cloud

[mellis481]

@flotwig I'm confused with the recommendation in the article you shared. Those triggers are used in different use cases. on: pull_request should be used for PR gate workflows. on: push should be used for workflows that should be run when a PR is merged.

[mjhenkes]

on push, happens whenever you push changes to a branch, be it the feature branch, master or main

on pull_request happens when 'the workflow runs when a pull request is opened or reopened or when the head branch of the pull request is updated'.

So after you've created a pull request, on push and on pull_request will both happen whenever you update the head branch.

According to the article the information provided by on push is more accurate than on pull_request but I haven't double checked that myself.

It's worth a shot to try though right? 🤷‍♂

[mellis481]

@mjhenkes The example workflow in the article you shared doesn't specify a branch so it will run the Cypress action on any pushed commit to any branch. That's basically the only scenario I can think of where the push and pull_request triggers could be used interchangeably.

In my situation (which is exceedingly common I can only assume), I only want to run Cypress tests for 1. PRs 2. against certain branches (in my case, just one: master). This means I have to use the pull_request trigger which will run when there is "activity on a pull request in the workflow's repository occurs" (on the master branch). If I were to use the push trigger like you recommend, the only way it would run would be is if I changed the branch targeted by the trigger from master to the branch the PR is for. And then for the next PR, the workflow would need to target that PR branch name. That, of course, isn't how PR gate workflows would ever work.

[mjhenkes]

@mellis481, you do have a common use case, I agree, but as an experiment could you try using the push request and let us know if the jobs still have the 'merge suffix'? That will either narrow down the problem or eliminate the event as the cause of the issue.

[MikeMcC399]

• It looks like this issue is also related to a previous conversation in Inconsistent cypress runs between github action and cypress dashboard #124 involving @mellis481

[mellis481]

@MikeMcC399 I didn't include it in my original post listing code, but I do include COMMIT_MESSAGE which appears to be what solved the issue in that thread.

Here is my whole Cypress step FWIW:

- id: cypress-e2e-tests
  name: Cypress end-to-end tests
  uses: cypress-io/github-action@v4
  with:
    wait-on: 'https://localhost:9000/index.js'
    start: npm run start:local
    config-file: cypress/config/e2e.ts
    record: true
    install: false
    group: 'E2E'
    ci-build-id: ${{ needs.prepare.outputs.uuid }}
    tag: 'pull-request'
  env:
    NODE_TLS_REJECT_UNAUTHORIZED: 0
    CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title }}
    COMMIT_INFO_SHA: ${{ github.event.pull_request.head.sha }}

[MikeMcC399]

@mellis481

This issue seems to have gone stale and I'm unsure what the next steps might be.

A new version of the action has been published which automatically pulls in any available PR information when recording. See https://github.com/cypress-io/github-action#automatic-pr-number--url-detection

You would access this functionality using
cypress-io/github-action@v6 or
cypress-io/github-action@v6.4.0

Again, I'm unsure whether that would change your experience, but I thought I would let you know anyway!

[mellis481]

@MikeMcC399 I'm still seeing this issue using cypress-io/github-action@v6.

[MikeMcC399]

@mellis481

Many thanks for your update to say that you are still experiencing this issue.

It looks like it is not directly an issue with cypress-io/github-action but more with the Cypress Cloud GitHub Integration as described on Cypress Cloud > Integrations > Source Control > GitHub.

We'd need some assistance from the Cypress Cloud team on this, since this part of the solution is not open source. We can't see what it is doing or fix it in this repo.

@emilyrohrbough
This issue is assigned to you. Are you still involved?

[mellis481]

@MikeMcC399 Interestingly, when you just said you think this is a Cypress Cloud thing, I thought I'd compare the project settings for the repo I'm having a problem with with another that I am not. The only thing that I found that was different was that the problem repo did not have a "Default branch" set in Cypress Cloud. I created a PR after setting the Default branch in Cypress Cloud (to master) and it appears to have resolved this issue.

I'm finding the symptoms of a missing "Default branch" in Cypress Cloud strange. As I described in the thread, it appended "(merge)" to the status checks, but what I also found is that, if a GitHub user other than the person who created the PR re-ran the workflow, the status checks reported fine.

In any event, I'll close this issue.

[MikeMcC399]

@mellis481

Congratulations on solving the issue! Thanks also for sharing what you did.

[mellis481]

@MikeMcC399 Nevermind the issue is back again. :(

[MikeMcC399]

@mellis481

Nevermind the issue is back again.

That is rather disappointing 🙁 !

My previous comments still hold though, that this is unlikely to be an issue with github-action and so the Cypress Cloud team would need to assist.

If you don't get any reaction here and you are on a paid Cypress Cloud plan (Team, Business, or Enterprise), then presumably you could also open a Cypress Cloud support ticket.

[mellis481]

(merge) is added if it's detected that the commit message is generated from a merge commit, and our logic around that hasn't changed.

@MikeMcC399 Thanks. I opened a Cypress Cloud support ticket. As you can see, I'm seeing COMMIT_INFO_MESSAGE: ${{ github.event.pull_request.title }}. Does anything about that seem problematic to you?

@flotwig Can you describe the logic that is used to append (merge) to status checks?