PR created but action not started

[unional]

You can see that the PR is opened correctly: clibuilder/clibuilder#375
But there is no workflow started: https://github.com/unional/clibuilder/actions

I'm setting it up using shared workflow:

jobs:
  code:
    uses: unional/.github/.github/workflows/yarn2-library-verify-cross-platform.yml@main

  release:
    uses: unional/.github/.github/workflows/yarn2-library-release.yml@main
    needs: code
    secrets: inherit

https://github.com/unional/clibuilder/blob/main/.github/workflows/release.yml

      - name: Create Release Pull Request or Publish to npm
        id: changesets
        uses: changesets/action@v1
        with:
          version: yarn changeset version
          publish: yarn changeset publish
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

https://github.com/unional/.github/blob/main/.github/workflows/yarn2-library-release.yml

Maybe somehow the action is getting wrong information because of that?

[unional]

I have another repo with the same problem:
stable-packages/store#118

[abdonrd]

Same here! Qiskit/web-components#83

[Saul-Mirone]

Same here! Milkdown/milkdown#531

[yishayweb]

Hey guys
I'm not sure that what I'm experiencing is the same issue, but it seems related
For me, when the 'Version Packages' pr is created, the workflow I have do run
But when I merge other changes to master (with a changeset file), and the 'Version Packages' pr is being updated, the workflow doesn't run anymore.
This is the workflow I'm refering to:
yishayweb/monorepo_pnpm_changesets
and as you can see, I'm triggering it with
on: pull_request: types: [opened, synchronize, reopened, edited] branches: - master
Hence I expect it to be run each time the 'Version Packages' pr is getting updated
Is it the issue you guys experience?

[sindrisig]

same issue in my private repo

[airtonix]

@yishayweb you can turf all that custom code to test if changesets exist and use either:

      - uses: dorny/paths-filter@v2
        id: filter
        with:
          base: ${{ github.ref }}
          filters: |
            Changesets:
              - added|modified: '.changesets/!(README)*.md'

            Apps:
              - 'apps/!(*storybook)**/*'

      - run: node -e 'console.log(${{toJson(steps.filter.outputs)}})'

or you can just use the changesets/cli status command with the json output:

https://github.com/changesets/changesets/tree/main/packages/cli#status

[kachkaev]

From GitHub docs:

When you use the repository’s GITHUB_TOKEN to perform tasks, events triggered by the GITHUB_TOKEN will not create a new workflow run. This prevents you from accidentally creating recursive workflow runs. For example, if a workflow run pushes code using the repository’s GITHUB_TOKEN, a new workflow will not run even when the repository contains a workflow configured to run when push events occur.

Use a repo scoped Personal Access Token (PAT) created on an account that has write access to the repository that pull requests are being created in. This is the standard workaround and recommended by GitHub. However, the PAT cannot be scoped to a specific repository so the token becomes a very sensitive secret. If this is a concern, the PAT can instead be created for a dedicated machine account that has collaborator access to the repository. Also note that because the account that owns the PAT will be the creator of pull requests, that user account will be unable to perform actions such as request changes or approve the pull request.

So the solution could be this:

       env:
-         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+         GITHUB_TOKEN: ${{ secrets.CUSTOM_GITHUB_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

CUSTOM_GITHUB_TOKEN needs to be generated and added to repo secrets. Note that the name of a secret cannot start with GITHUB_.

[Andarist]

Yes, @kachkaev's answer is the correct one. Would you be so kind to add a note about this in the README of this action?

[kachkaev]

I’m still a bit confused by these tokens actually. After setting a custom GitHub token in blockprotocol/blockprotocol#510, I saw a new PR authored by the corresponding user and CI ran fine. However, after mere PRs got merged into main, the updates in ‘Version Packages’ PR still came from github-actions[bot] and so CI did not re-run.

First Commit Actions (full set):

Actions for one of subsequent force-pushes (no checks for on: pull_request):

Merging is blocked:

Seems like the custom GITHUB_TOKEN env is not used when creating commits 🤔 Or maybe it is used, but this is trumped by commit metadata Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>.

Any tips anyone?

[kachkaev]

I tried configuring custom git credentials in https://github.com/blockprotocol/blockprotocol/pull/535/files

        env:
+         GIT_AUTHOR_EMAIL: custom-value@example.com
+         GIT_AUTHOR_NAME: Custom Value
+         GIT_COMMITTER_EMAIL: custom-value@example.com
+         GIT_COMMITTER_NAME: Custom Value
          GITHUB_TOKEN: ${{ secrets.CUSTOM_GITHUB_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

However, this did not help. Here is what I see in the ‘Version Packages’ PR:

Commit metadata matches new environment variables that I’ve set:

commit 7fda46e1286e2b815b5f94b76e8532ce491edcd5 (HEAD, origin/changeset-release/main)
Author: hashdotai <63675132+hashdotai@users.noreply.github.com>
Date:   Thu Aug 25 12:53:17 2022 +0000

    Version Packages

The push though is still done by github-actions[bot] so checks are not triggered. 🤔 Because Linting is a required check, the PR cannot be merged unless a developer force-pushes to the branch just before releasing new versions.

[Andarist]

This is surprising to me and, unfortunately, I don't know the answer to this problem. Perhaps it would be worth opening a ticket with GitHub support to learn what are the exact requirements for workflows to be executed. However, if I take a look at this commit and the related PR then I can see that workflows were correctly executed for both. So perhaps this is an issue in your setup. As far as I can see they only provide a custom token and nothing more:
https://github.com/developit/microbundle/blob/60b5edcf5d96c26877134febbdf0a247ee56b9c7/.github/workflows/release.yml#L34

[kachkaev]

Thanks for sharing a link to developit/microbundle#958 @Andarist!

If we look at individual commits in this PR, we can see the same pattern as in blockprotocol/blockprotocol#529. The first commit has ✅, but all the following force-pushed ones lack CI checks (1, 2, 3, ...). All commits are authored by github-actions[bot] despite GITHUB_TOKEN: ${{ secrets.REPO_SCOPED_TOKEN }} in release.yml.

The PR is then closed and reopened, which seems to work as a trigger for GitHub Actions. CI runs for the latest github-actions[bot]’s commit but is marked as triggered by a human.

I tried closing and re-opening blockprotocol/blockprotocol#529 too and this did trigger GitHub Actions. That’s a bit hacky but I don’t see any alternatives at this point. Unless I miss something, redefining GITHUB_TOKEN only changes the author of the PR and triggers actions for the first commit in it.

[Andarist]

This is bizarre... 🙈 it would really be lovely if we could create a GitHub support ticket about this. Would you be willing to do this? You already have all the required context to explain the issue properly

[kachkaev]

I think I figured it out. Here is my final solution:

 jobs:
   release:
     runs-on: ubuntu-latest
     if: github.repository == 'org/repo'
     steps:
       - uses: actions/checkout@v3
+        with:
+          token: ${{ secrets.CUSTOM_GITHUB_TOKEN }}

       - uses: ./.github/actions/warm-up-repo
 
       - name: Create Release Pull Request or Publish to npm
         uses: changesets/action@v1.3.0
         with:
           publish: yarn changeset publish
         env:
+          GITHUB_TOKEN: ${{ secrets.CUSTOM_GITHUB_TOKEN }}
           NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

release.yml

When the action calls git push, it uses the same token as the one that was used during checkout. This explains why setting GITHUB_TOKEN only in changesets/action does not change the account that has force-pushed.

Resulting PR: blockprotocol/blockprotocol#542

CI runs for each new commit:

• ✅ blockprotocol/blockprotocol@5d912de
• ✅ blockprotocol/blockprotocol@ae9b69f
• ...

Now that I know that closing and reopening a PR replaces GITHUB_TOKEN, I wonder if it’s actually better. 😅 If every new push to ‘Version Packages’ triggers all GitHub workflows, that’s quite a few CI minutes wasted 😬

[Andarist]

If every new push to ‘Version Packages’ triggers all GitHub workflows, that’s quite a few CI minutes wasted 😬

Yeah, in theory, you could craft a more complex pipeline that would wait for the CI checks on your base branch before updating the versioning PR and skip jobs there because they would be somewhat redundant if you have already run jobs on the base branch. It would be great to block the ability to merge the versioning PR when there is a pending CI job on the base branch too.

Would you be willing to add all of this information to the README, with the tradeoffs mentioned there?

[kachkaev]

Would you be willing to add all of this information to the README, with the tradeoffs mentioned there?

I can try next week, but feel free to do this earlier if you want! A change in README should close this issue I guess.

[kachkaev]

Yeah, in theory, you could craft a more complex pipeline that would wait for the CI checks

Possibly, yes. Here is another idea, which may work together with yours. Before force-pushing, the action can:

1. Create local commit

2. Checkout latest commit from changeset-release/main

3. If commit diffs are equal, don't force-push at all

‘Version Packages’ will be out of date with main, but that's OK as it is always possible to press the Update button. Importantly, CI checks in this PR will only run when someone creates a new changeset/*.md file rather than on every PR merge to main. This would help a lot in our monorepo, because we rarely change packages, but make a lot of tweaks to a web app. Only a fraction of PRs affect the diff in ‘Version Packages’.

[BPScott]

Github just shipped https://github.blog/changelog/2022-09-08-github-actions-use-github_token-with-workflow_dispatch-and-repository_dispatch/

Perhaps there could be a mechanism where users specify a set of actions that need to be triggered on the Version Packages PR using workflow_dispatch after it is created/updated. It seems like that'd allow CI runs to occur without needing custom tokens.

Something like:

• user adds workflow_dispatch as an allowed trigger on an action (e.g. my-ci-action.yml) that runs CI checks
• user adds a with: {trigger_jobs_on_pr: ['my-ci-action.yml']} to the with object of the changsets action
• Changesets triggers those jobs on the created PR.

[Andarist]

If you could prototype this - it would be kinda cool.

[shawnthye-guru]

Another option is to allow the setting to create Draft PR(there is already a PR, but I'm not sure why it doesn't merge, maybe the person who created this PR is retired), so that when an actual person made [Ready for Review], then workflow will run