githubv4.Query: Resource not accessible by integration

[fmarier]

I'm using the default GitHub Action workflow, though with tags instead of commit hashes in https://github.com/brave/simplepadding/blob/master/.github/workflows/scorecards.yml and I'm seeing this error now that I've upgraded to v2:

Error: one or more checks had a runtime error: internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration

Is there a permission missing from the example workflow?

I see it's also been reported in ossf/scorecard#1097, but I'm not sure I'm seeing the same issue since it was working fine before I upgraded from v1.1.2 to v2.

[azeemshaikh38]

Thanks for the report. I can reproduce this on my end. Seems like switching on specific branch-protection settings (like Enable signed commits) can make the action throw up this error. I haven't tested exhaustively to see what all settings lead to this error, but for now I'll try to send out a fix to ensure Scorecard exists gracefully in such cases.

[fmarier]

Here are the branch protection settings I have enabled in the above repo:

[nijel]

It happens for https://github.com/WeblateOrg/weblate/ as well, the only branch protection we have enabled is "Require status checks to pass before merging". Failing workflow run: https://github.com/WeblateOrg/weblate/runs/8262284657

PS: It also started to happen with upgrade to version 2.

[dirien]

Not adding additional noise, but happens in https://github.com/dirien/minectl-sdk too.

https://github.com/dirien/minectl-sdk/runs/8267086312?check_suite_focus=true

Only change is the update to 2.0.0.

[dirien]

I compared a the output from cosign (Ping @cpanato) and my and see following difference by same pipline file:

cosign:

Repository: sigstore/cosign
Fork repository: false
Private repository: false
Publication enabled: false
Format: json
Policy file: 
Default branch: main
Using payload from: results.json

the failed one:

Private repository: false
Publication enabled: false
Format: sarif
Policy file: /policy.yml
Default branch: main
Error: one or more checks had a runtime error: internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
2022/09/09 09:35:31 error during command execution: one or more checks had a runtime error: internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration

Why is there a /policy.yaml?

[azeemshaikh38]

Found the issue - seems to have started after we upgraded to scorecard from v4.5.0 -> v4.6.0. Confirmed that #894 fixes this. Will cut a release for v2.0.1 with this PR to fix the breakages while I investigate the real issue.

[varunsh-coder]

@azeemshaikh38 I think the root cause is due to this line
https://github.com/ossf/scorecard/blob/8de962e91d860f81916dbeb3b9a1a943fcf8a80f/cmd/root.go#L163

When Scorecard is run with the GITHUB_TOKEN, some checks (e.g. for branch protection) are expected to error out. Due to the above change, that causes the overall run to fail.

You can search for this string one or more checks had a runtime error in the output for the Action runs that failed.
e.g. https://github.com/WeblateOrg/weblate/runs/8262284657#step:4:168

[azeemshaikh38]

fyi - https://github.com/ossf/scorecard-action/releases/tag/v2.0.1

@varunsh-coder - good point, I think that's it. Will look into it.

[laurentsimon]

strange that the policy file is not set for the cosign repo. This should be set, otherwise we will be reporting results in the scanning dashboard that we should not.

[dirien]

strange that the policy file is not set for the cosign repo. This should be set, otherwise we will be reporting results in the scanning dashboard that we should not.

Works for me now! https://github.com/dirien/minectl-sdk/runs/8274045473?check_suite_focus=true

@laurentsimon now I have the policy.yaml also not anymore like cosign!

Repository: dirien/minectl-sdk
Fork repository: false
Private repository: false
Publication enabled: false
Format: json
Policy file: 
Default branch: main
Using payload from: results.json
Generating ephemeral keys...

[spencerschrock]

@azeemshaikh38 I think the root cause is due to this line https://github.com/ossf/scorecard/blob/8de962e91d860f81916dbeb3b9a1a943fcf8a80f/cmd/root.go#L163

When Scorecard is run with the GITHUB_TOKEN, some checks (e.g. for branch protection) are expected to error out. Due to the above change, that causes the overall run to fail.

You can search for this string one or more checks had a runtime error in the output for the Action runs that failed. e.g. https://github.com/WeblateOrg/weblate/runs/8262284657#step:4:168

Yes I made the change as part of ossf/scorecard#2124, unaware it would spill over to scorecard-action. Fixing it would probably involve using a specific error here that we check for using errors.Is here and here (or better yet, doing something in the entrypoint to wrap it for all uses)

[laurentsimon]

Sounds like a plan. Let's also add some e2e tests to catch these moving forward.

[spencerschrock]

Not fixed quite yet.
Still need 1 PR on the scorecard-action side of things after a new scorecard release is cut.

[konstruktoid]

This is still an issue when creating an action using "Code scanning alerts" within the GitHub web interface, e.g https://github.com/konstruktoid/hardening/blob/6149f73c708347911362d147cd099d2382b2bc43/.github/workflows/scorecards.yml#L33.
It is however caught by renovate, https://github.com/konstruktoid/ansible-role-hardening/blob/master/.github/workflows/scorecards.yml#L33

[azeemshaikh38]

This is still an issue when creating an action using "Code scanning alerts" within the GitHub web interface, e.g https://github.com/konstruktoid/hardening/blob/6149f73c708347911362d147cd099d2382b2bc43/.github/workflows/scorecards.yml#L33. It is however caught by renovate, https://github.com/konstruktoid/ansible-role-hardening/blob/master/.github/workflows/scorecards.yml#L33

actions/starter-workflows#1746 should fix this.

[konstruktoid]

Yeah, looks good @azeemshaikh38

[varunsh-coder]

Not fixed quite yet.
Still need 1 PR on the scorecard-action side of things after a new scorecard release is cut.

I am curious when is the next release planned for scorecard? So that the scorecard-action can be updated. Thanks!

[konstruktoid]

@varunsh-coder, ossf/scorecard-action@865b4092859256271290c77adbd10a43f4779972 # tag=v2.0.3 does work, see e.g https://github.com/konstruktoid/hardening/blob/master/.github/workflows/scorecards.yml

[spencerschrock]

Yes. v2.0.3 should work. And now that #948 is merged, future releases will use a different fix.
Please reopen if you have any issues.

[varunsh-coder]

I tried using the latest commit a346ade7f481e0463415114a4240c27ae25e3e7d. But it used the older scorecard-action docker image gcr.io/openssf/scorecard-action:v2.0.3.
Here is the workflow: https://github.com/step-security/secure-workflows/actions/runs/3132110410/jobs/5084123121#step:2:22

@spencerschrock did you also need to update the action.yml with the latest docker image? Or is that only done as part of the scorecard-action release? I am also curious which workflow publishes this scorecard-action docker image. Thanks!

[spencerschrock]

@spencerschrock did you also need to update the action.yml with the latest docker image? Or is that only done as part of the scorecard-action release?

Yes, there should be a proper v2.0.4 tag soon. There's a bit of a chicken-and-egg problem in the release procedure, you can read more here

I am also curious which workflow publishes this scorecard-action docker image. Thanks!

It's done as part of the cloudbuild workflows. There are two, but the tagged releases are handled by https://github.com/ossf/scorecard-action/blob/main/cloudbuild-tag.yaml

[LeviPesin]

I still get this same error on my organization's private repository:

Event / repo information:
Event file: /github/workflow/event.json
Event name: push
Fork repository: false
Private repository: true
Publication enabled: false
Default branch: dev
Error: RunScorecard: internal error: ListCommits:error during graphqlHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
2023/05/17 14:05:46 error during command execution: RunScorecard: internal error: ListCommits:error during graphqlHandler.setup: internal error: githubv4.Query: Resource not accessible by integration

Here are "GITHUB_TOKEN Permissions":

  Actions: read
  Contents: read
  Metadata: read

What should I do?

[spencerschrock]

What should I do?

Here is the query that's failing which is a good starting point for what permissions the Scorecard Action may need.
https://github.com/ossf/scorecard/blob/157a509b23bd152635acd2843b90df590c22289a/clients/githubrepo/graphql.go#L40-L131

Skimming through this and the permissions list, I would expect you may also need to add:

• pull-requests
• issues

Although I suspect there will be more read permissions you'll need to grant.

[konstruktoid]

@LeviPesin https://github.com/konstruktoid/hardening/blob/master/.github/workflows/scorecards.yml might help

[LeviPesin]

@spencerschrock This works, thank you!