Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃尡 Update Scorecard API usage #336

Merged
merged 2 commits into from
Jun 8, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
51 changes: 51 additions & 0 deletions Dockerfile.golang
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Copyright 2021 Security Scorecard Authors
azeemshaikh38 marked this conversation as resolved.
Show resolved Hide resolved
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Testing: docker run -e GITHUB_REF=refs/heads/main \
# -e GITHUB_EVENT_NAME=branch_protection_rule \
# -e INPUT_RESULTS_FORMAT=sarif \
# -e INPUT_RESULTS_FILE=results.sarif \
# -e GITHUB_WORKSPACE=/ \
# -e INPUT_POLICY_FILE="/policy.yml" \
# -e INPUT_REPO_TOKEN=$GITHUB_AUTH_TOKEN \
# -e GITHUB_REPOSITORY="ossf/scorecard" \
# laurentsimon/scorecard-action:latest

#v1.17 go
FROM golang@sha256:bd9823cdad5700fb4abe983854488749421d5b4fc84154c30dae474100468b85 AS base
WORKDIR /src
ENV CGO_ENABLED=0
COPY go.* ./
RUN go mod download
COPY . ./

FROM base AS build
ARG TARGETOS
ARG TARGETARCH
RUN CGO_ENABLED=0 make build

# TODO: use distroless:
# FROM gcr.io/distroless/base:nonroot@sha256:02f667185ccf78dbaaf79376b6904aea6d832638e1314387c2c2932f217ac5cb
FROM debian:11.3-slim@sha256:78fd65998de7a59a001d792fe2d3a6d2ea25b6f3f068e5c84881250373577414

RUN apt-get update && \
apt-get install -y --no-install-recommends \
# For debugging.
jq ca-certificates curl
COPY --from=build /src/scorecard-action /

# Copy a test policy for local testing.
COPY policies/template.yml /policy.yml

ENTRYPOINT [ "/scorecard-action" ]
21 changes: 21 additions & 0 deletions cloudbuild.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Copyright 2021 Security Scorecard Authors
azeemshaikh38 marked this conversation as resolved.
Show resolved Hide resolved
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

steps:
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '.',
'-t', 'gcr.io/openssf/scorecard-action:latest',
'-t', 'gcr.io/openssf/scorecard-action:$COMMIT_SHA',
'-f', 'Dockerfile.golang']
images: ['gcr.io/openssf/scorecard-action']
69 changes: 55 additions & 14 deletions e2e/README.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,70 @@
# What

e2e Scorecard action tests for differences in functionality between Scorecard
action implemented in Bash and the updated version implemented using Golang.
These e2e tests will be used until the release of Scorecard Golang action after
which these tests will be modified to run regular e2e testing.

# Setup

For testing functionality difference between the 2 implementations, we need a
setup which can invoke these implementations through a GitHub Action on the same
repo/commitSHA. We achieve this by:

1. The 2 implementations are built using 2 separate Dockerfiles. `./Dockerfile`
for Bash and `./Dockerfile.golang` for Golang.
2. A CloudBuild trigger uses `./cloudbuild.yaml` to continuously build and
generate the Golang Docker image. This also helps reduce run time during the
actual GitHub Action run. The generated Docker image is tagged
`scorecard-action:latest`.
3. Bash implementation at `HEAD` is invoked by referencing: `uses:
ossf/scorecard-action@main` in a GitHub workflow file.
4. The same repository invokes Golang implementation by referencing: `uses:
gcr.io/openssf/scorecard-action:latest`
5. The artifact (SARIF file) produced by these 2 implementations are diff-ed to
verify functional similarity. This step is not yet automated and is largely
manual.

# e2e tests

The `e2e` tests for the action is run by running the action every day on a cron for different use cases. The action that run points to `@main` which helps in catching issues sooner.
The `e2e` tests for the action is run by running the action every day on a cron
for different use cases. The action that run points to `@main` which helps in
catching issues sooner.

If these actions fails to run these actions would create an issue in the repository using https://github.com/naveensrinivasan/Create-GitHub-Issue
If these actions fails to run these actions would create an issue in the
repository using https://github.com/naveensrinivasan/Create-GitHub-Issue

The actions primarily run out of https://github.com/ossf-tests organization.

## Status

| Testcase | Repository | Status. |
| -------- | -------- | -------- |
| Fork | https://github.com/ossf-tests/scorecard-action | [![Fork](https://github.com/ossf-tests/scorecard-action/actions/workflows/scorecards.yml/badge.svg)](https://github.com/ossf-tests/scorecard-action/actions/workflows/scorecards.yml) |
| Non-main-branch | https://github.com/ossf-tests/scorecard-action-non-main-branch | [![non-main-branch](https://github.com/ossf-tests/scorecard-action-non-main-branch/actions/workflows/scorecard-analysis.yml/badge.svg?branch=other)](https://github.com/ossf-tests/scorecard-action-non-main-branch/actions/workflows/scorecard-analysis.yml) |
|Private repository|https://github.com/test-organization-ls/scorecard-action-private-repo-tests| [![Scorecards supply-chain security](https://github.com/test-organization-ls/scorecard-action-private-repo-tests/actions/workflows/scorecard.yml/badge.svg)](https://github.com/test-organization-ls/scorecard-action-private-repo-tests/actions/workflows/scorecard.yml) |

| Fork-golang-staging | https://github.com/ossf-tests/scorecard-action |[![Scorecards supply-chain security](https://github.com/ossf-tests/scorecard-action/actions/workflows/scorecards-golang.yml/badge.svg)](https://github.com/ossf-tests/scorecard-action/actions/workflows/scorecards-golang.yml)
| Non-main-branch-golang-staging | https://github.com/ossf-tests/scorecard-action-non-main-branch | [![Scorecards supply-chain security golang](https://github.com/ossf-tests/scorecard-action-non-main-branch/actions/workflows/scorecard-golang.yml/badge.svg)](https://github.com/ossf-tests/scorecard-action-non-main-branch/actions/workflows/scorecard-golang.yml)
|Private repository-golang-staging|https://github.com/test-organization-ls/scorecard-action-private-repo-tests|[![Scorecards supply-chain security golang](https://github.com/test-organization-ls/scorecard-action-private-repo-tests/actions/workflows/scorecards-golang.yml/badge.svg)](https://github.com/test-organization-ls/scorecard-action-private-repo-tests/actions/workflows/scorecards-golang.yml)
Testcase | Repository | Status.
------------------ | --------------------------------------------------------------------------- | -------
Fork | https://github.com/ossf-tests/scorecard-action | [![Fork](https://github.com/ossf-tests/scorecard-action/actions/workflows/scorecards.yml/badge.svg)](https://github.com/ossf-tests/scorecard-action/actions/workflows/scorecards.yml)
Non-main-branch | https://github.com/ossf-tests/scorecard-action-non-main-branch | [![non-main-branch](https://github.com/ossf-tests/scorecard-action-non-main-branch/actions/workflows/scorecard-analysis.yml/badge.svg?branch=other)](https://github.com/ossf-tests/scorecard-action-non-main-branch/actions/workflows/scorecard-analysis.yml)
Private repository | https://github.com/test-organization-ls/scorecard-action-private-repo-tests | [![Scorecards supply-chain security](https://github.com/test-organization-ls/scorecard-action-private-repo-tests/actions/workflows/scorecard.yml/badge.svg)](https://github.com/test-organization-ls/scorecard-action-private-repo-tests/actions/workflows/scorecard.yml)

| Fork-golang-staging | https://github.com/ossf-tests/scorecard-action
|[![Scorecards supply-chain security](https://github.com/ossf-tests/scorecard-action/actions/workflows/scorecards-golang.yml/badge.svg)](https://github.com/ossf-tests/scorecard-action/actions/workflows/scorecards-golang.yml)
| Non-main-branch-golang-staging |
https://github.com/ossf-tests/scorecard-action-non-main-branch |
[![Scorecards supply-chain security golang](https://github.com/ossf-tests/scorecard-action-non-main-branch/actions/workflows/scorecard-golang.yml/badge.svg)](https://github.com/ossf-tests/scorecard-action-non-main-branch/actions/workflows/scorecard-golang.yml)
|Private
repository-golang-staging|https://github.com/test-organization-ls/scorecard-action-private-repo-tests|[![Scorecards supply-chain security golang](https://github.com/test-organization-ls/scorecard-action-private-repo-tests/actions/workflows/scorecards-golang.yml/badge.svg)](https://github.com/test-organization-ls/scorecard-action-private-repo-tests/actions/workflows/scorecards-golang.yml)

## Diff between golang-staging branch and main

- Here is the sarif results diff between main and golang-staging. There are few text diffs https://github.com/ossf-tests/scorecard-action-results/pull/1/files. The PR is for golang run results. The `main` branch has the `scorecard-action` `main` branch run results.
- Here is the sarif results diff between main and golang-staging. There are
few text diffs
https://github.com/ossf-tests/scorecard-action-results/pull/1/files. The PR
is for golang run results. The `main` branch has the `scorecard-action`
`main` branch run results.

## Steps to add a new test case

1. Create a new repository in the `ossf-tests` organization
2. Clone this workflow https://github.com/ossf-tests/scorecard-action-non-main-branch/blob/other/.github/workflows/scorecard-analysis.yml which has the steps to create an issue if the action fails to run. If the action fails it should create an issue like this https://github.com/ossf/scorecard-action/issues/147
1. Create a new repository in the `ossf-tests` organization
2. Clone this workflow
https://github.com/ossf-tests/scorecard-action-non-main-branch/blob/other/.github/workflows/scorecard-analysis.yml
which has the steps to create an issue if the action fails to run. If the
action fails it should create an issue like this
https://github.com/ossf/scorecard-action/issues/147
3 changes: 2 additions & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@ func main() {
// Processes json results.
repoName := os.Getenv(options.EnvGithubRepository)
repoRef := os.Getenv(options.EnvGithubRef)
if err := signing.ProcessSignature(jsonPayload, repoName, repoRef); err != nil {
accessToken := os.Getenv(options.EnvInputRepoToken)
if err := signing.ProcessSignature(jsonPayload, repoName, repoRef, accessToken); err != nil {
log.Fatalf("error processing signature: %v", err)
}
}
Expand Down
24 changes: 16 additions & 8 deletions signing/signing.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
"io"
"io/ioutil"
"net/http"
"net/url"
"os"
"time"

Expand Down Expand Up @@ -86,12 +87,17 @@ func GetJSONScorecardResults() ([]byte, error) {
}

// ProcessSignature calls scorecard-api to process & upload signed scorecard results.
func ProcessSignature(jsonPayload []byte, repoName, repoRef string) error {
func ProcessSignature(jsonPayload []byte, repoName, repoRef, accessToken string) error {
azeemshaikh38 marked this conversation as resolved.
Show resolved Hide resolved
// Prepare HTTP request body for scorecard-webapp-api call.
// TODO: Use the `ScorecardResult` struct from `scorecard-webapp`.
resultsPayload := struct {
JSONOutput string
Result string `json:"result"`
Branch string `json:"branch"`
AccessToken string `json:"accessToken"`
}{
JSONOutput: string(jsonPayload),
Result: string(jsonPayload),
Branch: repoRef,
AccessToken: accessToken,
}

payloadBytes, err := json.Marshal(resultsPayload)
Expand All @@ -101,13 +107,15 @@ func ProcessSignature(jsonPayload []byte, repoName, repoRef string) error {

// Call scorecard-webapp-api to process and upload signature.
// Setup HTTP request and context.
url := "https://api.securityscorecards.dev/verify"
req, err := http.NewRequest("POST", url, bytes.NewBuffer(payloadBytes)) //nolint
rawURL := fmt.Sprintf("https://api.securityscorecards.dev/projects/github.com/%s", repoName)
parsedURL, err := url.Parse(rawURL)
if err != nil {
return fmt.Errorf("parsing Scorecard API endpoint: %w", err)
}
req, err := http.NewRequest("POST", parsedURL.String(), bytes.NewBuffer(payloadBytes)) //nolint
if err != nil {
return fmt.Errorf("creating HTTP request: %w", err)
}
req.Header.Set("X-Repository", repoName)
req.Header.Set("X-Branch", repoRef)

ctx, cancel := context.WithTimeout(req.Context(), 10*time.Second)
defer cancel()
Expand All @@ -121,7 +129,7 @@ func ProcessSignature(jsonPayload []byte, repoName, repoRef string) error {
}
defer resp.Body.Close()

if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusCreated {
bodyBytes, err := io.ReadAll(resp.Body)
if err != nil {
return fmt.Errorf("reading response body: %w", err)
Expand Down
5 changes: 3 additions & 2 deletions signing/signing_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -77,14 +77,15 @@ func Test_ProcessSignature(t *testing.T) {
t.Parallel()

jsonPayload, err := ioutil.ReadFile("testdata/results.json")
repoName := "rohankh532/scorecard-OIDC-test"
repoName := "ossf-tests/scorecard-action"
repoRef := "refs/heads/main"
accessToken := ""

if err != nil {
t.Errorf("Error reading testdata:, %v", err)
}

if err := ProcessSignature(jsonPayload, repoName, repoRef); err != nil {
if err := ProcessSignature(jsonPayload, repoName, repoRef, accessToken); err != nil {
t.Errorf("ProcessSignature() error:, %v", err)
return
}
Expand Down