Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[QT-589] Use the go module cache between CI and build
In order to reliably store Go test times in the Github Actions cache we need to reduce our cache thrashing by not using more than 10gb over all of our caches. This change reduces our cache usage significantly by sharing Go module cache between our Go CI workflows and our build workflows. We lose our per-builder cache which will result in a bit of performance hit, but we'll enable better automatic rebalancing of our CI workflows. Some preliminary investigation into this new strategy: Prior build workflow strategy on a cache miss: Download modules: ~20s Build Vault: ~40s Upload cache: ~30s Total: ~1m30s Prior build workflow strategy on a cache hit: Download and decompress modules and build cache: ~12s Build Vault: ~15s Total: ~28s New build workflow strategy on a cache miss: Download modules: ~20 Build Vault: ~40s Upload cache: ~6s Total: ~1m6s New build workflow strategy on a cache hit: Download and decompress modules: ~3s Build Vault: ~40s Total: ~43s Expected time if we used no Go caching: Download modules: ~20 Build Vault: ~40s Total: ~1m Signed-off-by: Ryan Cragun <me@ryan.ec>
- Loading branch information
1 parent
bfa93fd
commit ffb7fa7
Showing
11 changed files
with
143 additions
and
157 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| --- | ||
| name: Set up Go with a shared module cache | ||
| description: Set up Go with a shared module cache | ||
|
|
||
| inputs: | ||
| no-restore: | ||
| description: "Whether or not to restore the Go module cache on a cache hit" | ||
| type: boolean | ||
| default: false | ||
|
|
||
| outputs: | ||
| cache-key: | ||
| description: "The Go modules cache key" | ||
| value: ${{ steps.metadata.outputs.cache-key }} | ||
| cache-path: | ||
| description: "The GOMODCACHE path" | ||
| value: ${{ steps.metadata.outputs.cache-path }} | ||
| go-version: | ||
| description: "The version of Go in the .go-version file" | ||
| value: ${{ steps.go-version.outputs.go-version }} | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3 | ||
| - id: go-version | ||
| shell: bash | ||
| run: echo "go-version=$(cat ./.go-version)" >> "$GITHUB_OUTPUT" | ||
| - uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 | ||
| with: | ||
| go-version: ${{ steps.go-version.outputs.go-version }} | ||
| cache: false # We use our own caching strategy | ||
| - id: metadata | ||
| shell: bash | ||
| run: | | ||
| echo "cache-path=$(go env GOMODCACHE)" >> "$GITHUB_OUTPUT" | ||
| echo "cache-key=go-modules-${{ hashFiles('**/go.sum') }}" >> "$GITHUB_OUTPUT" | ||
| - id: cache-modules | ||
| uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3.3.1 | ||
| with: | ||
| enableCrossOsArchive: true | ||
| lookup-only: ${{ inputs.no-restore }} | ||
| # We need to be very considerate of our caching strategy because Github only allows 10gb | ||
| # of caches per repository before it starts to evict older caches. This is usually fine | ||
| # if you only use the actions cache for cache, but we also use it for Go test time results. | ||
| # These results are used to balance our Go test groups, without which we could have | ||
| # painfully unbalanced Go test execution times. We have to ensure current caches for all | ||
| # active release branches and main do not exceed 10gb. Ideally we'd cache Go modules | ||
| # and Go build cache on a per version/platform/architecture/tag/module basis, but that | ||
| # would result in several hungred gb over all of our build workflows and release branches. | ||
| # Instead, we've chosen a middle ground approach where were share Go modules between build | ||
| # workflows but lose the Go build cache. | ||
| # We intentionally do not use partial restore keys. If we get dont get an exact cache hit | ||
| # we only want to download the latest modules, not append them to a prior cache. This | ||
| # keeps cache upload time, download time, and storage size to a minimum. | ||
| path: ${{ steps.metadata.outputs.cache-path }} | ||
| key: ${{ steps.metadata.outputs.cache-key }} | ||
| - if: steps.cache-modules.outputs.cache-hit != 'true' | ||
| name: Download go modules | ||
| shell: bash | ||
| run: | | ||
| git config --global url."https://${{ secrets.ELEVATED_GITHUB_TOKEN }}@github.com".insteadOf https://github.com | ||
| # go list ./... forces downloading some additional versions of modules that 'go mod | ||
| # download' misses. We need this because we make use of go list itself during | ||
| # code generation in later builds that rely on this module cache. | ||
| go list ./... | ||
| go list -test ./... | ||
| go mod download | ||
| ( cd sdk && go mod download ) | ||
| ( cd api && go mod download ) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.