Strategies for 'large' builds on GitHub actions runners with limited disk space

[sebhoss]

Hi folks,

I'm currently running into a 'no space left on device' error while trying to release one of my projects (latest run). The final release artifacts are around 50MB and I build for 6 different platforms. Since GitHub runners have 14GB of disk space, this should fit nicely, however this disk space is used for tooling as well as some compile artifacts produces by the Go compiler which eats up all the available space. My guess is that the compile artifacts take up the majority of disk space since the project is huge (millions of lines of code).

Is there any way to call something like --clean after each compiled binary, so that I have a higher chance of succeeding with a build?

Thanks!

[caarlos0]

what would you clean after each build?

also, 6*50mb would be 300mb... far away from 14gb... are you sure there isn't something else going on?

that said, if the problem is temp files, maybe use a reduced --parallelism would help 🤔

[caarlos0]

I would also try goreleaser release --clean --snapshot locally

running it here, and watch -n1 'du -hs /tmp/go-build*' at the same time shows interesting results:

• go-build folders >1.5GB
• final binary on ./dist >700MB

[sebhoss]

The build log shows something like compile: writing output: write $WORK/b1263/_pkg_.a: no space left on device. I'm not entirely sure what $WORK is but I wanted to clean that folder. I'm already running with --parallelism 1 to reduce the stress on those limited runners. The release workflow looks simple (link) - it basically just calls actions/setup-go and then runs goreleaser/goreleaser-action.

That said ... you are right. I just build this thing locally and get binaries > 700MB as well which did not happen before and certainly needs a deeper look on my end 😓 Thanks for the pointer!

[jftuga]

I submitted a PR that may help a small bit by having goreleaser create .tar.xz archives on Darwin and Linux. Since these have a better compression ratio vs zip, it will save some disk space.

[caarlos0]

if its failing at build time (looks like it), the archiving part will never even happen...

[sebhoss]

😅 yeah this is super weird: release 2024.4.8 was built from commit 629a9dffa256ddee324779d31e8c89b83ac5eb48 in this action which used go 1.22.1. Building that exact commit with that exact version of the go compiler using the exact same build arguments produces a ~700MB file but produced a ~40MB file 3 weeks ago.

I have no clue what could have changed in the last few weeks to cause this. What could possibly affect multiple machines and span across operating systems? I really don't know but I'm certain it's not goreleaser, thus I'm closing this one here and schedule an appointment with a shrink to talk about this. Thanks for the help here guys!

[sebhoss]

Oh wait - maybe it is something in goreleaser? I was just testing the size differences in metio/terraform-provider-k8s#137 (thanks @jftuga) and realized that I only looked at the size of the compiled binary before but not the .zip file that is created at the end. Zipping that ~700MB file produces a ~46MB file.

So I guess what I'm asking is: Can I somehow tell goreleaser to produce a .zip per OS/ARCH before continuing with another build target and clean up inbetween those builds?

[caarlos0]

So I guess what I'm asking is: Can I somehow tell goreleaser to produce a .zip per OS/ARCH before continuing with another build target and clean up inbetween those builds?

unfortunately no...

I would recommend you use something like https://github.com/jondot/goweight to see what is causing your binaries to be this big...

And/or reduce the number of target OSes.

[sebhoss]

ok thanks again for your time & help!