refactor: minio -> s3

[oprypkhantc]

Closes #21

[DrJume]

Looking good so far. Thanks 👍

[oprypkhantc]

@DrJume @LouisHaftmann Added defaults for the s3 driver that point to AWS S3. The title of the docs page now includes both S3 and MinIO, and the verbiage is now S3 storage to make it more clear the driver works with any S3-compatible storage

[LouisHaftmann]

We should probably test whether the prune function actually works with s3/minio. It is currently only used for unit tests so it's not that important.

[LouisHaftmann]

only one small change. after that we can merge this pr and I'll create a new pre-release build.

Thanks for your work so far! ❤️

[oprypkhantc]

Done. I've made sure to test the pruning - it works with S3 as expected :)

[oprypkhantc]

By the way, have you thought about making an issue/PR to actions/runner to allow setting a custom ACTIONS_CACHE_URL without patching the runner? We're not running them in Docker (as this has it's own challenges with Docker-in-Docker setup), meaning we had to disable automatic self-updates for the patch to stay. So maybe there's a change GitHub can support this natively 🙃

[DrJume]

Yeah this would be nice, but sadly it seems that GitHub is not interested in this change.

See here for more context:

• Caching on self-hosted runners github/docs#2271
• https://github.com/orgs/community/discussions/18549
• Add support for non-GitHub-hosted caching for self-hosted runners actions/toolkit#1051
• Ability to overwrite base url in actions/cache actions/toolkit#947

[DrJume]

By the way, have you thought about making an issue/PR to actions/runner to allow setting a custom ACTIONS_CACHE_URL without patching the runner? We're not running them in Docker (as this has it's own challenges with Docker-in-Docker setup), meaning we had to disable automatic self-updates for the patch to stay. So maybe there's a change GitHub can support this natively 🙃

Yeah, self-updates would be nice. We didn't think about that, because we use ephemeral runners, but it's a valid use case.
Maybe it is possible to hook into the self-update with a script which is executed afterwards to re-patch the binary?

[oprypkhantc]

Yeah this would be nice, but sadly it seems that GitHub is not interested in this change.

😢

Yeah, self-updates would be nice. We didn't think about that, because we use ephemeral runners, but it's a valid use case.
Maybe it is possible to hook into the self-update with a script which is executed afterwards to re-patch the binary?

Not sure, our devops manage that. We'll have to do something like that as GitHub does not promise compatibility with older version runners 😞 :

If you do not perform a software update within 30 days, the GitHub Actions service will not queue jobs to your runner. In addition, if a critical security update is required, the GitHub Actions service will not queue jobs to your runner until it has been updated.

[oprypkhantc]

@DrJume So it turns out it's almost impossible to make it work with non-ephemeral runners :/

Looking through the runner's code, there are no hooks we could use. Most of the self-update stuff is done on the C# side and there's a single sh script that runs to finalize the update, but the problem is that it's copied from the newly downloaded binary as opposed to just root of the runner. Moreover, there's no command to run the self-update - the only way it can be triggered is through GitHub sending specific messages to the runner.

That's unfortunate because it looks like all we can do is also run ephemeral runners :(

[DrJume]

@DrJume So it turns out it's almost impossible to make it work with non-ephemeral runners :/

Looking through the runner's code, there are no hooks we could use. Most of the self-update stuff is done on the C# side and there's a single sh script that runs to finalize the update, but the problem is that it's copied from the newly downloaded binary as opposed to just root of the runner. Moreover, there's no command to run the self-update - the only way it can be triggered is through GitHub sending specific messages to the runner.

That's unfortunate because it looks like all we can do is also run ephemeral runners :(

A last option maybe could be to watch the runner binary and patch it automatically.
I would recommend https://github.com/parcel-bundler/watcher for file watching with NodeJS.

But still, we would also recommend to use ephemeral runners.

[LouisHaftmann]

we are using ephemeral runners in our setup but kubernetes is required for docker in docker to work. maybe we could put together a guide for you if you're interested