Explicit toolchain dependency (or stop using jsii/superchain) #1065
Comments
|
@RomainMuller @Chriscbr @iliapolo curious what y'all think about this. |
|
Sounds like this can go towards this issue as well. I think having a new dependency type for this sort of thing would be super useful. |
Just to put a number on this - from what I'm seeing, it takes about 50-70 seconds to pull the jsii/superchain at the beginning of every GitHub workflow run. (Q: will be longer/shorter with the newer superchain images?) Meanwhile running Part of me wonders if running on container images is the correct solution in the long run, since they're supposed to be this universal-ish tool for defining build/runtime environments. At some point once you have enough dependencies, saving them as an image should (in theory) save you some time. But this assumes you have already have the image built and downloaded... I feel like any long term solution should probably support both kinds of strategies:
|
|
This issue is now marked as stale because it hasn't seen activity for a while. Add a comment or it will be closed soon. |
|
Keep |
|
This issue is now marked as stale because it hasn't seen activity for a while. Add a comment or it will be closed soon. |
|
This issue is now marked as stale because it hasn't seen activity for a while. Add a comment or it will be closed soon. If you wish to exclude this issue from being marked as stale, add the "backlog" label. |
|
There is also a huge issue with running Docker in Docker on GitHub action and sharing files on the file system. Also related #2094 |
… in workflows BREAKING CHANGE: default to node16 To use any other Node version, explicitly provide the desired version number BREAKING CHANGE: remove `jsii/superchain` image from AwsCdkConstructLibrary workflows Using `jsii/superchain` provides no tangible benefit over installing dependencies with GitHub Actions. However because AWS CDK Constructs often require to run docker commands, with image GitHub Action workflows end up attempting to run Docker in Docker. This is not trivial to achieve (see #2094 & aws/aws-cdk#8799). Additionally the existing build and package workflows had an inconsistent usage of the image, causing further problems. To restore the old behavior, set `options.workflowContainerImage` to the desired image. Fixes #2094 Closes #1065
… in workflows BREAKING CHANGE: default to node16 To use any other Node version, explicitly provide the desired version number BREAKING CHANGE: remove `jsii/superchain` image from AwsCdkConstructLibrary workflows Using `jsii/superchain` provides no tangible benefit over installing dependencies with GitHub Actions. However because AWS CDK Constructs often require to run docker commands, with image GitHub Action workflows end up attempting to run Docker in Docker. This is not trivial to achieve (see #2094 & aws/aws-cdk#8799). Additionally the existing build and package workflows had an inconsistent usage of the image, causing further problems. To restore the old behavior, set `options.workflowContainerImage` to the desired image. Fixes #2094 Closes #1065
… in workflows BREAKING CHANGE: default to node16 To use any other Node version, explicitly provide the desired version number BREAKING CHANGE: remove `jsii/superchain` image from AwsCdkConstructLibrary workflows Using `jsii/superchain` provides no tangible benefit over installing dependencies with GitHub Actions. However because AWS CDK Constructs often require to run docker commands, with image GitHub Action workflows end up attempting to run Docker in Docker. This is not trivial to achieve (see #2094 & aws/aws-cdk#8799). Additionally the existing build and package workflows had an inconsistent usage of the image, causing further problems. To restore the old behavior, set `options.workflowContainerImage` to the desired image. Fixes #2094 Closes #1065
…flows (#2510) BREAKING CHANGE: default to node16 To use any other Node version, explicitly provide the desired version number BREAKING CHANGE: remove `jsii/superchain` image from AwsCdkConstructLibrary workflows Using the `jsii/superchain` image provides no tangible benefit over installing dependencies with GitHub Actions. However AWS CDK Constructs often require to run docker commands, and thus using the image forces GitHub Action to exectute commands as Docker in Docker. This is does not work in many situations and generic fixes are not reliable (see #2094 & aws/aws-cdk#8799). Additionally the existing build and package workflows had an inconsistent usage of the image, causing more problems. To restore the old behavior, set `options.workflowContainerImage` to the desired image. Fixes #2094 Closes #1065 --- By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Currently, all JSII projects use a "kitchen sink" container image called
jsii/superchainwhich bundles all the tools needed to build & package JSII projects. This is a pragmatic solution, but it's not optimal. It means, for example, that users don't have much control over their build environment and that builds & releases are quite slow since they have to pull this huge image on every run.Projen should offer a fine-grained model for toolchain dependencies and have the various workflows synthesize based on this model. This will also allow us to offer developer-side support for tool installations (via tools like homebrew for example).
Abstractly, project types should be able to declare their toolchain dependencies via the Dependencies API. Probably via some special type of dependencies (or repurpose one of the existing types).
Then, we need some kind of a provider pattern that will allow mapping toolchain dependency to a provider-specific definition. For example, GitHub workflows will use one of the
actions/setup-xxxactions to install tools in workflows based on the project model. We will be able to then offer a local provider that will check that a certain tool is installed locally and even control it via tools likenvmorrvm, etc.The text was updated successfully, but these errors were encountered: