projen workflows cannot run docker commands by default

[kaizencc]

Problem:

Sometimes docker commands are bundled into projen commands. For example, in https://github.com/cdklabs/awscdk-asset-awscli, we specify a pre-compile step that executes a build.sh script. This script runs a docker build command, which builds a local image. This encounters the following problem:

Run npx projen build
👾 build » default | node .projenrc.js
👾 build » pre-compile | layer/build.sh
>> Building AWS Lambda layer inside a docker image...
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: \
Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&shmsize=0&t=aws-lambda-layer&target=&ulimits=null&version=1": \
dial unix /var/run/docker.sock: connect: permission denied

Under the hood, projen by default is using the jsii/superchain:1-buster-slim-node14 container image and docker commands are not set up to be run inside this image. Therefore, the following solution is necessary:

workflowBootstrapSteps: [
    {
      name: 'Change permissions on /var/run/docker.sock',
      run: 'sudo chown superchain /var/run/docker.sock',
    },
  ],

Which gives the superchain user ownership of the docker socket.

Proposal:

This is a sharp edge that we had to figure out, and I can't imagine there aren't other scenarios where we want to run docker commands in a projen-managed repository. The proposal is to surface this command as a boolean property on the projen library construct, like so:

const project = new awscdk.AwsCdkConstructLibrary({
  ...
  allowDockerCommands: true,
});

---

And, because of aws/jsii#3746, configuring passwordless sudo requires additional changes to .projenrc.js. We have been able to work around this via projen escape hatches:

const buildWorkflow = project.tryFindObjectFile('.github/workflows/build.yml');
buildWorkflow.patch(JsonPatch.add('/jobs/build/container/options', '--group-add sudo'));

Until the issue in jsii is somehow addressed, this will need to be baked into this proposal as well.

[berenddeboer]

I propose allowDockerCommands would be the default. It would be a surprise not to have that, and another bump in using projen. These are all little problems you want to fix. Unless there's a security or size option, features should be enabled I feel.

[berenddeboer]

Thank you so much @kaizencc you just saved me heaps of time here. Googled your post, applied the fixes, and everything now runs in GitActions, and my cdk-rds-sql library has now turned up on Construct Hub. Couldn't have done this without projen in this short amount of time.

[mrgrain]

What worked for me, was to add the docker group. Unfortunately this is just a number and not a nice name. I guess it might change and break again without warning.

const buildWorkflow = project.tryFindObjectFile('.github/workflows/build.yml');
buildWorkflow.patch(JsonPatch.add('/jobs/build/container/options', '--group-add 121'));

[mrgrain]

Finally open an issue with the GitHub team
actions/runner-images#6505

[mrgrain]

Update: I found the best workaround for this was to simply not use the superchain image:

https://github.com/mrgrain/streamlink-serverless/blob/main/projenrc/WorkflowNoDockerPatch.ts

Additionally requires Node.js to be set-up, but that's okay.

Why does this work? projen doesn't actually run any non-Node.js tasks in the main build. So there really is no inherent advantage of using the superchain image for this. All the subsequent packaging and release jobs already setup the required tools as needed.

I'm strongly considering to make this the default. Particularly since running inside Docker also suffers from the issue that Docker bundling won't work (see aws/aws-cdk#8799)