Missing equivalent for --cgroupns=[host|private]

[thediveo]

On hosts with only a unified cgroup v2 controller hierarchy Docker now defaults (unless configured otherwise) to automatically creating private cgroup namespaces for created containers. The rationale here is to reduce leakage of potentially sensitive information about the cgroup hierarchy and configuration of the host, and thus other containers. Still, there are valid usecases to allow only specific containers to use the initial (host) cgroup namespace, especially for system diagnosis containers. For this, docker run introduced the CLI flag --cgroupns=[host|private] as of Docker-CE 20.0.0 and API 1.41.

Unfortunately, docker-compose doesn't offer any means to use the CLI flag functionality from docker compose files. This makes it impossible to use system diagnosis containers, such as cAdvisor, ... on cgroup v2 unified hierarchy-only systems anymore.

This situation would be solved by adding a cgroup field to service objects that would allow to specify the values of either "private" or "host", mirroring today's CLI flag functionality for handling the cgroup namespace creation for new containers.

services:
  foo:
    pid: host
    cgroup: host

[carlosabalde]

I think this has become an important limitation for many users since Docker Desktop uses cgroupv2. Any plans of adding this?

[ndeloof]

This require changes in the compose file format, please open a discussion on https://github.com/compose-spec/compose-spec

[thediveo]

already opened beginning of march 21: compose-spec/compose-spec#148

[ndeloof]

@thediveo right. Would you like to create a PR to add this attribute to the compose spec?

[thediveo]

I would like to, but I'm currently completely tied up in (software) architectural work for my employer and as I'm yet lacking the required orientation in nerdctl's software architecture I wouldn't be of any help or use. What might be helpful in the future to onboard more hands to nerdctl might be some kind of sw orientation document to better understand where to add/plug-in various features and how nerdctl way of interacting with runc is (which is an area where I'm currently lost).

[aem12f]

+1 here. This would be huge. This is getting overwhelming as new docker desktop versions for windows no longer support cgroupv1, and mac is hanging by a thread with the settings.json deprecatedcgroupv1 change.

Staying on an older version of docker desktop for now...but who knows how long that'll last

I think this has become an important limitation for many users since Docker Desktop uses cgroupv2. Any plans of adding this?

[thediveo]

(bump) any E.T.A.?

[hendrikmaus]

I was able to work around this limitation of compose by setting "default-cgroupns-mode": "host" in daemon.json.
Docker's documentation is misleading, since it claims that "host" is the default value, where in reality it defaults to "private".

So if you can change the daemon's defaults, this could be a valid workaround for the meantime.

[thediveo]

So if you can change the daemon's defaults, this could be a valid workaround for the meantime.

Unfortunately, that doesn't work for me/us. We have diagnosis containers that get deployed to production systems (not cloud production, but we're talking about real production systems for tangible goods) in order to check and diagnose workload-related issues that only tend to raise their ugly heads in production, but not in testing. And these are "sealed" Docker hosts where I simply cannot change the Docker daemon configuration. I fully agree with customers that this is an absolute "no-no-NO-NO-NOOOOO" 😀

[hendrikmaus]

Then you could work around the issue by not using Docker Compose at all, but by passing --cgroupns=host to docker run.

[thediveo]

That's not an option either, as the sealed system enforces Docker compose for deployment. FYI, https://industrial-edge.io/

[huapox]

+1

[litnimax]

+1

[nurturenature]

Docker compose's lack of ability to configure containers is not "stale".

It's impact only grows.

[ndeloof]

fixed by #10106

[carlosabalde]

I've seen #10106 was referenced in the changelog of Docker Compose 2.15.0, so I've done a quick check:

❯ /tmp/docker-compose version
Docker Compose version v2.15.0

❯ cat /tmp/docker-compose-test.yml
---
services:
  foo:
    image: ubuntu:jammy
    cgroup: host

❯ /tmp/docker-compose --file /tmp/docker-compose-test.yml ps 
services.foo Additional property cgroup is not allowed

I guess I'm missing something obvious here but I don't know what. Any ideas?

[ndeloof]

indeed, something went wrong here. Investigating

[nervo]

@ndeloof any news on this ?
I've just updated my docker desktop, coming with docker compose 2.18.1, and still:

services.foo Additional property cgroup is not allowed

[ndeloof]

$ cat compose.yaml 
services:
   web:
      cgroup: host
      image: nginx
$ docker compose config
services:
  web:
    cgroup: host
    image: nginx
...

please check docker compose version

[nervo]

@ndeloof thanks for your response :)

$ docker compose version
Docker Compose version v2.18.1

But i've just realized i use mutagen-compose as a wrapper to docker compose, which is stick on older versions (https://github.com/mutagen-io/mutagen-compose/blob/main/go.mod)

Sorry for the inconvenience, i will open an issue on their repo 😗

[EFinish]

Hey, I'm also experiencing an issue similiar to @carlosabalde and @nervo . I'm not sure if I haven't written group correctly as an element, but it seems good according to the docs.

# docker-compose.yaml
version: '3.8'
services:
   potato:
    build: .
    cgroup: host

I run the commands

$ docker compose version
Docker Compose version v2.23.0-desktop.1
$ docker-compose up --build
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services.potato: 'cgroup'

Does anyone know if I am doing something incorrectly?

[ndeloof]

@EFinish your docker compose version is probably outdated. Check docker compose version

[EFinish]

@ndeloof I showed in my first post that I checked already. The version is 2.23.0. I was previously running 2.15 and I had the same issue nonetheless.

[nervo]

@EFinish you mix both docker compose v1 ($ docker-compose) and v2 ($ docker compose) in your commands :)

[ndeloof]

@EFinish You should indeed use docker compose to run commands, not docker-compose:

$ cat compose.yaml 
services:
   potato:
    build: .
    cgroup: host
 $ docker compose config
services:
  potato:
    build:
      context: /Users/nicolas/truc
      dockerfile: Dockerfile
    cgroup: host
...

[EFinish]

Yeah, that was it. Thanks for the help @ndeloof @nervo