Docker service support for: --userns=host

[jefflill]

FEATURE REQUEST

Description

--userns for docker service create and update would be useful.

I'm trying to use userns-remap to secure my cluster containers and services transparently by default but I have a few services that need to mount files from the Docker host and I was hoping to use docker service create --userns=host ... for these few services but there doesn't appear to be a way to do this using the normal command line. I suppose I could deploy a stack but I'd rather not because it mangles the service name to be stack-service instead of just service.

I was hoping to use --user=ID to explicitly run the service with a specific host user but this doesn't work because (I believe) the container's file system restricts access to only the root user. I might be able to work around this by modifying the Dockerfiles by adding a USER and setting the container filesystem permissions, but this breaks my hope for transparency.

Specifically, I'm requesting support for:

docker service create --userns=host ...
docker service update --userns=host ...

This will make it possible to secure Docker service by default without needing to munge Dockerfiles for any exceptions.

Related issue: #25303
Related PR (docker run): #20111

Output of docker version:

  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.5
  Git commit:   9ee9f40
  Built:        Thu Apr 26 07:15:30 2018
  OS/Arch:      linux/amd64
  Experimental: false
rClient:: command not found
root@manager-0:~#  Version:      18.03.1-ce
Version:: command not found

Output of docker info:

Containers: 7
 Running: 2
 Paused: 0
 Stopped: 5
Images: 3
Server Version: 18.03.1-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: fluentd
Cgroup Driver: cgroupfs
Plugins:
 Volume: local neon
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
 NodeID: sw6o4jpzdwri2dz5zgvq6mju0
 Is Manager: true
 ClusterID: gq87clficzj7iaxahgafwri5w
 Managers: 1
 Nodes: 4
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 10
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
 Autolock Managers: false
 Root Rotation In Progress: false
 Node Address: 10.0.0.30
 Manager Addresses:
  10.0.0.30:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 773c489c9c1b21a6d78b5c538cd395416ec50f88
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
 userns
Kernel Version: 4.4.0-119-generic
Operating System: Ubuntu 16.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 3.936GiB
Name: manager-0
ID: BIYB:LQF3:PO5J:Z2EN:CCNF:3ERB:I7JV:KZNF:OVL4:KXUQ:RKKW:JJ75
Docker Root Dir: /mnt/docker/231072.231072
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Registry Mirrors:
 https://manager-0.neon-registry-cache.hive:5001/
 https://docker.io/
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

Running on Windows 10/Pro on local Hyper-V Ubuntu 16.04 VMs.

[thaJeztah]

/cc @estesp @justincormack

[Finkes]

I think this should also be possible for stacks using userns_mode: "host" inside compose files. I have the same problem for our docker swarm mode cluster using the userns-remap feature to make the containers more secure.

However some core services like traefik need to be run as privileged user and this is only possible in combination with the userns flag as described here.

Unfortunately the userns-remap feature (called userns_mode in compose file) is not supported for docker swarm mode yet:

userns_mode

userns_mode: "host"
Disables the user namespace for this service, if Docker daemon is configured with user namespaces. See dockerd for more information.

source: https://docs.docker.com/compose/compose-file/#userns_mode

@thaJeztah could you maybe share some insights why this is not (yet) possible? (I'm pretty sure that there is a good reason for that).

[thaJeztah]

could you maybe share some insights why this is not (yet) possible? (I'm pretty sure that there is a good reason for that).

I don't know about this particular option specific, but the primary reason for not every option being available as option for swarm services, is that not all of the options on docker run are portable. Some of the options are didn't make much sense in a distributed / orchestrated situation (at least, not without a proper design about how they would behave in those situations).

There are many options on docker run that make sense for some use-cases; some of which are very low-level, and effectively a one-on-one mapping to kernel features.

In situations where decisions are not trivial, maintainers stick to the "no is temporary; yes is forever" saying, which means that in doubt, the safe choice is to not implement a feature. Adding a feature is easier than removing, or changing a feature (both will break users; including those that run this in production), so an initial "no" can change into a "yes" in future (if more insight is gained).

Looking for a more portable/generic approach, including where possible, cross-platform portability, work was started on defining "entitlements" for containers (see #32801). Entitlements are still on the (long term) roadmap, but got out-prioritised by other work 😅.

Back to this feature; I can see this being useful (given that user namespaces are a daemon-level option). Having said that, I'd be interested to check with some other maintainers to see if;

• Adding this option would "conflict" with entitlements (if work continues on that), and/or if that's a concern
• More flexibility is possible once migration to containerd is complete (currently containerd is used for the runtime, but not yet for image management); I'm wondering if per-container user-namespace mapping would be more easily implemented in that situation (instead of only a global mapping), and if the design should be adjusted to fit that.

[CWiesbaum]

We got the same requirements here as @Finkes. Running Docker Swarm mode we want to deploy a stack for administration and monitoring purpose. The userns remap prevents access to some host resources.

[eoftedal]

We would also like this to implemented to support running cAdvisor in swarm from a stack.

[mikesir87]

Chiming in too... the only workaround we have right now is to have a few nodes within our cluster run without the userns enabled, label them appropriately, and only schedule specific workloads on it. Definitely a workaround/hack, but can suffice for right now. Would be nice to not have special nodes and have services support this directly.

[cpuguy83]

docker service create --mount type=bind,src=/var/run/docker.sock,target=/var/run/docker.sock docker:latest docker run --rm --userns=host myImage

[sirlatrom]

docker service create --mount type=bind,src=/var/run/docker.sock,target=/var/run/docker.sock docker:latest docker run --rm --userns=host myImage

But that would only work if userns was not remapped on the Docker engine that the socket is being mounted from, right? Because the user inside the container would not have permissions to connect to the socket, the whole point of userns...

[thaJeztah]

I'd still be ok with having this implemented. I don't think "entitlements" are moving fast, and given that we have this option for docker run (#20111), seems ok to have this on services as well (my 0.02c).

[alphaDev23]

Adding further input to this feature request. This feature would be extremely valuable for certain stacks that require userns_mode set to host and deployment set to global.

[jefflill]

We’ve moved onto neonKUBE.

[thaJeztah]

Let me keep this one open, as it's still possible that it will be added

[gflcampos]

Any news on this?

[jagmsing]

Is there any update on this request ?

regards,
Jagmeet

[sidxz]

Still waiting for an update

[maxfield-allison]

very curious about this. If not implemented is there an alternative?