[v0.10.0-rc2 regression] rootless: buildkitd: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"

[AkihiroSuda]

v0.10.0-rc1 and older:

$ rootlesskit ./bin/buildkitd 
INFO[2022-07-16T22:16:43+09:00] auto snapshotter: using overlayfs            
INFO[2022-07-16T22:16:43+09:00] found worker "wytloo7jcwd808vhkf76xsl05", labels=map[org.mobyproject.buildkit.worker.executor:oci org.mobyproject.buildkit.worker.hostname:suda-ws01 org.mobyproject.buildkit.worker.snapshotter:overlayfs], platforms=[linux/amd64 linux/amd64/v2 linux/amd64/v3 linux/amd64/v4 linux/386] 
WARN[2022-07-16T22:16:43+09:00] rootless mode is not supported for containerd workers. disabling containerd worker. 
INFO[2022-07-16T22:16:43+09:00] found 1 workers, default="wytloo7jcwd808vhkf76xsl05" 
WARN[2022-07-16T22:16:43+09:00] currently, only the default worker can be used. 
INFO[2022-07-16T22:16:43+09:00] running server on /run/user/1001/buildkit/buildkitd.sock
(Works as expected)

v0.10.0-rc2 and newer:

$ rootlesskit ./bin/buildkitd 
INFO[2022-07-16T22:17:43+09:00] auto snapshotter: using overlayfs            
INFO[2022-07-16T22:17:43+09:00] found worker "wytloo7jcwd808vhkf76xsl05", labels=map[org.mobyproject.buildkit.worker.executor:oci org.mobyproject.buildkit.worker.hostname:suda-ws01 org.mobyproject.buildkit.worker.snapshotter:overlayfs], platforms=[linux/amd64 linux/amd64/v2 linux/amd64/v3 linux/amd64/v4 linux/386] 
buildkitd: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
failed to connect client to "/run/containerd/containerd.sock" . make sure containerd is running
github.com/moby/buildkit/worker/containerd.NewWorkerOpt
        /src/worker/containerd/containerd.go:33
main.containerdWorkerInitializer
        /src/cmd/buildkitd/main_containerd_worker.go:262
main.newWorkerController
        /src/cmd/buildkitd/main.go:686
main.newController
        /src/cmd/buildkitd/main.go:630
main.main.func3
        /src/cmd/buildkitd/main.go:263
github.com/urfave/cli.HandleAction
        /src/vendor/github.com/urfave/cli/app.go:526
github.com/urfave/cli.(*App).Run
        /src/vendor/github.com/urfave/cli/app.go:288
main.main
        /src/cmd/buildkitd/main.go:313
runtime.main
        /usr/local/go/src/runtime/proc.go:255
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1581
[rootlesskit:child ] error: command [./bin/buildkitd] exited: exit status 1
[rootlesskit:parent] error: child exited: exit status 1

Thanks to a user who reported in the slack

[AkihiroSuda]

cc @ktock

[AkihiroSuda]

Workaround

rootlesskit buildkitd --containerd-worker=false --oci-worker=true

• docs/rootless.md: add --containerd-worker=false --oci-worker=true explicitly #2966

[tonistiigi]

What change introduced this?

[AkihiroSuda]

What change introduced this?

21aeba6#diff-e5aa7a28d320265f96e05c055008645785a4e6b8658537b776edaee7db284f26L153

[tonistiigi]

So that change does not work and containerd worker does not work with rootless?

[AkihiroSuda]

The containerd worker does work, but only when the buildkitd and the containerd are executed in the same user/mount/network namespaces.
This behavior is expected, but it shouldn't error out when the buildkitd is being executed in its own namespaces and lacks access to containerds.cok

[ktock]

LGTM for the workaround. I'll submit a patch to fix this issue.

[ktock]

Fixed in #2968