setting net.ipv4.ping_group_range fails (uidmap/gidmap)

[yogo1212]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

When spinning up a container for opensearch, there's an error (further down in the template 🔥 ).

Steps to reproduce the issue:

1. Run podman run --rm --uidmap 1000:0:1 --gidmap 1000:0:1 -v /mnt/opensearch1:/usr/share/opensearch/data -p 9200:9200 -p 9600:9600 docker.io/opensearchproject/opensearch:latest
   You probably don't want the -v.

Describe the results you received:

Error: OCI runtime error: crun: write to /proc/sys/net/ipv4/ping_group_range: Invalid argument

Describe the results you expected:

Preferably, a running container - but, honestly: Loads of other errors from opensearch.

Additional information you deem important (e.g. issue happens only occasionally):

The calling user's UID/GID is 1003/1003.
Opensearch uses UID/GID 1000/1000 in their Docker image, so there are --uidmap and --gidmap parameters mapping that the respective ID of the current user.

The value on the host: net.ipv4.ping_group_range = 0 2147483647

Because of the mapping and previous errors with sysctl, I think this could be related to #13194 and/or #11922.
Podman is a bit newer on my machine though (version further down in the template 🥇)

There's a docker issue with a similar error message.

Setting default_sysctls = [] in ~/.config/containers/containers.conf is a workaround but not very practical (have to create another user to not interfere with other containers).

Output of podman version:

podman version 4.1.1

Output of podman info:

host:
  arch: amd64
  buildahVersion: 1.26.2
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.2-2.el9_0.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.2, commit: c22a156d8b67bfc2959fafd1ae718134423045ab'
  cpuUtilization:
    idlePercent: 93.97
    systemPercent: 2.38
    userPercent: 3.65
  cpus: 2
  distribution:
    distribution: '"rocky"'
    version: "9.0"
  eventLogger: file
  hostname: redacted
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1002
      size: 1
    - container_id: 1
      host_id: 231072
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1002
      size: 1
    - container_id: 1
      host_id: 231072
      size: 65536
  kernel: 5.14.0-70.26.1.el9_0.x86_64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 128610304
  memTotal: 4004556800
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.4.5-2.el9_0.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.4.5
      commit: c381048530aa750495cf502ddb7181f2ded5b400
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1002/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_NET_RAW,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-2.el9_0.x86_64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 2140327936
  swapTotal: 2147479552
  uptime: 29h 32m 25.61s (Approximately 1.21 days)
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.access.redhat.com
  - registry.redhat.io
  - docker.io
store:
  configFile: /home/opensearch/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/opensearch/.local/share/containers/storage
  graphRootAllocated: 40756367360
  graphRootUsed: 11996954624
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/user/1002/containers
  volumePath: /home/opensearch/.local/share/containers/storage/volumes
version:
  APIVersion: 4.1.1
  Built: 1660063436
  BuiltTime: Tue Aug  9 16:43:56 2022
  GitCommit: ""
  GoVersion: go1.17.12
  Os: linux
  OsArch: linux/amd64
  Version: 4.1.1

Package info (e.g. output of rpm -q podman or apt list podman):

podman-4.1.1-1.el9_0.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes.

The specified ID lies within ping_group_range.

I have tested with podman 4.2.0 on my machine because I can't install anything on the server.
There's a different error but it looks to me like that's happening earlier.

Error: copying system image from manifest list: writing blob: adding layer with blob "sha256:fd06b60db4951fd95b43a9e73fb084fe7d2afcd4ff464472ead5548cd1c28717": processing tar file(container ID 0 cannot be mapped to a host ID): exit status 1

Additional environment details (AWS, VirtualBox, physical, etc.):

Rocky Linux 9 on Kernel 5.14 on a virtual server.

[Luap99]

@giuseppe PTAL

[giuseppe]

I think --uidmap 1000:0:1 --gidmap 1000:0:1 is not enough since you are not mapping any user to root in the container.

Could you try something like: --uidmap 1000:0:1 --uidmap 0:1:1? Tip: if --gidmap is not specified, then it uses the same values as --uidmap.

Does that make any difference?

[yogo1212]

@giuseppe thank you!

i've noticed there is actually an error message for that:

$ podman run --rm --uidmap 1000:0:1 -it docker.io/library/alpine:latest sh
Error: container uses ID mappings ([]specs.LinuxIDMapping{specs.LinuxIDMapping{ContainerID:0x3e8, HostID:0x0, Size:0x1}}), but doesn't map UID 0

adding a -u somehow circumvents the check leading to the more helpful error message:

$ podman run --rm --uidmap 1000:0:1 -u 1000:1000 -it docker.io/library/alpine:latest sh
Error: OCI runtime error: crun: write to /proc/sys/net/ipv4/ping_group_range: Invalid argument

i assume the code setting the uid for the application has it and the code writing to ping_group_range doesn't.

it works with the added --uidmap 0:1:1 - as you promised - but with a few warnings:

$ podman run --rm --uidmap 1000:0:1 --uidmap 0:1:1 -it docker.io/library/alpine:latest sh
WARN[0000] Additional gid=1 is not present in the user namespace, skip setting it 
WARN[0000] Additional gid=2 is not present in the user namespace, skip setting it 
WARN[0000] Additional gid=3 is not present in the user namespace, skip setting it 
WARN[0000] Additional gid=4 is not present in the user namespace, skip setting it 
WARN[0000] Additional gid=6 is not present in the user namespace, skip setting it 
WARN[0000] Additional gid=10 is not present in the user namespace, skip setting it 
WARN[0000] Additional gid=11 is not present in the user namespace, skip setting it 
WARN[0000] Additional gid=20 is not present in the user namespace, skip setting it 
WARN[0000] Additional gid=26 is not present in the user namespace, skip setting it 
WARN[0000] Additional gid=27 is not present in the user namespace, skip setting it 
/ #

these warnings go away when adding -u back:

$ podman run --rm --uidmap 1000:0:1 --uidmap 0:1:1 -u 1000:1000 -it docker.io/library/alpine:latest sh
~ $ 

not having to specify --gidmap is practical but it isn't documented. is it "stable"?
(the example "Set UID/GID mapping in a new user namespace" also uses both)

[eriksjolund]

@yogo1212 To fix the error
Error: OCI runtime error: crun: write to /proc/sys/net/ipv4/ping_group_range: Invalid argument
in the command

[test@fedora ~]$ podman run \
             --rm \
             --uidmap 1000:0:1 \
             -u 1000:1000 \
             -it \
                docker.io/library/alpine:latest sh
Error: OCI runtime error: crun: write to /proc/sys/net/ipv4/ping_group_range: Invalid argument
[test@fedora ~]$ 

add the command-line option --sysctl "net.ipv4.ping_group_range=1000 1000"

[test@fedora ~]$ podman run \
              --sysctl "net.ipv4.ping_group_range=1000   1000" \
              --rm  \
              --uidmap 1000:0:1 \
              -u 1000:1000 \
              -it \
                 docker.io/library/alpine:latest sh
~ $ exit
[test@fedora ~]$

The test was run on Fedora 36 with Podman 4.2.1

[test@fedora ~]$ podman --version
podman version 4.2.1

[giuseppe]

$ podman run --rm --uidmap 1000:0:1 --uidmap 0:1:1 -u 1000:1000 -it docker.io/library/alpine:latest sh
~ $ 

not having to specify --gidmap is practical but it isn't documented. is it "stable"?
(the example "Set UID/GID mapping in a new user namespace" also uses both)

if you do not specify a gidmap, then its value is copied from the uidmap.

To avoid the issue with ping_group_range, I've added this line to the /etc/containers/containers.conf file:

[containers]
  default_sysctls = ["net.ipv4.ping_group_range=0 0"]

[yogo1212]

@giuseppe

Thanks for addressing this recurring error/mistake.

To avoid the issue with ping_group_range, I've added this line to the /etc/containers/containers.conf file:

Is that that a fix or just hiding an underlying error with unmapped uid 0?

Either way, it's an improvement, imo :-)

[yogo1212]

@eriksjolund

thanks - interesting!
sure! just restrict the range to the user.
i was thinking in a completely different direction.

atm, i'm sticking to the 'just map uid 0' approach.
but it's good to know there are alternatives

[github-actions]

A friendly reminder that this issue had no activity for 30 days.