add "podman container update" command to update the cgroup configuration of running containers

[yule-sun]

I found that the memory limit of the container does not take effect in my Ceph cluster environment.

1. podman version
   [root@zl-k8s-ceph72 zzcloud]# podman version
   Client: Podman Engine
   Version: 4.0.2
   API Version: 4.0.2
   Go Version: go1.17.7

Built: Wed Mar 16 03:15:06 2022
OS/Arch: linux/amd64

2. Memory usage shown by podman stats
   

3. memory limit displayed by ceph orch ps
   

What parameters should I use, or modify those config files to adjust the memory limit of containers in podman?
Looking forward to your reply, thanks

[vrothberg]

Thanks for reaching out, @yule-sun.

It's not possible with podman directly but probably by tweaking the cgroups after the fact. But we may be leaving supported territory.

@giuseppe WDYT?

[mheon]

Using the OCI runtime's update command should allow it, but @vrothberg is correct that this is not supported directly from Podman (and likely will never be - Podman containers are immutable once created). As such, if the container were to restart, the altered resource limit would be lost.

This is the use-case that podman clone was intended for, though. If you can tolerate re-creating the container, podman container clone can alter the memory limit.

[rhatdan]

Should podman container clone have an option to copy the read/write layer? Or do a Replace?
Copy the read/write layer to a tar ball, create new container with content, remove old container, rename new container to old container name.
Lots of moving parts, but this would get you pretty close to update.
I know container ids would change.

[giuseppe]

but it would still bring down the process running in the container.

I think we could have a podman update that updates the cgroup at runtime without changing the container configuration.

[mheon]

I would not be opposed to that.

[giuseppe]

that would simply be a wrapper around $OCI_RUNTIME update.

In the meanwhile,as @vrothberg suggested, you can try to use directly the $OCI_RUNTIME update command:

$ podman run --rm -ti --memory 100M fedora sh
sh-5.1# cat /sys/fs/cgroup/memory.max
104857600

From another terminal:

$ crun update --memory=200000000 3fa4986e1ace56a9e758341f7e69794fddff95cad9f4964393741c7a36b84dc6

and back to the running container:

sh-5.1# cat /sys/fs/cgroup/memory.max
199999488

[mheon]

@rhatdan I think one of the stretch goals for clone was to commit the container being cloned and start from the new image

[yule-sun]

I cann't found the clone subcommand in my system, and i cann't use the update command too.

[yule-sun]

那将只是一个包装器$OCI_RUNTIME update。

同时，作为@vrothberg建议，你可以尝试直接使用$OCI_RUNTIME update命令：

$ podman run --rm -ti --memory 100M fedora sh
sh-5.1# cat /sys/fs/cgroup/memory.max
104857600

从另一个终端：

$ crun update --memory=200000000 3fa4986e1ace56a9e758341f7e69794fddff95cad9f4964393741c7a36b84dc6

并返回到正在运行的容器：

sh-5.1# cat /sys/fs/cgroup/memory.max
199999488

Hi sir
Can this command be used for containers that are already running, or only for new containers? Running on my system gives the following error.

[giuseppe]

it can be used only for running or paused containers

[rhatdan]

So Podman update would only support cgroups configuration flags on a running container. And would only last for the length of time the container is running. I think that makes some sense.

[cdoern]

I am just seeing this now. I could take this on either from the clone approach or the podman update approach. @mheon @giuseppe @rhatdan which one are you more in favor of? IMO, I think having a seprate update command makes sense if there is a runtime function to do this already.

[mheon]

I'd go with the update command - improves our Docker compat, even if changes are not persistent.

[rhatdan]

I agree. this would at least allow users to setup scripts to modify the containers on each start.