Error while enabling composefs with crio

[harche]

I am trying enable composefs with crio. To make sure crio has required versions of dependencies, I built it from this PR - cri-o/cri-o#7852

It seem to have the required dependencies vendored in like, storage v1.53.0, image/v5 v5.30.0, podman/v5 v5.0.0. I also made sure that the erofs module is loaded in the kernel.

# lsmod | grep erofs
erofs                  81920  0
libcrc32c              16384  6 nf_conntrack,nf_nat,erofs,openvswitch,nf_tables,xfs

In /etc/containers/storage.conf I have added following entries,

[storage.options]
# Storage options to be passed to underlying storage drivers
pull_options = {convert_images = "true", enable_partial_images = "true", use_hard_links = "false", ostree_repos=""}

and

[storage.options.overlay]
# Storage Options for overlay
use_composefs = "true"

yet, when the crio is started I see following error in the logs,

# journalctl -u crio | grep composefs
Mar 22 13:47:21 ip-10-0-29-210 crio[2094]: time="2024-03-22T13:47:21Z" level=warning msg="Failed to decode the keys [\"storage.options.overlay.use_composefs\"] from \"/etc/containers/storage.conf\""

I also verified that the crio binary is indeed using c/storage that has references to the composefs implementation,

# strings /bin/crio | grep -i composefs | wc -l 
14

Not sure what I am missing at this point. Any pointer is highly appreciated. Thanks.

[giuseppe]

have you vendored c/common?

[harche]

The crio PR, https://github.com/cri-o/cri-o/pull/7852/files#diff-33ef32bf6c23acb95f5902d7097b7a1d5128ca061167ec0716715b0b9eeaa5f6R21, seems to vendor containers/common v0.58.0 .