improve error message on podman load with disk full

[martinetd]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind feature

Description

podman load can fail with ENOSPC in a few places.
When failing reading an archive from stdin, it first copies it to /var/tmp and that has a reasonable error if FS is full (Error: error copying file write /var/tmp/podman110002836: copy_file_range: no space left on device), but in later steps there is no error indicating the FS is full, only a vague message saying the archive does not contain anything useful:

# podman load -i demo-image.tar 
Getting image source signatures
Copying blob 9d93ee5b513b skipped: already exists  
Copying blob 851421b54ec9 [===========>--------------------------] 828.5MiB / 2.5GiB
Error: payload does not match any of the supported image formats (oci, oci-archive, dir, docker-archive)

or, if failing a bit later:
# podman load -i demo-image.tar 
Getting image source signatures
Copying blob 9d93ee5b513b skipped: already exists  
Copying blob 851421b54ec9 done  
Copying config fe78706b20 done  
Writing manifest to image destination
Storing signatures
Error: payload does not match any of the supported image formats (oci, oci-archive, dir, docker-archive)

Steps to reproduce the issue:

1. make a tiny /var/tmp and/or storage, and produce a large image file with podman save (afaiu first error is tmp, second storage)

2. podman load -i said image

3. observe error

Describe the results you received:

difficult to understand error message.

Describe the results you expected:

the string "no space left on device" or equivalent should be present

Additional information you deem important (e.g. issue happens only occasionally):

Semi-unrelated, note that in this case the person who contacted me had 6.2GB free, for a 2.8GB image -- that's more than 2 times the image size, so they did not suspect ENOSPC at all.
Using podman load -i file.tar instead of podman load < file.tar reduces the image requirement from a factor 3 to 2 (iff the tar is not compressed), but that is still a severe limitation for usage on embedded devices with constrained space. We're basically telling our customers that, yes, they have over 6GB available in the container partition but they can only ever have containers as big as ~3GB because we can't import bigger layers. (well the image could be split in layers imported one at a time...)

I looked at the code to see if streaming tar content would be possible instead of storing an intermediate copy in /var/tmp but that does not look easy (looking at containers/image/v5/docker/internal/tarfile/reader.go), is there interest in making streaming possible? I might be able to spend some time on it early next year.

Output of podman version:

podman version 3.2.3

Output of podman info --debug:

host:
  arch: arm64
  buildahVersion: 1.21.3
  cgroupControllers:
  - cpuset
  - cpu
  - cpuacct
  - blkio
  - memory
  - devices
  - freezer
  - net_cls
  - perf_event
  - net_prio
  - pids
  - rdma
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: Unknown
    path: /usr/bin/conmon
    version: 'conmon version 2.0.29, commit: b388b959974dee50d451f88949b3499c3ca6ca42'
  cpus: 4
  distribution:
    distribution: alpine
    version: 3.14.2
  eventLogger: file
  hostname: armadillo
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.10.35-2-at
  linkmode: dynamic
  memFree: 479137792
  memTotal: 1745121280
  ociRuntime:
    name: crun
    package: Unknown
    path: /usr/bin/crun
    version: |-
      crun version 0.21
      commit: c4c3cdf2ce408ed44a9e027c618473e6485c635b
      spec: 1.0.0
      +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 78h 25m 28.33s (Approximately 3.25 days)
registries: {}
store:
  configFile: /etc/atmark/containers_storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.imagestore: /var/app/storage
    overlay.mountopt: nodev
  graphRoot: /var/app/storage_persistent
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 3
  runRoot: /run/containers/storage
  volumePath: /var/app/storage_persistent/volumes
version:
  APIVersion: 3.2.3
  Built: 1626474627
  BuiltTime: Sat Jul 17 07:30:27 2021
  GitCommit: b228095d5959f9c0f65c154056ab8a548ab2ddc6
  GoVersion: go1.16.4
  OsArch: linux/arm64
  Version: 3.2.3

Package info (e.g. output of rpm -q podman or apt list podman):

# apk list podman
podman-3.2.3-r0 aarch64 {podman} (Apache-2.0) [installed]

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

No

Additional environment details (AWS, VirtualBox, physical, etc.):

embedded arm64 system

[vrothberg]

Thanks for reaching out, @martinetd. I will improve on that.

[martinetd]

Thanks!

[vrothberg]

I opened containers/common#784 to address the issue. The output now looks as follows:

podman (main) $ ./bin/podman load -i libpod/
Error: payload does not match any of the supported image formats (oci, oci-archive, dir, docker-archive):
 * oci: parsing "localhost/libpod/": parsing named reference "localhost/libpod/": invalid reference format
 * oci-archive: creating temp directory: untarring file "/var/tmp/oci144332297": read /home/vrothberg/go/src/github.com/containers/podman/libpod: is a directory
 * dir: parsing "localhost/libpod/": parsing named reference "localhost/libpod/": invalid reference format
 * docker-archive: detecting compression for file "libpod/": detecting compression: read libpod/: is a directory

[vrothberg]

Actually, the (oci, oci-archive, dir, docker-archive) can be removed since the formats are now listed below.

[martinetd]

Ah, I had forgotten we're guessing the format here... I wonder if we could make a difference between "probing failed, try next format" error which would not report a message, and fatal-ish error (like ENOSPC) that would? It's probably hard to make the distinction in some cases though, I'm not sure it's a good idea.
(My feeling is that this is still going to be somewhat confusing for "normal" users.)

That being said, it's a net improvement -- at least more advanced users have something to look at in logs, so this is good enough for me.
Thanks again!

[vrothberg]

podman pull {oci,dir,oci-archive,docker-archive}:$path allows for loading a specific format. I think that load needs to continue doing the dancing.

[vrothberg]

containers/common#784 merged and addressed the issue. Note that this will not make it into Podman 3.4 but the release after.