Light weight Nvidia docker images

[agunapal]

Description

It would be great to have light weight NVIDIA docker images.

Example:

• The compressed size of Nvidia CUDA 11.8 developer docker image is 2.92 GB.
• The uncompressed size is 7.37 GB

nvidia/cuda          11.8.0-devel-ubuntu20.04          ff3049f3773d   2 months ago   7.37GB

To support state of the AI libraries, the developer image may be needed.

Wondering if there is a common request yet from docker community - To have a light weight/slim version of these images?
With the explosion of generative AI use-cases, I wonder if it makes sense for the community to think about this. This could have a big impact on cost savings

In my research, I have seen docker enthusiasts trying to build a solution using slim as the base image, but I couldn't come across a good solution.

Wanted to get your thoughts on this request

[thaJeztah]

I don't think this is the right place to ask; that image is maintained by NVIDIA (but they happen to use Docker Hub to distribute the image); from the information on Docker Hub, it looks like the source-repository is https://gitlab.com/nvidia/cuda, so that may be the best location to ask the maintainers of that image;

[chauhang]

@thaJeztah We are from the PyTorch team and looking for ways to improve the docker footprint for PyTorch and related libraries installs. We would like to partner with Docker to find a better solution. Current images from Nvidia have lot of bloat making it challenging to use esp for Inference workloads. Every customer has to go through the pain point of creating optimized images for their environment. Are there any other suggestions from Docker development team that we can explore? Will be happy to discuss more in a meeting

[polarathene]

I have a bit of experience with optimizing image builds. Here's some tips:

• Use a tool like dive to get insights on an images layers and where the bulk of the size is introduced.
• You can use RUN --mount=type=
  • with type cache for package installs as a nice way to keep this extra weight out of an image, and retain the package manager cache for future builds should the image layer be invalidated in a future build (speeds up this part when the cache can be reused).
  • with type bind when you need to only have some data temporarily during build. This can be used to bring in files from other Docker images or earlier build stages instead of a COPY, avoiding redundant image weight.
    • Here is an example I contributed before, symlinks could not be copied directly (they'd be resolved to another copy of the file) so the needed binaries are accessed from an earlier image stage that had them to create the symlink allowing for a much slimmer image.
• Use the HereDoc feature to avoid && \ noise for multi-line RUN or many separate RUN as the nvidia Dockerfile has (that one isn't related to any of the DockerHub images though).
• I gave some informal review tips / advice on improving a Dockerfile which may be helpful (compression estimate, avoiding VOLUME, and build-time only ENV via ARG).
• Using a base image like Fedora or openSUSE there is support for configuring an "install root", which will allow you to specify only the packages you need and create a whole new base image tailored to only supporting that.
  • Here's an example with openSUSE and --installroot, it also uses the RUN --mount=type=cache and HereDoc features for you to reference. After that single RUN layer, a new build stage in the image is introduced to transfer that install root location to it's own image.
  • You may want to try the "install root" optimization last, as it doesn't always go smoothly. Personally I found openSUSE had compatibility issues within the container environment. Fedora was a better experience, but I don't have a reference to link you to for that.

DockerHub images (12.2.0 - Ubuntu 22.04):

• Base
• Runtime
• Devel

The bulk of the runtime weight is from a single layer that installs the runtime cuda libs, there's not much you can do about that AFAIK (unless you know what you can remove):

I was not able to run dive on the devel variant as this consumed all 17GB of disk I had remaining (WSL2 with docker pull command), and failed. Part of the issue there is WSL2 memory usage for the file buffer / cache seems to also use up actual disk space in addition to memory.

Presumably there isn't really much that can be saved at a glance. Bulk of the weight is from the package installs, so it's unclear what portion of that is "bloat" as I guess it'd vary by project?

If you already have identified optimized images being created for environments, perhaps you could document those and find a common optimized base with additional advice to customize for other needs?

[neersighted]

If you're interested in working with Docker on base images, the place to ask/make suggestions is probably docker-library/official-images.

That being said, generally images there are required to be free software (and blessed or directly maintained by the 'upstream,' in this case NVIDIA), which means that this may be out of scope for the project. In that case, you would be best off partnering with NVIDIA to create more suitable base images/improve their images, as DOI/docker-library is the only Docker Inc. sponsored image-authoring project/process.