Support other OCI artifact types

[gaocegege]

skopeo can work with OCI images as well as the original Docker v2 images.

It will be great if we could support Helm Chart, CNAB and other OCI artifact types.

[rhatdan]

Please open PRs probably in github.com:containers/image and I am sure we would consider it.

[rhatdan]

@mtrmac @vrothberg WDYT?

[mtrmac]

Support… how? c/image reads a manifest+layers, converts manifests. How does any of this apply to non-image artifacts?

Sure, c/image/docker has a registry client, and that can be in principle used to copy arbitrary kinds of data, but that’s not trivially generalizable any other formats like docker-archive: or containers-storage: or even oci: .

It might make sense to factor out the generic c/image client… to be able to use it separately, I’m not sure.

I’m not quite saying “no” but it would be a pretty significant change to the c/image library goals/focus.

Maybe we can pretend that Helm charts are “images” that can’t be converted to any other format, and that would allow at least to use docker:// and dir: to copy them back and forth. I’m not sure that’s a good idea, but it’s, I suppose, something to try and the copy pipeline might adapt to it reasonably easily.

[rhatdan]

I am fine without supporting all transports. The question here is what is the purpose of skopeo. Is it for just copying OCI Images between different types of container storage or do users expect it to be able to handle different types of content stored at container registries?

[gaocegege]

Harbor and many other registries are supporting Helm Chart, CNAB, and other types. We can have a bigger picture, for skopeo. Users (like me) do expect that skopeo can work with OCI-based artifacts.

[vrothberg]

Can you share an example along with an error/failure of Skopeo?

[thaney071]

I don't have a direct example with skopeo. I am using the image.Copy from containers/image repos that skopeo uses. The oci copy only support OCI layouts with a mediaType with OCI. The problem is OCI layout can contain other mediType images such as Docker or CNAB.

Our team is developing a CNAB client and we are looking for a way to copy images in an OCI layout into a registry, but since the images in the OCI layout can be Docker or OCI or others, the copy method fails to find the details in the index.json.

I tracked it down to this https://github.com/containers/image/blob/master/oci/layout/oci_transport.go#L180
Where given an OCI layout, unless the media type matches, it fails to copy the image.

I can understand not wanting to support every media type, but would be nice to allow custom media type drivers to allow copies of others in OCI layout. It would then be up to the user of the copy to implement the needed steps to extract and copy non OCI type images from the layout.

[mtrmac]

I don’t think removing that check can work. The c/image code must understand the format of the “manifest” (or its equivalent in the format) just to be able to enumerate the config+blobs to be copied. It can’t very well interpret arbitrary JSON without a specification of that JSON.

(I don’t really follow OCI artifacts), but I understand those are represented as ordinary OCI-standard manifests, with a non-container-image config.json MIME type. That might not currently work with c/image because we assume OCI-manifested images have OCI configs, but it should be fairly easy to support.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[dmesser]

@mtrmac @rhatdan I think generic copying of OCI artifacts should be supported by skopeo. In Quay, we rely on skopeo to implement repository mirroring. There is a general trends towards storing non-image OCI artifacts in a registry, e.g. helm charts [1], ML models [2] or operator bundles [3]. We'd like to be able to continue to use skopeo for mirroring those kind of artifacts. Projects like ORAS [4] also propose to widen the use of the OCI artifact spec to store any kind of blob on an OCI compatible registry.

[1] https://cloud.redhat.com/blog/quay-oci-artifact-support-for-helm-charts
[2] https://goharbor.io/blog/harbor-extending-its-reach/
[3] https://olm.operatorframework.io/docs/tasks/creating-operator-bundle/
[4] https://oras.land/

[nevermore-muyi]

Support other OCI types are very useful.

[bt-lemery]

A Harbor user here who would also find this very useful.

[mtrmac]

With #1574 , c/image should now support copying OCI artifacts. Some operations, like inspecting an image or format conversions, are rejected.

Note that the only specific example of a failure in this issue, #1370 (comment) , was referring to a non-OCI manifest/index type. Those objects are not OCI artifacts, and are not enabled by #1574 .