New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Generic OCI artifact transport? #1150
Comments
Thanks for reaching out! Did you try copying an artifact that failed? |
Hi Valentin, thank you for your reply. The artifacts that I am talking about are not (OCI or Docker) images, but configuration stuff that adheres to the Artifact Authors Guidance. When I try to inspect such an artifact, I get an $ skopeo inspect docker://eu.gcr.io/gardener-project/cc/cnudie-test-3/component-descriptors/github.com/gardener/cc-utils@sha256:95df958f73f74e81e14ce83d199e4f3a29dd2957e832c7971a28b302c0b595f4
time="2021-02-17T12:32:35+01:00" level=fatal msg="Error parsing manifest for image: unsupported docker v2s2 media type: \"application/vnd.oci.gardener.cloud.cnudie.component-descriptor.config.v2+yaml+tar\"" A sample manifest is $ curl -k -v https://eu.gcr.io/v2/gardener-project/cc/cnudie-test-3/component-descriptors/github.com/gardener/cc-utils/manifests/sha256:95df958f73f74e81e14ce83d199e4f3a29dd2957e832c7971a28b302c0b595f4
> GET /v2/gardener-project/cc/cnudie-test-3/component-descriptors/github.com/gardener/cc-utils/manifests/sha256:95df958f73f74e81e14ce83d199e4f3a29dd2957e832c7971a28b302c0b595f4 HTTP/2
> Host: eu.gcr.io
> user-agent: curl/7.75.0
> accept: */*
>
< HTTP/2 200
< docker-distribution-api-version: registry/2.0
< content-type: application/vnd.docker.distribution.manifest.v2+json
< docker-content-digest: sha256:95df958f73f74e81e14ce83d199e4f3a29dd2957e832c7971a28b302c0b595f4
< content-length: 513
< date: Wed, 17 Feb 2021 10:52:44 GMT
< server: Docker Registry
<
{
"config": {
"digest": "sha256:e9615b60fb0495d914d3e478e8714557fc372852baebf894b020e47d72a85471",
"size": 227,
"mediaType": "application/vnd.oci.gardener.cloud.cnudie.component-descriptor-metadata.config.v2+json"
},
"layers": [
{
"digest": "sha256:6c08b425efa1be28a3c6223bb937eeeb1dc5886ac2190e994f705f31ef1ce801",
"size": 1536,
"mediaType": "application/vnd.oci.gardener.cloud.cnudie.component-descriptor.config.v2+yaml+tar"
}
],
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"schemaVersion": 2
} It seems that the non-Docker values in the |
Thanks for sharing the details, @cfiderer. I am aware of OCI Artifacts. In a sense they are OCI images but special since they exploit the "extensibility clause" which states that unknown properties must be ignored. That being said, Artifacts must be images with a valid OCI manifest. The media types of configs and layers, however, can differ. Looking at the upper manifest, I see
It's a Docker v2s2 image but Artifacts are limited to OCI. So the image is neither a valid Docker v2s2 image, nor a valid OCI artifact. I think containers/image is right to reject it since the config and the layers do not conform with the Docker v2s2 spec:
That doesn't necessarily mean that we cannot change the behavior. Compatibility with other tools is quite often a good reason to relax some checks. Do Docker and containerd eat the image? |
I doubt that Several registries (e.g. GCR, JFrog Artifactory) however, accept the artifact (store & retrieve). Our intent is to have an agnostic copy tool that processes the manifest and passes it and the layers unmodified. |
Yes. Currently the image claims to be a Docker v2s2 image but it should be an ordinary OCI image just with the specific config and layer media types.
I have mixed feelings since the image doesn't comply with Docker v2s2. However, the registry seems to happily serve the image which suggests that other tools are more relaxed. An example OCI manifest looks as follows: {
"schemaVersion": 2,
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json",
"digest": "sha256:e2a8572846157b37916ac1c9141b618998dc08b1808c2af4a4931adc479d5922",
"size": 584
},
"layers": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:4c0d98bf9879488e0407f897d9dd4bf758555a78e39675e72b5124ccf12c2580",
"size": 2811321
}
]
} An OCI Artifact is conceptually the same but with a different config media type and (optionally) custom layers. |
Hmmm... Debugging through the code, I get the impression that the value from the Assuming I have no chance to control the |
I am not sure to understand the question correctly. Can you elaborate? The underlying problem is that the image is build incorrectly. Docker v2s2 images don't support artifacts. Only images in the OCI format support artifacts. Docker is also rejecting the image but Docker is supporting artifacts (as containers/image does). The only way I see to fix the issue is to replace the image with a new one in the OCI format. I am quite surprised that Note that
|
OK, we created another test image with correct? mediaTypes:
But when trying to copy this artifact to a local Docker registry,
How can I prevent this on-the-fly compression for unknown MIME types? |
Excellent! Yes, that image is almost correct. The media type of the layer must be changed. The |
Oh, there's actually another issue:
You need to remove this line ^^^ |
When reading the OCI Artifacts spec, it refers to the OCI Image Manifest spec, which in turn defines the Observing that the control over the I am still struggling to fully understand how a non-image artifact can be correctly handled using the OCI specs. As far as I've observed, I am lost once a But when registries are free to provide my manifest with whatever |
I don't think that registries are free to do that. The manifest still "looked" like a Docker one as it was explicitly declaring it's manifest media type. Since OCI manifests don't do that, I assume that registry may just have "guessed" the type. An OCI image will always be served and declared as an OCI image. Regarding OCI artifacts. Below is a {
"schemaVersion": 2,
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json",
"digest": "sha256:d2248bd8b5884b9653464153ea67d224b9b216a23b97b0fd8e7bf3c7a643ea02",
"size": 585
},
"layers": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:ba3557a56b150f9b813f9d02274d62914fd8fce120dd374d9ee17b87cf1d277d",
"size": 2811657
}
]
} The An OCI artifact looks similar to that but declares a different config media type and (optionally) different layer media types. The OCI spec states that "unknown media types" must be ignored which left enough space for artifacts. |
https://oras.land/ covers similar ground to the original issue description, as I understand it. Not the same thing but close. |
A few updates for the record:
|
OCI artifacts should now be supported with #1574 (but note that the originally-reported schema2 images with non-image layer MIME types, #1150 (comment) , will continue to be rejected). |
Hello all,
I need to transfer non-image artifacts (that adhere to the OCI Artifact spec) between repositories and the file system (repo -> repo, repo -> file system, file system -> repo), I thought about using the
containers/image
library.But unfortunately, the
docker://
prefix is the only prefix that I've found for repositories (on the network), and the code related to Docker repositories handles only Docker images (and refuses everything else).I'd like to ask:
Kind regards, Christian
The text was updated successfully, but these errors were encountered: