Replies: 2 comments 5 replies
-
|
`docker version Server: Docker Engine - Community |
Beta Was this translation helpful? Give feedback.
-
|
When docker pulls an image, it gets the image-manifest from the registry; that manifest contains the list of layers to pull. From your output, it looks like you created an image ( |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
Oh! I missed that you were referring to the last layer, and thought you looked at the Wondering if this is the same reason under the hood as docker/cli#4607 (comment) (if the layer was built with the same content, but the compressed layer has a different digest). I'd have to check though which digest we're showing in the progress output (and which digest is checked against to show "already exists"). |
Beta Was this translation helpful? Give feedback.
-
|
docker info is： docker info
Client: Docker Engine - Community
Version: 24.0.6
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.11.2
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.21.0
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 6
Running: 3
Paused: 0
Stopped: 3
Images: 25
Server Version: 24.0.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2 nvidia
Default Runtime: runc
Init Binary: docker-init
containerd version: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
runc version: v1.1.9-0-gccaecfc
init version: de40ad0
Security Options:
seccomp
Profile: builtin
Kernel Version: 5.15.90.1-microsoft-standard-WSL2
Operating System: Ubuntu 22.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 12
Total Memory: 15.59GiB
Name: jammy
ID: ca1f2a07-18a4-4e09-a1e2-ff722d53f7ac
Docker Root Dir: /var/lib/docker
Debug Mode: true
File Descriptors: 49
Goroutines: 74
System Time: 2023-10-17T21:08:22.500551327+08:00
EventsListeners: 1
Experimental: true
Insecure Registries:
127.0.0.1:5000
localhost:5000
127.0.0.0/8
Live Restore Enabled: false
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device supportbuildx version is: $ docker buildx version
github.com/docker/buildx v0.11.2 9872040and I create a builder instance for buidx： docker buildx inspect container
Name: container
Driver: docker-container
Last Activity: 2023-10-17 12:27:19 +0000 UTC
Nodes:
Name: container0
Endpoint: unix:///var/run/docker.sock
Driver Options: network="host"
Status: running
Flags: security.insecure network.host
Buildkit: v0.12.2
Platforms: linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386
Labels:
org.mobyproject.buildkit.worker.executor: oci
org.mobyproject.buildkit.worker.hostname: d9e4aa969fe1
org.mobyproject.buildkit.worker.network: host
org.mobyproject.buildkit.worker.oci.process-mode: sandbox
org.mobyproject.buildkit.worker.selinux.enabled: false
org.mobyproject.buildkit.worker.snapshotter: overlayfs
GC Policy rule#0:
All: false
Filters: type==source.local,type==exec.cachemount,type==source.git.checkout
Keep Duration: 48h0m0s
Keep Bytes: 488.3MiB
GC Policy rule#1:
All: false
Keep Duration: 1440h0m0s
Keep Bytes: 94.06GiB
GC Policy rule#2:
All: false
Keep Bytes: 94.06GiB
GC Policy rule#3:
All: true
Keep Bytes: 94.06GiBI can reproduce this phenomenon by following the steps. step 1: start a local registry: step 2: Prepare files for building the image: create a Dockerfile with following content: # syntax=docker/dockerfile:1.6
FROM ubuntu:22.04
WORKDIR /opt/projects/xmov
ADD . /opt/projects/xmov/
COPY --link --from=model_dir . ./assert/
CMD [ "bash"]step 3: build an Image with docker buildx: $ docker buildx build --builder container --load --build-context model_dir=/home/bar/ . -t localhost:5000/test:v1
[+] Building 18.0s (16/16) FINISHED docker-container:container
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 199B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1.6 3.1s
=> CACHED docker-image://docker.io/docker/dockerfile:1.6@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021 0.0s
=> => resolve docker.io/docker/dockerfile:1.6@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:22.04 1.6s
=> [context model_dir] load .dockerignore 0.0s
=> => transferring model_dir: 2B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [stage-0 1/4] FROM docker.io/library/ubuntu:22.04@sha256:2b7412e6465c3c7fc5bb21d3e6f1917c167358449fecac8176c6e496e5c1f05f 12.1s
=> => resolve docker.io/library/ubuntu:22.04@sha256:2b7412e6465c3c7fc5bb21d3e6f1917c167358449fecac8176c6e496e5c1f05f 0.0s
=> => sha256:aece8493d3972efa43bfd4ee3cdba659c0f787f8f59c82fb3e48c87cbb22a12e 29.54MB / 29.54MB 11.3s
=> => extracting sha256:aece8493d3972efa43bfd4ee3cdba659c0f787f8f59c82fb3e48c87cbb22a12e 0.7s
=> [context model_dir] load from client 0.0s
=> => transferring model_dir: 38B 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 239B 0.0s
=> [stage-0 2/4] WORKDIR /opt/projects/xmov 0.4s
=> [stage-0 3/4] ADD . /opt/projects/xmov/ 0.0s
=> [stage-0 4/4] COPY --link --from=model_dir . ./assert/ 0.0s
=> exporting to docker image format 0.3s
=> => exporting layers 0.1s
=> => exporting manifest sha256:b1e2a8058fa7898e598ed747d5118582610bc639f1fa6db9790f1c54cfc10655 0.0s
=> => exporting config sha256:64c84bf0fb486bb0f7c915a6de975dcc09fa21a37f8f86cc2b014dddc6350bb6 0.0s
=> => sending tarball 0.2s
=> importing to docker step 4: update foo.txt and build image again $ echo "foo update" > ./foo.txt
$ docker buildx build --builder container --load --build-context model_dir=/home/chao/bar/ . -t localhost:5000/test:v1
[+] Building 1.4s (15/15) FINISHED docker-container:container
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 199B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1.6 0.7s
=> CACHED docker-image://docker.io/docker/dockerfile:1.6@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021 0.0s
=> => resolve docker.io/docker/dockerfile:1.6@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:22.04 0.3s
=> [context model_dir] load .dockerignore 0.0s
=> => transferring model_dir: 2B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [stage-0 1/4] FROM docker.io/library/ubuntu:22.04@sha256:2b7412e6465c3c7fc5bb21d3e6f1917c167358449fecac8176c6e496e5c1f05f 0.0s
=> => resolve docker.io/library/ubuntu:22.04@sha256:2b7412e6465c3c7fc5bb21d3e6f1917c167358449fecac8176c6e496e5c1f05f 0.0s
=> [context model_dir] load from client 0.0s
=> => transferring model_dir: 28B 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 246B 0.0s
=> CACHED [stage-0 2/4] WORKDIR /opt/projects/xmov 0.0s
=> [stage-0 3/4] ADD . /opt/projects/xmov/ 0.0s
=> [stage-0 4/4] COPY --link --from=model_dir . ./assert/
=> exporting to docker image format 0.2s
=> => exporting layers 0.0s
=> => exporting manifest sha256:dc1af2160c3e017cdd98daac91a58a9a02478e756aa25fa5cfebd649e29b70d9 0.0s
=> => exporting config sha256:f1b9344f0847cdbe1b526e9697f4da015543fdd771d330980060b5759922e60a 0.0s
=> => sending tarball 0.2s
=> importing to docker 0.0sstep 5: push images to local registry and delete them from docker: $docker push localhost:5000/test:v1
The push refers to repository [localhost:5000/test]
cf817db22221: Pushed
7df667bfe747: Pushed
9fcbbf5a4318: Pushed
256d88da4185: Pushed
$ docker push localhost:5000/test:v2
The push refers to repository [localhost:5000/test]
cf817db22221: Layer already exists
8ccbaba0e7b7: Pushed
9fcbbf5a4318: Layer already exists
256d88da4185: Layer already exists
v2: digest: sha256:2a56af442e17de502b484046a17b0b0865877210d6dc11708e29153ab179ca22 size: 1150
$ docker image rm localhost:5000/test:v1
Untagged: localhost:5000/test:v1
Untagged: localhost:5000/test@sha256:c55434e21eeda0fbf7068e3e83d02600f74f09472691128b168538449550cb78
Deleted: sha256:64c84bf0fb486bb0f7c915a6de975dcc09fa21a37f8f86cc2b014dddc6350bb6
Deleted: sha256:f14b58b47fdeef3556e6887e77f853a3ba7703c500b311e1944dcd939e27bf85
Deleted: sha256:6cd1b47a28b1291947956a75a048029a9abf945e4c085d628e4af54220bb0d7b
$ docker image rm localhost:5000/test:v2
Untagged: localhost:5000/test:v2
Untagged: localhost:5000/test@sha256:2a56af442e17de502b484046a17b0b0865877210d6dc11708e29153ab179ca22
Deleted: sha256:f1b9344f0847cdbe1b526e9697f4da015543fdd771d330980060b5759922e60a
Deleted: sha256:32a8c2cee479289967afa1840074913bbe8960fc056bab19a5f9059ea4135c3f
Deleted: sha256:5d6682eac21a10659f5b9b367afc6792cdd7dfc4e182ffb1504abd743201ba27
Deleted: sha256:8c57c2108f94597777d223cf7af98e139fc2edeba444605f38c5a382239ab455step 6: pull images from local registry: $ docker image pull localhost:5000/test:v1
v1: Pulling from test
43f89b94cd7d: Already exists
3a824c8b6149: Pull complete
afb2cc22459e: Pull complete
3d1709da15fc: Pull complete
Digest: sha256:c55434e21eeda0fbf7068e3e83d02600f74f09472691128b168538449550cb78
Status: Downloaded newer image for localhost:5000/test:v1
localhost:5000/test:v1Note: The output information shows that the hash value of the last layer is 3d1709da15fc docker image pull localhost:5000/test:v2
v2: Pulling from test
43f89b94cd7d: Already exists
3a824c8b6149: Already exists
efc33777b4ad: Pull complete
3d1709da15fc: Pull complete
Digest: sha256:2a56af442e17de502b484046a17b0b0865877210d6dc11708e29153ab179ca22
Status: Downloaded newer image for localhost:5000/test:v2
localhost:5000/test:v2Note: the output information show that: the last layer 3d1709da15fc has been pulled again |
Beta Was this translation helpful? Give feedback.
-
|
Also worth noting that if a download is aborted after some layers are downloaded but not others, the ones that have been downloaded already are discarded, they aren't saved. So in this sense you can't resume a download, if you are have downloaded 99 of 100 layers and you cancel, you'll have to download all 100 again. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
fast docker pull:
`$ docker pull localhost:5000/aigqa:test
test: Pulling from aigqa
0a9573503463: Pull complete
1ccc26d841b4: Pull complete
800d84653581: Pull complete
7c632e57ea62: Pull complete
f9a1922eee8a: Pull complete
ebb263a37449: Pull complete
46a466baf778: Pull complete
5bb253f012e0: Pull complete
5ae054a6d508: Pull complete
4c4ce67f32ab: Pull complete
44b56b9646bc: Pull complete
7bacbe176e76: Pull complete
fe7e125d8b2f: Pull complete
b947d8e1c8fa: Pull complete
20e5ea4fc7e2: Pull complete
287367d67a3a: Pull complete
f436538f8599: Pull complete
e0e1d9c4824d: Pull complete
Digest: sha256:dbdb6858f523945acc6b857a7839ff8697b96389106d52ca3a5242920008982d
Status: Downloaded newer image for localhost:5000/aigqa:test
localhost:5000/aigqa:test`
second pull:
`docker pull localhost:5000/aigqa:test1
test1: Pulling from aigqa
0a9573503463: Already exists
1ccc26d841b4: Already exists
800d84653581: Already exists
7c632e57ea62: Already exists
f9a1922eee8a: Already exists
ebb263a37449: Already exists
46a466baf778: Already exists
5bb253f012e0: Already exists
5ae054a6d508: Already exists
4c4ce67f32ab: Already exists
44b56b9646bc: Already exists
7bacbe176e76: Already exists
fe7e125d8b2f: Already exists
b947d8e1c8fa: Already exists
20e5ea4fc7e2: Already exists
287367d67a3a: Already exists
e58dfe837108: Pull complete
e0e1d9c4824d: Downloading [=========================> ] 1.59GB/3.115GB`
Why docker download layer e0e1d9c4824d again?
Beta Was this translation helpful? Give feedback.
All reactions