Issue with push to GHCR - buildx failed with: error: failed to solve: failed to copy: cannot reuse body, request must be retried #626
Labels
Comments
|
Looking at your workflow and previous builds, it seems to happen since May 13, 2022 without relevant changes in your Dockerfile and using the same version of buildx and BuildKit so I would say it's related to the GHCR registry: https://github.com/SAP-samples/btp-setup-automator/actions/workflows/docker-build-and-push.yml?page=2&query=event%3Aworkflow_dispatch
BuildKit logs |
|
I strongly suspect it is a GHCR issue at this point as I am able to push to Docker Hub consistently with no issues and only encounter this with GHCR. It does seem quite random as previously working jobs will stop working and then resolve itself a day or so later. Not quite sure which repo to open an issue against for GHCR though. Any suggestions @crazy-max ? |
crazy-max
changed the title
|
agree that the action was always a bit shaky against GHCR, but things got worse over the recent weeks (from "time to time it fails" towards "time to time it succeeds"). There was no official change mentioned on the GitHub release notes, which does not mean that there might be changes under the hood. Is there an option to get more output from the push step that currently just returns "cannot reuse body, request must be retried" ? |
We have sent a message to the GitHub team about this issue. Will keep you in touch. |
|
Got a bit more detail on one of my failed jobs. Unclear as to why the connection is being reset. |
|
I put a thread here on the discourse, hoping it's a better way to make it googleable and to provide evidence it's not just us. @jauderho @lechnerc77 if you'd like to reply or +1 it that might help? |
|
@crazy-max I'm a contributor to an OpenSource project. We have also been experiencing this issue since May 13th. It has got worse in the recent days. We eliminated the build-push-action and are still experiencing the same issue. The only docker actions we are using now are "setup-qemu-action" and "setup-buildx-action". The only thing I can find online points to an issue with containerd. I'll keep everyone updated if I find a solution. |
|
@mrflip Thanks a lot - added a comment (currently in moderation queue) |
|
Hi from ghcr team, |
|
@kaminipandey thanks for coming back. We just gave it a try (https://github.com/SAP-samples/btp-setup-automator/runs/6727577206?check_suite_focus=true) and the run was successful (which did not happen since a long time :-) ) By coincidence there was another run 3 hours ago that also was successful (which would fit to the time of your comment). We will keep an eye on upcoming runs, but that looks promising! |
|
@kaminipandey I can confirm that a round of builds that I kicked off after you posted have completed without issue. I was hoping that this would also sort out a separate Rust non-x86 out of mem build issue too but alas not. Is there a location that I can report this/create an issue so that it can be looked into? Thanks. |
|
@jauderho This doesn’t look related to ghcr. You can raise a support ticket for the actions team. Thanks |
|
From the comments it looks solved now. Thanks @kaminipandey |
|
@kaminipandey Are you able to share more on what the cause of the issue was? It would be helpful to understand so that we will know accordingly if this happens again in the future. Thanks. |
|
By the way, this still happens every now and then. Here's two examples:
I'm also doing this for both GHCR and Docker Hub, yet it only happens with the former. I do get a feeling that it might be an issue with availability, in which case perhaps it would be worth it to add a retry functionality to this action? Considering the build time usually takes significantly longer than the push time, it is a little unfortunate that I have to go through the build process again every time the push fails. |
|
It's still happening for us as well. Last week was really rough, the nightly job failed 4 out of 7 days. See: https://github.com/oddbird/odddocker/actions/runs/3927636198/jobs/6714427306
|
This comment was marked as resolved.
This comment was marked as resolved.
|
Unfortunately, even |
|
Same as #761 (comment) I'm locking this issue. Feel free to open a new one with BuildKit debug logs if it still occurs, thanks! |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Troubleshooting
Before submitting a bug report please read the Troubleshooting doc.
Behaviour
Steps to reproduce this issue
GitHub status was green when executing the run
Expected behaviour
The build and push should push the new image(s) to the GitHub registry
Actual behaviour
This is also reported in #452 which was closed referencing the GH issue this morning. However the error also occurs when the GH packages are in operational state. As proposed here is a new issue for the same error with the logs attached
Configuration
Logs
Attached:
log_build_and_push.txt
The text was updated successfully, but these errors were encountered: