Has something changed with github / api for creating releases?

[userdocs]

I am having a lot of problems with release assets being uploaded. I have tried other release actions so it's not a bug here but i feel something has changed because actions that were working fine before now have a consistent problem uploading assets

Same job run twice:

issue: https://github.com/userdocs/musl-cross-make/actions/runs/1403702718

no issues: https://github.com/userdocs/musl-cross-make/actions/runs/1404604922

And this job that was working fine until about a few days ago where i started to notice issues.

https://github.com/userdocs/test/actions/runs/1408303173

I feel like something with how the action are run or matrix builds maybe has changed on GitHub side and it's causing this.

Do you have any ideas what it may be?

[ncipollo]

From googling around it seems like this can happen when there is an intermittent outage on github. We do retry a few times if the artifact fails to upload but I'm not sure that would always help if the outage lasted awhile.

[userdocs]

So for me this started about 3 days ago and I hoped it was a transient issue but it seems consistent and is happening all the time. I have done hundreds of builds and it's not going away. I also tried https://github.com/softprops/action-gh-release and https://github.com/svenstaro/upload-release-action which just hard error on the ECONNRESET .

For example this action seem to not have an issue https://github.com/userdocs/qbittorrent_crossbuild/actions/runs/1405492687 but this super simple action is https://github.com/userdocs/musl-cross-make/actions/runs/1408568446/workflow

I thought I might be hitting limits or something but all actions are in public repos and i have NO listed quotas exceeded

I looked here to see if they changed anything but nothing stands out. https://github.blog/changelog/

I cannot see any cause

https://www.githubstatus.com/incidents/m78sypncxsdw
https://github.blog/changelog/
https://status.dev.azure.com/_history (i think they use azure?)

[userdocs]

@ncipollo I don't fully understand why this works but I can only assume either.

• GitHub is doing some weird new stuff in the runners (like auto caching?) - some kind of resource/performance management - maybe.

• The Alpine container or docker implementation or both are buggy when used in this way.

• Possibly that I also used qemu static on the host (and update the host runner first) in my other method, as one of the two main differences. I doubt this as I had issues on all matrix arch build outcomes and the host is always amd64

I also don't think I'm imagining this problem as I did a lot of builds recently as well as over the last few months and these actions were working fine until very recently so I'm sure something has changed, and it was not my setup (or your action). I can only guess at what that is.

One thing of interest I noticed was the failure for release artifacts (that actually uploaded) to be placed in alphabetical order when I was having issues. Whether this is a symptom or a coincidence I cannot say but mentioned it in case it means something to you.

Here is a brief summary of what did not work and what changes I made to fix a previously working setup in my matrix builds.

So, I was doing this kind of docker setup, which kept giving me errors

jobs:
  build:
    runs-on: ubuntu-latest
    container: alpine:edge
    ...
    steps:
  ...
    

After looking at one of my other successful actions I noticed that the main difference was how I implemented the docker container. There I bootstrap a container myself in a way the previous method cannot allow for. This is done using the docker installed on the runner as a step, like this:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    ...
    - name: Host - Create docker multiarch container
    run: docker run --name multiarch -it -d --env-file env.custom -w /root -v ${{ github.workspace }}:/root ${{ matrix.os_id }}:${{ matrix.os_version_id }}

This sort of changes how the action is structured but it now it works again as expected with no errors

So I do not know what changed to make the consistent problem go away, consistently, or why this difference fixes it but I am certain something was changed.

Unless you can see something I don't that would explain the issue? For now I have moved past this issue with a lot of confusion but minor adaption to my workflows so I leave it here in case someone else hits this weird issue for my main project having this issue

bugged - https://github.com/userdocs/qbittorrent-nox-static/actions/runs/1406134278

fixed - https://github.com/userdocs/qbittorrent-nox-static/actions/runs/1410538865

[ncipollo]

Hmm, that's strange. I just re-ran the actions in my test repo and it looks like everything uploaded without error: https://github.com/ncipollo/actions-playground/actions/runs/1412141034 (that one warning about the directory is expected). I typically just use the default ubuntu container so maybe there is something up with that alpine image you were using.

[userdocs]

It appears the image is the same in either method.

Failed

/usr/bin/docker pull alpine:edge
edge: Pulling from library/alpine
41dcc117e123: Pulling fs layer
41dcc117e123: Verifying Checksum
41dcc117e123: Download complete
41dcc117e123: Pull complete
Digest: sha256:e64a0b2fc7ff870c2b22506009288daa5134da2b8c541440694b629fc22d792e
Status: Downloaded newer image for alpine:edge
docker.io/library/alpine:edge

Successful

Unable to find image 'alpine:edge' locally
edge: Pulling from library/alpine
41dcc117e123: Pulling fs layer
41dcc117e123: Verifying Checksum
41dcc117e123: Download complete
41dcc117e123: Pull complete
Digest: sha256:e64a0b2fc7ff870c2b22506009288daa5134da2b8c541440694b629fc22d792e
Status: Downloaded newer image for alpine:edge

Unless somewhere here the issue is created

docker.io/library/alpine:edge
/usr/bin/docker create --name 54690e3c3fdf49baa70ab559b1c21900_alpineedge_4cd4b3 --label f88420 --workdir /__w/musl-cross-make/musl-cross-make --network github_network_b189718538124bffb164d0160d2f88d5  -e "HOME=/github/home" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work":"/__w" -v "/home/runner/runners/2.283.3/externals":"/__e":ro -v "/home/runner/work/_temp":"/__w/_temp" -v "/home/runner/work/_actions":"/__w/_actions" -v "/opt/hostedtoolcache":"/__t" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" --entrypoint "tail" alpine:edge "-f" "/dev/null"
632fc961a19aa9608ab76884863b54fb9254265dd338445405c917be7a34d2b1
/usr/bin/docker start 632fc961a19aa9608ab76884863b54fb9254265dd338445405c917be7a34d2b1
632fc961a19aa9608ab76884863b54fb9254265dd338445405c917be7a34d2b1
/usr/bin/docker ps --all --filter id=632fc961a19aa9608ab76884863b54fb9254265dd338445405c917be7a34d2b1 --filter status=running --no-trunc --format "{{.ID}} {{.Status}}"
632fc961a19aa9608ab76884863b54fb9254265dd338445405c917be7a34d2b1 Up Less than a second
/usr/bin/docker inspect --format "{{range .Config.Env}}{{println .}}{{end}}" 632fc961a19aa9608ab76884863b54fb9254265dd338445405c917be7a34d2b1
HOME=/github/home
GITHUB_ACTIONS=true
CI=true
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin