Cargo publish times out without failure

[tbrezot]

Problem

When trying to publish a crate, we recently experienced an issue reported here. The command cargo publish timed out without returning an error.

Steps

This may be not repeatable without hacking the cargo publish network communication. Our guess is that the crates.io server is currently experiencing some issue.

There is the GitHub CI recipe that failed (problem occurs on the cargo publish --token $CRATES_IO part):

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - name: Dump context for debug
        uses: crazy-max/ghaction-dump-context@v1

      - uses: actions/checkout@v1

      - name: Publishing - dry run
        if: inputs.publish == false
        run: |
          cargo publish --dry-run
      - name: Publishing
        if: inputs.publish == true
        run: |
          echo "[registry]" > ~/.cargo/credentials
          echo "token = \"$CRATES_IO\"" >> ~/.cargo/credentials
          cargo publish --token $CRATES_IO
        env:
          CRATES_IO: ${{ secrets.CRATES_IO }}

      - name: Release
        if: inputs.publish == true
        uses: softprops/action-gh-release@v1

Below is the screenshot of a local attempt to run cargo publish --allow-dirty.

Possible Solution(s)

Return with an error upon time out.

Notes

No response

Version

cargo 1.68.0-nightly (985d561f0 2023-01-20)
release: 1.68.0-nightly
commit-hash: 985d561f0bb9b76ec043a2b12511790ec7a2b954
commit-date: 2023-01-20
host: x86_64-unknown-linux-gnu
libgit2: 1.5.1 (sys:0.16.1 vendored)
libcurl: 7.86.0-DEV (sys:0.4.59+curl-7.86.0 vendored ssl:OpenSSL/1.1.1q)
os: Ubuntu 22.10 (kinetic) [64-bit]

[tbrezot]

#11222

[Turbo87]

somewhat off-topic, but if you use something like https://github.com/glide-rs/flarmnet-rs/blob/1aa7363ace21096fa5fd3107207d0c03c0be984f/.github/workflows/release.yml#L17-L19 you don't have to use --token :)

[epage]

Huh, I didn't document this behavior in #11062.

This was an intentional choice. We were transitioning from no timeout to a timeout and were concerned about the impact for existing users (ie not dependent on blocking, single crate) when sporadic, issues like Github being down, occur and (1) didn't want the timeout to be too long and (2) didn't want it to be an error when it wasn't before.

Now, whether we have a path to later turning this into an error is a different question. We were especially worried about the impact of this behavior change on alternative registeries and had #11222 as an escape hatch but we haven't yet seen a need or even interest in it or problem with blocking.

[weihanglo]

The situation has been improved since then. We have more HTTP info and a better block-wait notice when publishing.

• Add more information to wait-for-publish #11713
• Add more information to HTTP errors to help with debugging. #11878
• Publish docs: Clarify requirements about the state of the index after publish. #11926

To resolve this issue, the missing piece should be adding a simple sentence mentioning in cargo-publish man page, such as a simplified version of what #11926 did.

[epage]

@weihanglo while documenting helps, I think the core of this issue is about making it an error or am I missing something?

[weihanglo]

@weihanglo while documenting helps, I think the core of this issue is about making it an error or am I missing something?

I was thinking that it may break somebody's workflow. But since it already timed out, turning this into a hard error makes more sense than no error. What do people think?

[hi-rustin]

What do people think?

It makes more sense to me.

[epage]

More so I meant that we need to make sure we decide on the merit and timing of a hard error for closing this rather than closing this for documentation.

[after-ephemera]

I am admittedly not familiar with all of the implications of erroring out in this situation, but it seems appropriate given that the command's explicit intent (publish the crate) didn't succeed. I can't think of a use case where I would issue the publish command, manually or in automation, and find a failure acceptable. @epage I'd love your thoughts before digging deeper into this.

[epage]

@after-ephemera the publish endpoint reported success. However, the publish process is asynchronous to that. So the reasons someone will see a timeout are (1) there is some kind of service outage blip or (2) the rest of the publish pipeline failed. I'm assuming a failure in the publish pipeline would generally be considered a bug, be rare,and be actively monitored for by the crates.io team. I and others have seen service blips.

So timeout, doesn't mean the publish failed and more than likely it succeeded.

[after-ephemera]

Ah so there's a bit more nuance here, thank you for explaining it. When you say "service blips" are you referring to network connectivity issues that are intermittent? If so, I would venture a guess that it's fine to leave as is since either that or pipeline issues are expected to be rare and intermittent.

[epage]

Basically, yes.