Upgrade to hyper-rustls 0.27/rustls 0.23

[djc]

Sticks with ring as the default crypto provider for rustls for now (see #2136).

This is also a prerequisite for re-enabling a Quinn-based H3 backend, since Quinn is skipping rustls 0.22.

[seanmonstar]

Thanks so much! I do have some questions, to help plan how we could allow using the different crypto providers.

• If no provider is enabled, what happens? I tried removing "ring" from the dependencies in this PR locally, and it still builds...
  • Oh I see, I just ran the simple example, it panics at runtime.
• I suppose we could introduce rustls-tls-ring and rustls-tls-awslc features eventually. We don't need to do so yet, but is there anything to take into consideration?

[djc]

Thanks so much! I do have some questions, to help plan how we could allow using the different crypto providers.

• If no provider is enabled, what happens? I tried removing "ring" from the dependencies in this PR locally, and it still builds...

  • Oh I see, I just ran the simple example, it panics at runtime.

Yes. To be safe at compile time, we could use the builder_with_provider() API and pass rustls::crypto::ring::default_provider() explicitly instead.

• I suppose we could introduce rustls-tls-ring and rustls-tls-awslc features eventually. We don't need to do so yet, but is there anything to take into consideration?

Those features would be non-additive, because rustls will panic if both are enabled. Also note that people might want to use a third-party crypto provider -- this is of course already possible via the preconfigured_tls() path.

[seanmonstar]

Those features would be non-additive, because rustls will panic if both are enabled.

Oh, so reqwest would be picking the crypto provider for the application as a whole? If any other library depends on rustls and doesn't disable the default features, will the application start to see panics when they upgrade reqwest?

[djc]

The ClientConfig::builder() indirectly calls into here:

https://github.com/rustls/rustls/blob/main/rustls/src/crypto/mod.rs#L254

This will:

1. If a process-wide crypto provider has already been installed, use it
2. If no provider has been installed and only one has been enabled via Cargo features, use it and install it as the default
3. Otherwise, panic

As such, I think the idea is that reqwest should rely on this and if people run into a run-time panic they should call CryptoProvider::install_default() first before setting up reqwest's Client/ClientBuilder.

See more discussion in rustls/rustls#1877.

[NobodyXu]

Won't it be more reasonable, to prefer one crypto to another instead of panicking at runtime?

Or maybe a compile_error! is also better than runtime panic.

[djc]

Feel free to contribute to the discussion in the upstream issue.

[stormshield-gt]

I would like to suggest a change to be able to setup a custom provider without having to compile both ring and the custom provider.

• do not activate the ring feature on hyper-rustls, rustls and tokio-rustls
• activate the ring feature only if rustls-tls feature is enabled. rustls-tls = ["rustls-tls-webpki-roots", "rustls/ring"]
• optionally, add another feature that uses rustls but don't activate the ring feature. rustls-tls-providerless = ["rustls-tls-webpki-roots"]. This is optional as users could use the rustls-tls-webpki-roots directly

[djc]

FWIW, this PR was just intended to do the rustls upgrade while making as few other changes as possible. Obviously there could be a number of new Cargo features that help control the crypto provider used by rustls. But I think reqwest will want to pick a default crypto provider to make adopting rustls as easy as possible.

[stormshield-gt]

My suggested changes still pick up ring as a default provider without adding any new feature, and leaving the possibility to control the crypto provider used by rustls. That's why I think it's interesting to consider this now. This can also be, of course, in a follow-up PR as you suggest.

[seanmonstar]

I do want this to merge. As to the status of this upgrade: I'm hesitant to merge and publish since anyone upgrading that was already using rustls v0.23 may suddenly start seeing a panic, because reqwest enables the alternative backend. (And I don't really want to "fix" that by switching to aws-lc.)

My feeling is that I'd want to wait for a rustls version that doesn't have that issue. Discussion is in rustls/rustls#1877.

[djc]

@seanmonstar I think we can work around that issue by using ClientConfig::builder_with_provider(), explicitly passing whichever provider we have selected into the config builder explicitly. This should never panic, independent of which Cargo features other libraries have selected. Does that sound like a suitable mechanism for now?

I feel like a solution to rustls/rustls#1877 is unlikely to happen soon.