Fails behind corporate proxy (but plain cargo install works)

[arvid-norlander]

I'm behind a corporate transparent HTTPS proxy. The required SSL certificate has been installed in /etc/ca-certificates and works as expected for plain cargo install. However:

$ cargo install-update -a
    Polling registry 'https://index.crates.io/'................................

Package                Installed  Latest                               Needs update
cargo-binstall         v1.6.0     v1.6.1                               Yes
cargo-deny             v0.14.3    v0.14.10                             Yes
cargo-hack             v0.6.15    v0.6.16                              Yes
cargo-llvm-cov         v0.6.1     v0.6.3                               Yes
cargo-show-asm         v0.2.27    v0.2.29                              Yes
sccache                v0.7.4     v0.7.6                               Yes
cargo-about            v0.6.1     v0.6.1                               No
cargo-audit            v0.18.3    v0.18.3                              No
[...]

Updating cargo-binstall
 INFO resolve: Resolving package: 'cargo-binstall'
 INFO resolve:fetch_crate_matched{self=SparseRegistry { url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("index.crates.io")), port: None, path: "/", query: None, fragment: None }, dl_template: OnceCell { value: None } } client=Client(Inner { client: Client { accepts: Accepts { gzip: true, brotli: true, deflate: true }, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*", "user-agent": "cargo-binstall/1.6.0"} }, service: DelayRequest { inner: Mutex { data: Inner { client: Client { accepts: Accepts { gzip: true, brotli: true, deflate: true }, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*", "user-agent": "cargo-binstall/1.6.0"} }, num_request: 1, per: 10ms, until: Instant { tv_sec: 62059, tv_nsec: 701819054 }, state: Ready { rem: 1 } }, poisoned: false, .. }, hosts_to_delay: Mutex { data: {}, poisoned: false, .. } } }) crate_name="cargo-binstall" version_req=VersionReq { comparators: [Comparator { op: Exact, major: 1, minor: Some(6), patch: Some(1), pre: Prerelease("") }] }}:do_send_request{self=Client(Inner { client: Client { accepts: Accepts { gzip: true, brotli: true, deflate: true }, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*", "user-agent": "cargo-binstall/1.6.0"} }, service: DelayRequest { inner: Mutex { data: Inner { client: Client { accepts: Accepts { gzip: true, brotli: true, deflate: true }, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*", "user-agent": "cargo-binstall/1.6.0"} }, num_request: 1, per: 10ms, until: Instant { tv_sec: 62059, tv_nsec: 701819054 }, state: Ready { rem: 1 } }, poisoned: false, .. }, hosts_to_delay: Mutex { data: {}, poisoned: false, .. } } }) request=Request { method: GET, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("index.crates.io")), port: None, path: "/config.json", query: None, fragment: None }, headers: {} } url=Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("index.crates.io")), port: None, path: "/config.json", query: None, fragment: None }}: Received timeout error from reqwest. Delay future request by 200ms
 INFO resolve:fetch_crate_matched{self=SparseRegistry { url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("index.crates.io")), port: None, path: "/", query: None, fragment: None }, dl_template: OnceCell { value: None } } client=Client(Inner { client: Client { accepts: Accepts { gzip: true, brotli: true, deflate: true }, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*", "user-agent": "cargo-binstall/1.6.0"} }, service: DelayRequest { inner: Mutex { data: Inner { client: Client { accepts: Accepts { gzip: true, brotli: true, deflate: true }, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*", "user-agent": "cargo-binstall/1.6.0"} }, num_request: 1, per: 10ms, until: Instant { tv_sec: 62059, tv_nsec: 701819054 }, state: Ready { rem: 1 } }, poisoned: false, .. }, hosts_to_delay: Mutex { data: {}, poisoned: false, .. } } }) crate_name="cargo-binstall" version_req=VersionReq { comparators: [Comparator { op: Exact, major: 1, minor: Some(6), patch: Some(1), pre: Prerelease("") }] }}:do_send_request{self=Client(Inner { client: Client { accepts: Accepts { gzip: true, brotli: true, deflate: true }, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*", "user-agent": "cargo-binstall/1.6.0"} }, service: DelayRequest { inner: Mutex { data: Inner { client: Client { accepts: Accepts { gzip: true, brotli: true, deflate: true }, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*", "user-agent": "cargo-binstall/1.6.0"} }, num_request: 1, per: 10ms, until: Instant { tv_sec: 62059, tv_nsec: 701819054 }, state: Limited }, poisoned: false, .. }, hosts_to_delay: Mutex { data: {"index.crates.io": Instant { tv_sec: 62059, tv_nsec: 940404063 }}, poisoned: false, .. } } }) request=Request { method: GET, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("index.crates.io")), port: None, path: "/config.json", query: None, fragment: None }, headers: {} } url=Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("index.crates.io")), port: None, path: "/config.json", query: None, fragment: None }}: Received timeout error from reqwest. Delay future request by 200ms
 INFO resolve:fetch_crate_matched{self=SparseRegistry { url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("index.crates.io")), port: None, path: "/", query: None, fragment: None }, dl_template: OnceCell { value: None } } client=Client(Inner { client: Client { accepts: Accepts { gzip: true, brotli: true, deflate: true }, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*", "user-agent": "cargo-binstall/1.6.0"} }, service: DelayRequest { inner: Mutex { data: Inner { client: Client { accepts: Accepts { gzip: true, brotli: true, deflate: true }, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*", "user-agent": "cargo-binstall/1.6.0"} }, num_request: 1, per: 10ms, until: Instant { tv_sec: 62059, tv_nsec: 701819054 }, state: Ready { rem: 1 } }, poisoned: false, .. }, hosts_to_delay: Mutex { data: {}, poisoned: false, .. } } }) crate_name="cargo-binstall" version_req=VersionReq { comparators: [Comparator { op: Exact, major: 1, minor: Some(6), patch: Some(1), pre: Prerelease("") }] }}:do_send_request{self=Client(Inner { client: Client { accepts: Accepts { gzip: true, brotli: true, deflate: true }, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*", "user-agent": "cargo-binstall/1.6.0"} }, service: DelayRequest { inner: Mutex { data: Inner { client: Client { accepts: Accepts { gzip: true, brotli: true, deflate: true }, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*", "user-agent": "cargo-binstall/1.6.0"} }, num_request: 1, per: 12ms, until: Instant { tv_sec: 62060, tv_nsec: 153045636 }, state: Limited }, poisoned: false, .. }, hosts_to_delay: Mutex { data: {"index.crates.io": Instant { tv_sec: 62060, tv_nsec: 375037122 }}, poisoned: false, .. } } }) request=Request { method: GET, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("index.crates.io")), port: None, path: "/config.json", query: None, fragment: None }, headers: {} } url=Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("index.crates.io")), port: None, path: "/config.json", query: None, fragment: None }}: Received timeout error from reqwest. Delay future request by 200ms
ERROR Fatal error:
  × For crate cargo-binstall: could not GET https://index.crates.io/config.json: error sending request for url (https://index.crates.io/config.json): error trying to connect: invalid peer
  │ certificate: UnknownIssuer
  ├─▶ could not GET https://index.crates.io/config.json: error sending request for url (https://index.crates.io/config.json): error trying to connect: invalid peer certificate: UnknownIssuer
  ├─▶ error sending request for url (https://index.crates.io/config.json): error trying to connect: invalid peer certificate: UnknownIssuer
  ├─▶ error trying to connect: invalid peer certificate: UnknownIssuer
  ╰─▶ invalid peer certificate: UnknownIssuer

As I said above, running a plain cargo install cargo-binstall works perfectly. It seems cargo-update does not respect the system certificate store.

System & environment info:

• cargo-install-update 13.3.0
• cargo 1.75.0 (1d8b05cdd 2023-11-20)
• Ubuntu 22.04
• The proxy is transparent, so no HTTP_PROXY/HTTPS_PROXY variable needs to be set.

[nabijaczleweli]

Hm. It looks like, to the extent of, cargo-update talking to the network, it did work – polling the registry completed, this output matches reality.

The step that failed is the updating step, i.e. fork()/exec("cargo", "[b]install", ...) – all the logs since INFO are from either cargo or cargo-binstall.

Given the formatting and them containing three 2kB blobs, I'd assume cargo-binstall, especially since you say cargo install cargo-binstall works – how about cargo binstall cargo-binstall?

[NobodyXu]

Hmmm it seems that cargo-binstall has problem dealing with your transparent proxy setup.

Would you please open a ticket in cargo-binstall and gives us the info required to reproduce/fix this?

Thanks!

[arvid-norlander]

I don't believe it is cargo-bininstall. It seems to apply to all packages I build via "cargo install-update" as opposed to "cargo install". Cargo-bininstall was just the example I pasted above.

[NobodyXu]

cargo install-update would use cargo-binstall whenever it is available.

Can you try using cargo-binstall directly?

[arvid-norlander]

Oh, didn't know that. I can try when I get back to the office. Right now it is hard to try: 1) everything is up-to-date 2) I'm working remotely today rather than at the office (and it only happens when in the office for some reason).

But I do find the line error sending request for url (https://index.crates.io/config.json): error trying to connect: invalid peer certificate: UnknownIssuer rather telling. Since the proxy performs a MITM attack basically you need to have the HTTPS certificate installed into your trust store. It is installed in the trust store of Ubuntu (/etc/ca-certificates) as well as of Firefox and Chrome. For some reason it seems cargo-bininstall doesn't pick up that certificate then.

[NobodyXu]

You can use cargo binstall --force cargo-binstall to force it to install it again.

P.S. it's cargo-binstall, not cargo-bininstall.

It is installed in the trust store of Ubuntu (/etc/ca-certificates) as well as of Firefox and Chrome. For some reason it seems cargo-bininstall doesn't pick up that certificate then.

Could you please open an issue in cargo-binstall?
I will investigate in it

[NobodyXu]

Actually I already have a hint on what went wrong.

Would open a PR tomorrow

[NobodyXu]

Opened cargo-bins/cargo-binstall#1589 for this, it should fix the issue for you.

[arvid-norlander]

Thanks I will check it out today when I'm at the office.

[arvid-norlander]

I had nothing that needed updating, but cargo auditable install-update -f cargo-about worked and was able to use cargo-binstall with the new version, so I suspect this is working properly now.