Attempting to install Rustup 1.24.2 hangs

[luqmana]

Problem

After #2756 rustup will just hang if it's not able to determine the effective max ram of the system. The differences seem to be:

1. The fallback for when effective_limits::memory_limit changed from 500MB to 32MB.
2. Rustup gets stuck waiting for some memory to free up here:

   rustup/src/dist/component/package.rs

   Lines 402 to 409 in a1d5912

   	while !io_executor.buffer_available(size as usize) {
   	flush_ios::<tar::Entry<'_, R>, _>(
   	&mut *io_executor,
   	&mut directories,
   	None,
   	&full_path,
   	)?;
   	}

I encountered this running on an illumos machine.

Steps
On a machine not supported by effective_limits v0.5.2`:

$ curl https://sh.rustup.rs -sSf | sh -s -- --profile minimal --default-toolchain nightly

Possible Solution(s)

1. Choose a larger default than 32MB (note, 64MB is still not enough for --profile complete).
2. v0.5.3-alpha of the effective-limits crate supports illumos but hasn't been released yet. (New release rbtcollins/effective-limits.rs#15)

Notes

Output of rustup --version: 1.24.2 (755e2b07e 2021-05-12)

[chrisduerr]

This also breaks on FreeBSD, which I've noticed when my CI suddenly started failing. The default 32MB aren't even enough for the minimal install, so I think platforms without effective limits just can't build right now.

[rbtcollins]

hmm, a hang is unexpected: the threaded executor allocates a chunk from every slab size, and that will allow at least one io to proceed; the streaming ios also don't have busy waits that I remember, so it should always just loop until something completes; mkdirs to let something flush don't get accounted for against the budget (sssh), so it should always be possible for flush_ios to complete.

obviously that isn't happening, which means figuring out why not.

you can workaround using single threaded mode: export RUSTUP_IO_THREADS=1 and you'll disable the fast path.

[Darksonn]

Adding the RUSTUP_IO_THREADS=1 environment variable fixed FreeBSD CI for us.

[kinnison]

I can confirm this is reproducable on Linux, which means we (well I) stand a chance of diagnosing at least. It's a little odd that you say 64M isn't enough for complete though - can you let me know which component it gets stuck on in complete which isn't in default ? I've managed to reproduce the issue at 32M with minimal profile because the rustc component seems enough.

Is the CI you use for FreeBSD compatible with Github ? I'd be prepared to add more CI options to our master branch build if it helps us catch this kind of problem in the future.

[Darksonn]

We have a Cirrus CI setup on the Tokio github repository that we use for FreeBSD.

[Thomasdezeeuw]

Since this breaking/blocking some stuff in the ecosystem would it be possible to just use 500MB for the time being as a workout?

[jclulow]

It's probably best to revert the regressing change for now -- the motivation was ostensibly a "hypothetical" memory fragmentation issue, whereas the hang is very real.

[kinnison]

@jclulow the fragmentation problem was very real on tier 1 platforms. I have what I think is a fix for this hang and I hope I'll get a beta out early next week with it.

[rbtcollins]

Yeah - to be clear, the problem was concrete, the hypothetical aspect was whether fragmentation was the cause.

[Thomasdezeeuw]

Can we get a release for this? It's breaking the CI for a lot of people still.

[kinnison]

@Thomasdezeeuw My intention is to prepare a beta release tonight, and hopefully to get it onto dev-static tomorrow. Keep an eye on the users forum for a call to test it.

[Tim-Zhang]

@kinnison How are things going?

[hi-rustin]

@kinnison How are things going?

The beta release has been released and you can find the test method here.

[hi-rustin]

on the user forum.

It seems to be on the internals forum?

[kinnison]

It seems to be on the internals forum?

Yeah, I get the two mixed up, sorry :(