Add cgroupv1 support to available_parallelism

[the8472]

Fixes #97549

My dev machine uses cgroup v2 so I was only able to test that code path. So the v1 code path is written only based on documentation. I could use some help testing that it works on a machine with cgroups v1:

$ x.py build --stage 1

# quota.rs
fn main() {
    println!("{:?}", std::thread::available_parallelism());
}

# assuming stage1 is linked in rustup
$ rust +stage1 quota.rs

# spawn a new cgroup scope for the current user
$ sudo systemd-run -p CPUQuota="300%" --uid=$(id -u) -tdS

# should print Ok(3)
$ ./quota

If it doesn't work as expected an strace, the contents of /proc/self/cgroups and the structure of /sys/fs/cgroups would help.

[rust-highfive]

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

• Stabilizing library features
• Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
• Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
• Changing public documentation in ways that create new stability guarantees
• Changing observable runtime behavior of library APIs

[rust-highfive]

r? @kennytm

(rust-highfive has picked a reviewer for you, use r? to override)

[Mark-Simulacrum]

What would it take for us to test both variants of cgroup in CI? Can we "just" mount them, or do we need qemu to emulate a different kernel or something?

[the8472]

From cgroups(7)

       In cgroups v2, all mounted controllers reside in a single unified
       hierarchy.  While (different) controllers may be simultaneously
       mounted under the v1 and v2 hierarchies, it is not possible to
       mount the same controller simultaneously under both the v1 and
       the v2 hierarchies.

So we can have the mountpoints, but not the cpu controller in both v1 and v2 at the same time.
With enough effort it might be possible to configure systemd and docker so that the controllers used in neither v1 nor v2 and we could reshuffle the controller at runtime but that's speculation and might require fiddling with a lot of things.

Running a full kernel in qemu could work.

If github actions runners can be rebooted with different kernel parameters then maybe we could have a separate job relying on a build artifact and tell systemd to use v1 instead of v2 or vice versa.

[ehuss]

I'm trying this out with Ubuntu 20 and Docker (with --cpus 4), and I'm unable to get it to work correctly. I'm not familiar with how this is supposed to work, but it seems to be looking for a path like /sys/fs/cgroup/cpu/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de which doesn't exist. I have:

root@02b29e5cb9b5:/# cat /proc/self/cgroup
14:name=systemd:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
13:rdma:/
12:pids:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
11:hugetlb:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
10:net_prio:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
9:perf_event:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
8:net_cls:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
7:freezer:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
6:devices:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
5:memory:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
4:blkio:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
3:cpuacct:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
2:cpu:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
1:cpuset:/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de
0::/

root@02b29e5cb9b5:/tmp/a# cat /proc/self/mountinfo  | grep cpu
1119 1074 0:33 /docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de /sys/fs/cgroup/cpuset ro,nosuid,no
dev,noexec,relatime master:66 - cgroup cpuset rw,cpuset
1120 1074 0:34 /docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de /sys/fs/cgroup/cpu ro,nosuid,nodev
,noexec,relatime master:67 - cgroup cpu rw,cpu
1121 1074 0:35 /docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de /sys/fs/cgroup/cpuacct ro,nosuid,n
odev,noexec,relatime master:68 - cgroup cpuacct rw,cpuacct

root@02b29e5cb9b5:/# ls -al /sys/fs/cgroup/cpu
total 0
drwxr-xr-x  2 root root   0 Jun  9 23:37 .
dr-xr-xr-x 16 root root 320 Jun  9 23:37 ..
-rw-r--r--  1 root root   0 Jun  9 23:38 cgroup.clone_children
-rw-r--r--  1 root root   0 Jun  9 23:37 cgroup.procs
-rw-r--r--  1 root root   0 Jun  9 23:37 cpu.cfs_period_us
-rw-r--r--  1 root root   0 Jun  9 23:37 cpu.cfs_quota_us
-rw-r--r--  1 root root   0 Jun  9 23:38 cpu.rt_period_us
-rw-r--r--  1 root root   0 Jun  9 23:38 cpu.rt_runtime_us
-rw-r--r--  1 root root   0 Jun  9 23:38 cpu.shares
-r--r--r--  1 root root   0 Jun  9 23:38 cpu.stat
-rw-r--r--  1 root root   0 Jun  9 23:38 notify_on_release
-rw-r--r--  1 root root   0 Jun  9 23:38 tasks

root@02b29e5cb9b5:/# cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us
400000

root@02b29e5cb9b5:/# cat /sys/fs/cgroup/cpu/cpu.cfs_period_us
100000

From what I can see from num_cpus, it parses /proc/self/mountinfo and strips the value from the path so that it ends up reading from /sys/fs/cgroup/cpu/cpu.cfs_quota_us.

[the8472]

That looks like your docker isn't using cgroup namespaces and uses bind-mounts to remap cgroupfs instead. Maybe it only does namespaces for v2.

In that case I think we might not get around parsing mountinfo.

CC @joshtriplett

[joshtriplett]

If I'm understanding the cgroup layout correctly, that seems to be a docker-specific hack: it's assuming that the root path (/docker/...) is the same as the cgroup path. cgroups could have any arbitrary name and be mounted anywhere, and there's zero guarantee that the cgroup path matches that field.

If we're going to do a docker-specific hack to handle this case, I think we could just do a simple fallback here: if we can't find a cgroup v1 by appending group_path, we can parse the information from the top-level cgroup if it exists, and use that limit, under the assumption that it's been bind-mounted. So, after failing to find /sys/fs/cgroup/cpu/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de, we'd check the top-level /sys/fs/cgroup/cpu.

That would work in the Docker case, as well as the case of a system using cgroup namespaces. And it would avoid parsing mountinfo, which on some large systems can have an arbitrarily large number of entries.

[the8472]

that seems to be a docker-specific hack

"docker specific" might be fairly widespread, especially if it actually happens in some shared component like runc.

So, after failing to find /sys/fs/cgroup/cpu/docker/02b29e5cb9b58c859327b532d77678965de9b116a516818c9f952fbf491f00de, we'd check the top-level /sys/fs/cgroup/cpu.

That would only work at the first level, but not if something inside the container creates another nested cgroup, it would then miss the inner quotas because it always jumps to the (visible) top.

And it would avoid parsing mountinfo, which on some large systems can have an arbitrarily large number of entries.

Well, we could still avoid that by default and only do it as fallback. If one is in a container with a mount namespace then fewer mounts should be visible in the first place. And if the program is running on the host then one should be seeing the vanilla cgroupfs rather than the bind mount shenanigans and we don't have to touch mountinfo.

We could try ablating the group_path from the left until we find the longest match. But that seems increasingly convoluted just to avoid doing the correct thing on what already is a fallback path anyway.

[joshtriplett]

That would only work at the first level, but not if something inside the container creates another nested cgroup, it would then miss the inner quotas because it always jumps to the (visible) top.

That seems like a special-case of a special-case: you have to be running docker or similar, using cgroup v1 without namespaces, and run a nested cgroup inside it.

Well, we could still avoid that by default and only do it as fallback. ... And if the program is running on the host then one should be seeing the vanilla cgroupfs rather than the bind mount shenanigans and we don't have to touch mountinfo.

The problem is that that "fallback" will happen 100% of the time if your process isn't in a cgroup at all.

However, here's another possibility: with a single metadata (stat) call, we can check first if /sys/fs/cgroup exists at all and has a link count larger than 2 (meaning it has subdirectories); if it doesn't exist, we skip all the cgroup handling; if it exists but has no subdirectories we look only in the top-level cgroup; if it exists and has subdirectories, then we do the full v2-then-v1 fallback including parsing mountinfo for v1.

That should allow us to avoid the linear-in-the-number-of-mounts fallback for all cases except the one where the system actually uses cgroups v1.

[the8472]

The problem is that that "fallback" will happen 100% of the time if your process isn't in a cgroup at all.

If the

• kernel has no cgroup support then /proc/self/cgroup doesn't exist and we already bail.
• kernel does have cgroup support but cgroupfs is not mounted then the cpu controller won't be bound and won't show up in /proc/self/cgroup, so we won't take the v1 path
• v1 cpu controller is mounted in the usual location we'll find it and only do mountinfo scanning if there are bindmount shenanigans
• v1 cpu controller is mounted in a non-standard location then we still know it exists based on its presence in /proc/self/cgroup and it would make sense to do the mountinfo scanning, but I suppose we could skip it anyway (as you're suggesting) on the same grounds that we don't do mountinfo scanning for v2

[joshtriplett]

kernel does have cgroup support but cgroupfs is not mounted then the cpu controller won't be bound and won't show up in /proc/self/cgroup, so we won't take the v1 path

Ah, that's a key part of the process that I missed! Yes, of course, if you're not using cgroups or the cgroup cpu controller then your process won't be in a cgroup with the cpu controller enabled.

Alright, I fully withdraw my objections to mountinfo scanning. Checking /proc/self/cgroup suffices to make sure it's worth checking. I do agree that if a process has the cgroup v1 cpu controller showing up in /proc/self/cgroup, then it's reasonable to check.

[the8472]

@ehuss can you give it another try?

[the8472]

I have tested it on a debian9 vm + docker. It works as expected.

open("/proc/self/cgroup", O_RDONLY|O_CLOEXEC) = 3
syscall_332(0, 0, 0, 0xfff, 0, 0x20)    = -1 (errno 38)
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
lseek(3, 0, SEEK_CUR)                   = 0
read(3, "10:blkio:/docker/0c8c48d5c6b0391"..., 128) = 128
read(3, "41c9f03b116d6ac4ffa7d6a3fbef7c3d", 32) = 32
read(3, "db1fb03\n8:pids:/docker/0c8c48d5c"..., 96) = 96
read(3, "uacct:/docker/0c8c48d5c6b0391423"..., 256) = 256
read(3, "evices:/docker/0c8c48d5c6b039142"..., 512) = 333
read(3, "", 179)                        = 0
close(3)                                = 0
stat("/sys/fs/cgroup/cpu/docker/0c8c48d5c6b03914239a8179941c9f03b116d6ac4ffa7d6a3fbef7c3ddb1fb03", 0x7ffc2c975b10) = -1 ENOENT (No such file or directory)
stat("/sys/fs/cgroup/cpu,cpuacct/docker/0c8c48d5c6b03914239a8179941c9f03b116d6ac4ffa7d6a3fbef7c3ddb1fb03", 0x7ffc2c975b10) = -1 ENOENT (No such file or directory)
open("/proc/self/mountinfo", O_RDONLY|O_CLOEXEC) = 3
read(3, "203 172 0:41 / / rw,relatime mas"..., 8192) = 4084
close(3)                                = 0
stat("/sys/fs/cgroup/cpu,cpuacct/", {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
open("/sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
lseek(3, 0, SEEK_CUR)                   = 0
read(3, "300000\n", 20)                 = 7
read(3, "", 13)                         = 0
close(3)                                = 0
open("/sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
lseek(3, 0, SEEK_CUR)                   = 0
read(3, "100000\n", 20)                 = 7
read(3, "", 13)                         = 0
close(3)                                = 0
sched_getaffinity(0, 128, [0, 1, 2, 3, 4]) = 64
write(1, "Ok(3)\n", 6Ok(3)
)                  = 6

[ehuss]

I tried, and it still doesn't seem to be working. It is still looking at paths like /sys/fs/cgroup/cpu/docker/0687f1a6e752fd14c7a9e3f85ac6eb71691966b8bbb98ec79e8d1d7fb2c3d83a/cpu.cfs_quota_us. But it needs to strip out that part of the path so that it is /sys/fs/cgroup/cpu/cpu.cfs_quota_us. num_cpus strips it here so that it accesses the path without the docker… stuff.

[the8472]

It is still looking at paths like /sys/fs/cgroup/cpu/docker/0687f1a6e752fd14c7a9e3f85ac6eb71691966b8bbb98ec79e8d1d7fb2c3d83a/cpu.cfs_quota_us.

Yes, it does that. But once it doesn't find it it should be accessing /proc/self/mountinfo and look for the bind mount prefix and do the stripping, which is what I see in the strace I posted.

Can you post your strace output?

[ehuss]

Here's the strace output:

strace

execve("./a", ["./a"], 0x7fff26161280 /* 11 vars */) = 0
brk(NULL)                               = 0x5596e2a81000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=31066, ...}) = 0
mmap(NULL, 31066, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f94941be000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0203\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=100736, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f94941bc000
mmap(NULL, 103496, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f94941a2000
mmap(0x7f94941a5000, 69632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7f94941a5000
mmap(0x7f94941b6000, 16384, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14000) = 0x7f94941b6000
mmap(0x7f94941ba000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7f94941ba000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 |\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=149520, ...}) = 0
mmap(NULL, 136304, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9494180000
mmap(0x7f9494187000, 65536, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x7f9494187000
mmap(0x7f9494197000, 20480, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7f9494197000
mmap(0x7f949419c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b000) = 0x7f949419c000
mmap(0x7f949419e000, 13424, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f949419e000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000\21\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=18688, ...}) = 0
mmap(NULL, 20752, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f949417a000
mmap(0x7f949417b000, 8192, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7f949417b000
mmap(0x7f949417d000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7f949417d000
mmap(0x7f949417e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7f949417e000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@n\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1839792, ...}) = 0
mmap(NULL, 1852680, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9493fb5000
mprotect(0x7f9493fda000, 1662976, PROT_NONE) = 0
mmap(0x7f9493fda000, 1355776, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x25000) = 0x7f9493fda000
mmap(0x7f9494125000, 303104, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x170000) = 0x7f9494125000
mmap(0x7f9494170000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1ba000) = 0x7f9494170000
mmap(0x7f9494176000, 13576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f9494176000
close(3)                                = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9493fb3000
arch_prctl(ARCH_SET_FS, 0x7f9493fb3bc0) = 0
mprotect(0x7f9494170000, 12288, PROT_READ) = 0
mprotect(0x7f949417e000, 4096, PROT_READ) = 0
mprotect(0x7f949419c000, 4096, PROT_READ) = 0
mprotect(0x7f94941ba000, 4096, PROT_READ) = 0
mprotect(0x5596e18bc000, 16384, PROT_READ) = 0
mprotect(0x7f94941f0000, 4096, PROT_READ) = 0
munmap(0x7f94941be000, 31066)           = 0
set_tid_address(0x7f9493fb3e90)         = 40
set_robust_list(0x7f9493fb3ea0, 24)     = 0
rt_sigaction(SIGRTMIN, {sa_handler=0x7f9494187690, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7f9494194140}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {sa_handler=0x7f9494187730, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0x7f9494194140}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
poll([{fd=0, events=0}, {fd=1, events=0}, {fd=2, events=0}], 3, 0) = 0 (Timeout)
rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[PIPE], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f9493ff0d60}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGSEGV, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGSEGV, {sa_handler=0x5596e1888de0, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_SIGINFO, sa_restorer=0x7f9494194140}, NULL, 8) = 0
rt_sigaction(SIGBUS, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGBUS, {sa_handler=0x5596e1888de0, sa_mask=[], sa_flags=SA_RESTORER|SA_ONSTACK|SA_SIGINFO, sa_restorer=0x7f9494194140}, NULL, 8) = 0
sigaltstack(NULL, {ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}) = 0
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f94941c3000
mprotect(0x7f94941c3000, 4096, PROT_NONE) = 0
sigaltstack({ss_sp=0x7f94941c4000, ss_flags=0, ss_size=8192}, NULL) = 0
brk(NULL)                               = 0x5596e2a81000
brk(0x5596e2aa2000)                     = 0x5596e2aa2000
openat(AT_FDCWD, "/proc/self/maps", O_RDONLY|O_CLOEXEC) = 3
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "5596e1868000-5596e186e000 r--p 0"..., 1024) = 1024
read(3, "          /lib/x86_64-linux-gnu/"..., 1024) = 1024
read(3, "568                     /lib/x86"..., 1024) = 1024
read(3, " 00:00 0 \n7f94941c6000-7f94941c7"..., 1024) = 830
close(3)                                = 0
sched_getaffinity(40, 32, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]) = 32
openat(AT_FDCWD, "/proc/self/cgroup", O_RDONLY|O_CLOEXEC) = 3
statx(0, NULL, AT_STATX_SYNC_AS_STAT, STATX_ALL, NULL) = -1 EFAULT (Bad address)
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0444, stx_size=0, ...}) = 0
lseek(3, 0, SEEK_CUR)                   = 0
read(3, "14:name=systemd:/docker/1f9c571e"..., 128) = 128
read(3, "a20d7b64b28f723a1d55c6de88c390dc", 32) = 32
read(3, "6c16df635a726784be6\n11:hugetlb:/"..., 96) = 96
read(3, "6784be6\n10:net_prio:/docker/1f9c"..., 256) = 256
read(3, "84be6\n7:freezer:/docker/1f9c571e"..., 512) = 512
read(3, "r/1f9c571e3d897a20d7b64b28f723a1"..., 1024) = 72
read(3, "", 952)                        = 0
close(3)                                = 0
statx(AT_FDCWD, "/sys/fs/cgroup/cpu/docker/1f9c571e3d897a20d7b64b28f723a1d55c6de88c390dc6c16df635a726784be6", AT_STATX_SYNC_AS_STAT, STATX_ALL, 0x7ffe3b9bee90) = -1 ENOENT (No such file or directory)
statx(AT_FDCWD, "/sys/fs/cgroup/cpu,cpuacct/docker/1f9c571e3d897a20d7b64b28f723a1d55c6de88c390dc6c16df635a726784be6", AT_STATX_SYNC_AS_STAT, STATX_ALL, 0x7ffe3b9bee90) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/proc/self/mountinfo", O_RDONLY|O_CLOEXEC) = 3
read(3, "1069 916 0:168 / / rw,relatime m"..., 8192) = 3872
read(3, "1092 1069 8:48 /version-pack-dat"..., 8192) = 1521
read(3, "", 8192)                       = 0
close(3)                                = 0
sched_getaffinity(0, 128, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]) = 32
write(1, "32\n", 3)                     = 3
32
sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=8192}, NULL) = 0
munmap(0x7f94941c3000, 12288)           = 0
exit_group(0)                           = ?
+++ exited with 0 +++

From what I can tell, in https://github.com/rust-lang/rust/blob/d41e4c8b779344412f07c8edc6c2326e7cb2a369/library/std/src/sys/unix/thread.rs#L517-L527

the call to try_exists always fails (because it unconditionally has the group_path on the end) which prevents it following through to the while loop below.

[the8472]

I don't think it's the try_exists because it only tries twice, but not a third time where it should:

https://github.com/rust-lang/rust/blob/d41e4c8b779344412f07c8edc6c2326e7cb2a369/library/std/src/sys/unix/thread.rs#L509-L514

So there must be an issue with the mountinfo parsing, it doesn't find a suitable candidate for the fallback attempt on your system but does on mine.

[the8472]

Can you provide the full mountinfo from inside the docker container? I suspect it contains something that makes my parser bail too early.

[ehuss]

Ah, I see. From /proc/self/mountinfo the cpu line looks like:

line="1076 1074 0:34 /docker/3062f9854490e1c5ff19d56585ae73d5280322fa45262ccbfa1ebdaea0325487 /sys/fs/cgroup/cpu ro,nosuid,nodev,noexec,relatime master:68 - cgroup cpu rw,cpu\n"

Notice the newline at the end. mount_opts ends up being "rw,cpu\n", and "cpu\n" != "cpu". I added a trim() and got the expected value (4 cpus).

[the8472]

Ahh, that explains it, debian had a combined cpu,cpuacct controller which masked the trailing newline. Thanks for investigating.

[bors]

📌 Commit 2e33c81 has been approved by joshtriplett

It is now in the queue for this repository.

[bors]

⌛ Testing commit 2e33c81 with merge 08e6a5af25d0d9399aa069e2bf7b4c1106199bb5...

[bors]

💔 Test failed - checks-actions

[rust-log-analyzer]

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

[the8472]

curl: (56) OpenSSL SSL_read: Connection was reset, errno 10054

network error.

@bors retry

[bors]

⌛ Testing commit 2e33c81 with merge e55c53c...

[bors]

☀️ Test successful - checks-actions
Approved by: joshtriplett
Pushing e55c53c to master...

[rust-timer]

Finished benchmarking commit (e55c53c): comparison url.

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

• Primary benchmarks: 🎉 relevant improvement found
• Secondary benchmarks: 🎉 relevant improvement found

	mean1	max	count2
Regressions 😿 (primary)	N/A	N/A	0
Regressions 😿 (secondary)	N/A	N/A	0
Improvements 🎉 (primary)	-5.7%	-5.7%	1
Improvements 🎉 (secondary)	-1.5%	-1.5%	1
All 😿🎉 (primary)	-5.7%	-5.7%	1

Cycles

Results

• Primary benchmarks: 😿 relevant regressions found
• Secondary benchmarks: 🎉 relevant improvement found

	mean1	max	count2
Regressions 😿 (primary)	2.8%	2.9%	2
Regressions 😿 (secondary)	N/A	N/A	0
Improvements 🎉 (primary)	N/A	N/A	0
Improvements 🎉 (secondary)	-4.6%	-4.6%	1
All 😿🎉 (primary)	2.8%	2.9%	2

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

Footnotes

1. the arithmetic mean of the percent change ↩ ↩²

2. number of relevant changes ↩ ↩²