Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Concurrent.usable_processor_count that is cgroups aware #1038

Merged
merged 3 commits into from Feb 1, 2024

Conversation

casperisfine
Copy link

Closes: #1035

A running gag since the introduction of containerization is software that starts one process per logical or physical core while running inside a container with a restricted CPU quota and totally blowing up memory usage in containerized environments.

The proper question to ask is how many CPU cores are usable, not how many the machine has. To do that we have to read the cgroup info from /sys. There is two way of doing it depending on the version of cgroups used.

Co-Authored-By: @usiegl00

@casperisfine
Copy link
Author

Arf:

     NoMethodError:
       undefined method `match?' for "linux":String

I'll fix 2.3 compat.

@casperisfine casperisfine force-pushed the cgroup-support branch 2 times, most recently from 2ad87f7 to aecdebb Compare January 29, 2024 16:43
Copy link
Collaborator

@eregon eregon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just need a bit more documentation care.

Closes: ruby-concurrency#1035

A running gag since the introduction of containerization is software
that starts one process per logical or physical core while running
inside a container with a restricted CPU quota and totally blowing
up memory usage in containerized environments.

The proper question to ask is how many CPU cores are usable, not how
many the machine has. To do that we have to read the cgroup info
from `/sys`. There is two way of doing it depending on the version
of cgroups used.

Co-Authored-By: usiegl00 <50933431+usiegl00@users.noreply.github.com>
Copy link
Collaborator

@eregon eregon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

spec/concurrent/utility/processor_count_spec.rb Outdated Show resolved Hide resolved
spec/concurrent/utility/processor_count_spec.rb Outdated Show resolved Hide resolved
spec/concurrent/utility/processor_count_spec.rb Outdated Show resolved Hide resolved
spec/concurrent/utility/processor_count_spec.rb Outdated Show resolved Hide resolved
@eregon eregon merged commit eae2851 into ruby-concurrency:master Feb 1, 2024
@casperisfine
Copy link
Author

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants