Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use psutil.cpu_count(logical=False) in auto_detect_cpus() #560

Merged
merged 4 commits into from Aug 3, 2020
Merged

Use psutil.cpu_count(logical=False) in auto_detect_cpus() #560

merged 4 commits into from Aug 3, 2020

Conversation

utapyngo
Copy link
Contributor

@utapyngo utapyngo commented Jul 23, 2020

Fixes #553.

@utapyngo utapyngo marked this pull request as ready for review July 23, 2020 03:55
Copy link
Member

@adamchainz adamchainz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


try:
n = cpu_count()
n = psutil.cpu_count(logical=False)
except NotImplementedError:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this something that psutil will raise?

Return the number of logical CPUs in the system (same as os.cpu_count in Python 3.4) or None if undetermined. logical cores means the number of physical cores multiplied by the number of threads that can run on each core (this is known as Hyper Threading). If logical is False return the number of physical cores only (Hyper Thread CPUs are excluded) or None if undetermined. On OpenBSD and NetBSD psutil.cpu_count(logical=False) always return None. Example on a system having 2 physical hyper-thread CPU cores:

if it never raises, we can use return psutil.cpu_count(logical=False) or 1

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also concerned about:

On OpenBSD and NetBSD psutil.cpu_count(logical=False) always return None.

should we fallback to the old code if we get a None?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we fallback to the old code if we get a None?

Not sure, we don't officially support OpenBSD and NetBSD... your suggestion above of using return psutil.cpu_count(logical=False) or 1 would at least prevent a crash, and I would rather keep the code cleaner. But if others disagree then I think your suggestion is on point.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this obviate the Travis workaround?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. 👍

Copy link
Contributor Author

@utapyngo utapyngo Jul 28, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this locally on pypy35:

image

Copy link
Member

@Zac-HD Zac-HD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 👍

@Zac-HD Zac-HD merged commit 0094b29 into pytest-dev:master Aug 3, 2020
nicoddemus added a commit to nicoddemus/pytest-xdist that referenced this pull request Aug 24, 2020
…ount"

This reverts commit 0094b29, reversing
changes made to c6255fa.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

When using "-n auto", count the number of physical CPU cores instead of logical ones
5 participants