Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add note about limiting thread oversubscription by default #9592

Merged
merged 2 commits into from Oct 25, 2022

Conversation

jrbourbeau
Copy link
Member

I suspect users interested in this topic would also find it useful to know these environment variables are set by default when using the distributed scheduler. This is a small follow-up to dask/distributed#7177

@github-actions github-actions bot added the documentation Improve or add to documentation label Oct 25, 2022
@jrbourbeau
Copy link
Member Author

Thanks for reviewing @ncclementi!

Noting that test failures are unrelated -- see #9597

1 similar comment
@jrbourbeau
Copy link
Member Author

Thanks for reviewing @ncclementi!

Noting that test failures are unrelated -- see #9597

@jrbourbeau jrbourbeau merged commit 6e7c49f into dask:main Oct 25, 2022
@jrbourbeau jrbourbeau deleted the auto-numpy-thread-set branch October 25, 2022 19:35
@jakirkham
Copy link
Member

jakirkham commented Oct 25, 2022

Wonder if we should be using threadpoolctl, given they help users do this sort of thing

@jrbourbeau
Copy link
Member Author

Ah, neat. I've been meaning to read up more on threadpoolctl -- what does it get us beyond what setting the *_NUM_THREADS environment variables do?

@jakirkham
Copy link
Member

It's also able to inspect how BLAS & LAPACK are configured and share that with the user. The package is pretty barebones, but maybe that makes it easier to adopt

Suppose the other option would be looking at using loky for our executors, but that might be a bigger change than we want to make (though it has some other perks)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improve or add to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants