What is the most efficient way to use CPUs in many different partitions? #7317
Unanswered
IncubatorShokuhou
asked this question in
General
Replies: 1 comment
-
I'll try to answer although I am out of my depth so people should feel free to correct me. Dask distributed is a general interface for interacting with dask clusters. Libraries like dask-jobqueue and dask-kubernetes implement different types of clusters. I think that people tend to use dask-jobqueue to create clusters when running on an HPC system. Then those clusters are passed to distributed. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
As far as I can see, there are several different ways to use multi CPUs in dask framework, e.g. dask-distributed, dask-mpi, dask-jobqueue ,etc.
So I am wondering which is the most efficient way when I have several CPU nodes on different partitions on a HPC, or that they are actually no difference?
Beta Was this translation helpful? Give feedback.
All reactions