Use distributed default clients even if no config is set
#9808
Conversation
| if getattr(thread_state, "key", False): | ||
| from distributed.worker import get_worker | ||
|
|
||
| return get_worker().client.get |
There was a problem hiding this comment.
This is equivalent to the get_client call I introduced below if getattr(thread_state, "key", False) is True so there is no need to have this additional indirection.
By having only a single call, we're changing precedence slightly s.t. a class providing the __dask_scheduler__ attribute wins but I believe this "more correct" considering it is passed explicitly. I doubt this has any relevance.
dask/base.py
Outdated
| try: | ||
| from distributed import get_client | ||
|
|
||
| return get_client().get | ||
| except ValueError: | ||
| pass |
There was a problem hiding this comment.
i.e. if we're not requesting anything specific but are executing the default we will always try to get a global distributed client before checking the collections default.
The precedence there is
- Worker client
- Current client
there is effectively no performance penalty in case there is no distributed worker or client other than raise+except
|
Test failure is #9793 |
jrbourbeau
changed the title
distributed default clients even if no config is set
Partially addresses #9807
Will follow up with the symmetric change in distributed after merging.