Replies: 2 comments
-
@averri Thanks for your question, we prefer Discourse for usage questions, would you mind moving this discussion there? |
Beta Was this translation helpful? Give feedback.
-
Hi @pavithraes, I have found a way to get the annotations from the tasks' code, it's a bit cumbersome, it requires executing a code in the scheduler. But it seems there is a bug with the implementation. I have opened this issue: dask/distributed#6930 I also have a suggestion to improve Dask, making the annotations available in the tasks' code, in a similar way the from distributed.worker import thread_state
# Get the tasks' key.
thread_state.key
# I also would like to have:
thread_state.annotations This is the issue for the feature suggestion: https://github.com/dask/dask/issues/9414 |
Beta Was this translation helpful? Give feedback.
-
I would like to pass any metadata to all tasks associated with a particular client. The main objective is to add a unique ID that is part of a large context, this unique ID must be printed in the log of the workers for correlation. I have tried using annotations, but it's not clear how to extract the annotations from the worker's code.
A simple example:
In the example above, 4 different threads submit a task to Dask. Each submission has a unique
request_id
that should be available for the tasks that belong to the same client that submitted the tasks. Therequest_id
must not be part of the task signature, because it does not belong to the domain model, this is a cross-cutting aspect that is used only for logging.How to extract the
request_id
from each task? If annotations are not for this purpose, how to achieve this goal?Beta Was this translation helpful? Give feedback.
All reactions