-
-
Notifications
You must be signed in to change notification settings - Fork 710
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
trouble setting dask distributed worker to log level debug #6124
Comments
Hi @lastephey ! Good to see you again. Try this maybe?
|
Hi @mrocklin, Thank you, same to you! Your suggestion does work, but it seems to turn all other Dask logging to Starting the scheduler like
and then some workers like
results in a lot of output, where it appears all parts of Dask distributed are now set to
Is there a way I can change the Thank you very much, |
Configuring individual loggers might not be possible from the command line. The distributed:
version: 2
logging:
distributed: info
distributed.client: warning
distributed.scheduler: debug |
Thanks @quasiben, changing the logging settings via the config file does what I need. Thank you both very much for your help. I'll go ahead and close this. |
@quasiben, I've implemented the possibility of configuring these loggers via environment variables in the following PR: |
What happened:
Dear Dask devs,
I am unable to set the only the dask distributed worker log level to 'DEBUG' when manually starting a cluster. I have seen there have been several issues and PRs on this subject, but I apologize-- it's still not totally clear to me what the state is or what the correct syntax is.
#2937
#2419
#2952
#4642
What you expected to happen:
I was trying to follow the advice in the dask distributed docs. I copy/pasted the logging yaml example into the configuration conversion utility to try to generate the right environment variables. The example formatting generated was
I saw though in this comment that the dot in the key should translate to a double underscore in the environment variable, so I have changed that in my test below.
Minimal Complete Verifiable Example:
My ultimate goal is to set the workers and maybe client to log level debug, but here I'll demonstrate with the scheduler since it shows the same behavior.
Based on the output of the config generator, I first tried:
although this syntax is apparently invalid. It gives the error:
Based on this comment I also tried:
which does work today, even if it didn't 2 years ago (which is confusing). This turns on logging for everything which is good, but of course it's really verbose. I was hoping I could turn it on only for particular parts of Dask distributed.
tl;dr I only stumbled onto a partially working solution. I would really appreciate it if you could clarify what should/shouldn't work, and perhaps document it somewhere like the logging page.
Anything else we need to know?:
Environment:
Python version:
3.8.5
Operating System: Running inside a custom container implementation shifter,
ubuntu 20.04
Install method (conda, pip, source):
conda
Cluster Dump State:
Thank you all very much for your work on Dask,
Laurie
The text was updated successfully, but these errors were encountered: