From 6e7c49fb1893abf2c9469da18e8660381e504fb7 Mon Sep 17 00:00:00 2001 From: James Bourbeau Date: Tue, 25 Oct 2022 14:35:51 -0500 Subject: [PATCH] Add note about limiting thread oversubscription by default (#9592) --- docs/source/array-best-practices.rst | 7 +++++++ docs/source/deploying-python-advanced.rst | 2 ++ 2 files changed, 9 insertions(+) diff --git a/docs/source/array-best-practices.rst b/docs/source/array-best-practices.rst index 2b4320294d1..346a89d06e8 100644 --- a/docs/source/array-best-practices.rst +++ b/docs/source/array-best-practices.rst @@ -66,6 +66,13 @@ Note that if you provide ``chunks='auto'`` then Dask Array will look for a Avoid Oversubscribing Threads ----------------------------- +.. tip:: + + When using the ``distributed`` scheduler, the ``OMP_NUM_THREADS``, + ``MKL_NUM_THREADS``, and ``OPENBLAS_NUM_THREADS`` environment variables + are automatically set to ``1`` when using :ref:`nanny` workers. + This helps avoid oversubscribing threads in common cases. + By default Dask will run as many concurrent tasks as you have logical cores. It assumes that each task will consume about one core. However, many array-computing libraries are themselves multi-threaded, which can cause diff --git a/docs/source/deploying-python-advanced.rst b/docs/source/deploying-python-advanced.rst index 16768ab121c..ebb6bcee298 100644 --- a/docs/source/deploying-python-advanced.rst +++ b/docs/source/deploying-python-advanced.rst @@ -166,6 +166,8 @@ In this example we don't wait on ``s.finished()``, so this will terminate relatively quickly. You could have called ``await s.finished()`` though if you wanted this to run forever. +.. _nanny: + Nanny -----