From ddc686bd9cbca29f5c0c1e23749d8694c3d70721 Mon Sep 17 00:00:00 2001 From: James Bourbeau Date: Tue, 25 Oct 2022 11:27:10 -0500 Subject: [PATCH 1/2] Add note about limiting thread oversubscription by default --- docs/source/array-best-practices.rst | 7 +++++++ docs/source/deploying-python-advanced.rst | 2 ++ 2 files changed, 9 insertions(+) diff --git a/docs/source/array-best-practices.rst b/docs/source/array-best-practices.rst index 2b4320294d1..a5ad27d1d68 100644 --- a/docs/source/array-best-practices.rst +++ b/docs/source/array-best-practices.rst @@ -66,6 +66,13 @@ Note that if you provide ``chunks='auto'`` then Dask Array will look for a Avoid Oversubscribing Threads ----------------------------- +.. tip:: + + When using the ``distributed`` scheduler, the ``OMP_NUM_THREADS``, + ``MKL_NUM_THREADS``, and ``OPENBLAS_NUM_THREADS`` environment variables + will all automatically be set to ``1`` when using :ref:`nanny` workers. + This helps avoid oversubscribing threads in common cases. + By default Dask will run as many concurrent tasks as you have logical cores. It assumes that each task will consume about one core. However, many array-computing libraries are themselves multi-threaded, which can cause diff --git a/docs/source/deploying-python-advanced.rst b/docs/source/deploying-python-advanced.rst index 16768ab121c..ebb6bcee298 100644 --- a/docs/source/deploying-python-advanced.rst +++ b/docs/source/deploying-python-advanced.rst @@ -166,6 +166,8 @@ In this example we don't wait on ``s.finished()``, so this will terminate relatively quickly. You could have called ``await s.finished()`` though if you wanted this to run forever. +.. _nanny: + Nanny ----- From 1a75a3c8bd455b3b4e935d9fbb648700be8ece75 Mon Sep 17 00:00:00 2001 From: James Bourbeau Date: Tue, 25 Oct 2022 11:28:34 -0500 Subject: [PATCH 2/2] typo --- docs/source/array-best-practices.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/array-best-practices.rst b/docs/source/array-best-practices.rst index a5ad27d1d68..346a89d06e8 100644 --- a/docs/source/array-best-practices.rst +++ b/docs/source/array-best-practices.rst @@ -70,7 +70,7 @@ Avoid Oversubscribing Threads When using the ``distributed`` scheduler, the ``OMP_NUM_THREADS``, ``MKL_NUM_THREADS``, and ``OPENBLAS_NUM_THREADS`` environment variables - will all automatically be set to ``1`` when using :ref:`nanny` workers. + are automatically set to ``1`` when using :ref:`nanny` workers. This helps avoid oversubscribing threads in common cases. By default Dask will run as many concurrent tasks as you have logical cores.