Failed to import datashader (missing dependency on dask_expr?)

[peterroelants]

Thanks for contacting us! Please read and follow these instructions carefully, then delete this introductory text to keep your issue easy to read. Note that the issue tracker is NOT the place for usage questions and technical assistance; post those at Discourse instead. Issues without the required information below may be closed immediately.

ALL software version info

• python=3.11.8
• datashader=0.16.0
• dask-core=2024.3.0

Description of expected behavior and the observed behavior

import datashader leads to ValueError: Must install dask-expr to activate query planning.

❯ python
Python 3.11.8 | packaged by conda-forge | (main, Feb 16 2024, 20:53:32) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> import datashader
Traceback (most recent call last):
  File "/home/peter/mambaforge/envs/lcms_polymer_env/lib/python3.11/site-packages/dask/dataframe/__init__.py", line 22, in _dask_expr_enabled
    import dask_expr  # noqa: F401
    ^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'dask_expr'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/peter/mambaforge/envs/lcms_polymer_env/lib/python3.11/site-packages/datashader/__init__.py", line 9, in <module>
    from .core import Canvas                                 # noqa (API import)
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/mambaforge/envs/lcms_polymer_env/lib/python3.11/site-packages/datashader/core.py", line 9, in <module>
    import dask.dataframe as dd
  File "/home/peter/mambaforge/envs/lcms_polymer_env/lib/python3.11/site-packages/dask/dataframe/__init__.py", line 87, in <module>
    if _dask_expr_enabled():
       ^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/mambaforge/envs/lcms_polymer_env/lib/python3.11/site-packages/dask/dataframe/__init__.py", line 24, in _dask_expr_enabled
    raise ValueError("Must install dask-expr to activate query planning.")
ValueError: Must install dask-expr to activate query planning.

Complete, minimal, self-contained example code that reproduces the issue

import datashader

• I may be interested in making a pull request to address this

[hoxbro]

Dask recently added this as a dependency (yesterday). See here. You can either install the package or run the following command dask config set dataframe.query-planning false.

I'm not currently sure what we will do in datashader.

[philippjfr]

This seems like an issue on the dask side tbh. If dask.dataframe depends on dask_expr it then it should depend on it.

[hoxbro]

He is using dask-core. Dask will install dask_expr ref

[raphaelquast]

Hey, since I just ran into the same issue...
It's true that conda install -c conda-forge dask installs dask_expr as well, but pip install dask does not !
For pip you have to explicitly specify pip install dask[dataframe] (see here)

As I see it, this would mean that pip install datashader is broken but conda install -c conda-forge datashader is not.

[hoxbro]

Does this still happen with 2024.3.1? Because of dask/dask#11003, this should now only give a warning and not an exception.

[weiji14]

Still hitting this with dask=2024.4.1. The ValueError has become an ImportError now after dask/dask#11007:

>>> import datashader
/home/user/mambaforge/envs/dev/lib/python3.11/site-packages/dask/dataframe/__init__.py:31: FutureWarning: 
Dask dataframe query planning is disabled because dask-expr is not installed.

You can install it with `pip install dask[dataframe]` or `conda install dask`.
This will raise in a future version.

  warnings.warn(msg, FutureWarning)
Traceback (most recent call last):
  File "/home/user/mambaforge/envs/dev/lib/python3.11/site-packages/dask/dataframe/__init__.py", line 97, in <module>
    import dask_expr as dd
ModuleNotFoundError: No module named 'dask_expr'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/mambaforge/envs/dev/lib/python3.11/site-packages/datashader/__init__.py", line 9, in <module>
    from .core import Canvas                                 # noqa (API import)
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/mambaforge/envs/dev/lib/python3.11/site-packages/datashader/core.py", line 9, in <module>
    import dask.dataframe as dd
  File "/home/user/mambaforge/envs/dev/lib/python3.11/site-packages/dask/dataframe/__init__.py", line 110, in <module>
    raise ImportError(msg) from e
ImportError: Dask dataframe requirements are not installed.

Please either conda or pip install as follows:

  conda install dask                     # either conda install
  python -m pip install "dask[dataframe]" --upgrade  # or python -m pip install

[hoxbro]

This will be fixed in the next dask release. See dask/dask#11048.