Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create show_versions.py #9144

Merged
merged 5 commits into from Jun 24, 2022
Merged

Create show_versions.py #9144

merged 5 commits into from Jun 24, 2022

Conversation

SultanOrazbayev
Copy link
Contributor

@SultanOrazbayev SultanOrazbayev commented May 28, 2022

For now the import is not integrated into __init__.py, not sure if that's feasible/reasonable.

In [2]: from dask.utils import show_versions

In [3]: show_versions()
{
  "Python": "3.10.4",
  "Platform": "Linux",
  "dask": "2022.5.2+8.geb5f4e200",
  "distributed": "2022.5.2",
  "numpy": "1.22.3",
  "pandas": "1.4.2",
  "cloudpickle": "2.1.0",
  "fsspec": "2022.5.0",
  "bokeh": "2.4.3",
  "fastparquet": null,
  "pyarrow": null,
  "zarr": null
}

@GPUtester
Copy link
Collaborator

Can one of the admins verify this patch?

Copy link
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @SultanOrazbayev! Overall this looks like things are moving in the right direct

Instead of adding a new module, could you just add show_versions to dask/utils.py? This is a relatively small utility that I don't think merits its own module

dask/show_versions.py Outdated Show resolved Hide resolved
dask/show_versions.py Outdated Show resolved Hide resolved
dask/show_versions.py Outdated Show resolved Hide resolved
dask/show_versions.py Outdated Show resolved Hide resolved
Copy link
Member

@jsignell jsignell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird issue, but I tried this out locally and it is giving a different version than dask.__version__

In [1]: import dask
   ...: from dask.utils import show_versions

In [2]: show_versions()
{
  "Python": "3.9.10",
  "Platform": "Linux",
  "dask": "2022.4.0",
  "distributed": "2022.4.0",
  "numpy": "1.22.3",
  "pandas": "1.4.2",
  "cloudpickle": "2.0.0",
  "fsspec": "2022.1.0",
  "bokeh": "2.4.2",
  "fastparquet": "0.8.0",
  "pyarrow": "6.0.1",
  "zarr": "2.10.3"
}
In [3]: dask.__version__
Out[3]: '2022.05.2+10.g0b9963154.dirty'

@SultanOrazbayev
Copy link
Contributor Author

This seems to be due to running it from a folder that contains dask (re: module search path). Not sure if this exception should be handled explicitly...

@jsignell
Copy link
Member

jsignell commented Jun 7, 2022

This seems to be due to running it from a folder that contains dask (re: module search path). Not sure if this exception should be handled explicitly...

I get that result in my dev env no matter where I run it from. It could be something about using pip install -e I suppose. But yes it would be very good to figure out what's going on and handle it explicitly.

@SultanOrazbayev
Copy link
Contributor Author

I'll try to reproduce, are you using codespaces for this by any chance?(trying to make sure I get a similar setup)

@jsignell
Copy link
Member

jsignell commented Jun 7, 2022

This is just in an ipython or python prompt in my terminal. The conda env that I have activated was created according to the developer guide (https://docs.dask.org/en/stable/develop.html#development-environment)

@SultanOrazbayev
Copy link
Contributor Author

SultanOrazbayev commented Jun 8, 2022

Here's what I did to reproduce:

git clone git@github.com:SultanOrazbayev/dask.git
cd dask
conda env create -n dask-dev -f continuous_integration/environment-3.9.yaml
# a few moments later
conda activate dask-dev

Now, let's compare the importlib.metadata and dask.__version__:

python -c 'from importlib.metadata import version; print(version("dask"))'
# 2022.5.2

python -c 'import dask; print(dask.__version__)'
# 2022.05.2+6.g56eeb0610

To check that this is due to module search problem, let's quickly test in some nested directory:

mkdir test_directory
cd test_directory
python -c 'import dask; print(dask.__version__)'
# 2022.05.2
python -c 'from importlib.metadata import version; print(version("dask"))'
# 2022.5.2

# clean-up
cd .. && rm -vr test_directory

So I can definitely reproduce this problem without getting to pip install -e . step. How to tackle this situation? Would checking if there is a folder with name dask and __init__.py in it be sufficient?

(edit: sorry had to repost the comment due to copy-paste error)

@jsignell
Copy link
Member

jsignell commented Jun 8, 2022

Can't we just do from dask import __version__ and use that? I looked at what bokeh is doing in their info command and it looks like that is what they are doing:

https://github.com/bokeh/bokeh/blob/fb1a54cd1852b762d948a76e9dec79f6b1edf89e/bokeh/command/subcommands/info.py#L125-L132

@ncclementi
Copy link
Member

@SultanOrazbayev have you had the chance to look at what Julia suggested in the comment above?

@SultanOrazbayev
Copy link
Contributor Author

@ncclementi Yes, Julia's suggestion will work... I wasn't sure if that's the way to proceed, though. (no particular reservations, just not sure if there's a 'best practice' to deal with this, as it seems different projects handle it differently...)

If that's the preferred way forward, kindly confirm and I will change the code to use __version__ approach.

@ncclementi
Copy link
Member

@jsignell since you made the suggestion to go with from dask import __version__ what do you think about Sultan's comment above? Should we go with this approach?

@jsignell
Copy link
Member

I think for dask and distributed we should feel free to use __version__ that will definitely not be going anywhere and it's important to get those versions right!

@SultanOrazbayev
Copy link
Contributor Author

Now dask and distributed use __version__, the whiteline between the imports looks odd, but that's what black/isort did. Here are the outputs I get:

python -c 'import dask; print(dask.__version__)'
# 2022.05.2+11.g4091fba32

python -c 'import dask; dask.utils.show_versions()'
# {
#   "Python": "3.10.4",
#   "Platform": "Linux",
#   "dask": "2022.05.2+11.g4091fba32",
#   "distributed": "2022.5.2",
#   "numpy": "1.22.3",
#   "pandas": "1.4.2",
#   "cloudpickle": "2.1.0",
#   "fsspec": "2022.5.0",
#   "bokeh": "2.4.3",
#   "fastparquet": null,
#   "pyarrow": null,
#   "zarr": null
# }

Copy link
Member

@jsignell jsignell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Thanks @SultanOrazbayev for sticking with this!

@jsignell jsignell merged commit 8f5d2c1 into dask:main Jun 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improving bug report template
5 participants