New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make JUPYTER_PREFER_ENV_PATH=1 default #283
Comments
The main issues I saw with making JUPYTER_PREFER_ENV_PATH set by default are:
Does anyone know of a way to reliably tell if sys.prefix comes from a virtual environment or not? On the other points:
That already happens in JupyterLab - extensions in earlier directories in the path override extensions in later directories.
I think you can set JUPYTER_CONFIG_DIR and JUPYTER_DATA_DIR to override the user-level directories - possibly setting them to empty effectively turns off the user level. I don't know of a way to turn off the system level (and not sure there is a real-world use case in practice). |
I'm confused by this. Does this mean you are overriding sys.prefix? If you are operating in a virtual environment sys.prefix should point to the virtual environment. Is there a way that it might not (other than the user specifically setting it otherwise). Thus, I believe the sys.prefix should take precedence. From the python documentation: sys.prefix A string giving the site-specific directory prefix where the platform independent Python files are installed; on Unix, the default is '/usr/local'. This can be set at build time with the --prefix argument to the configure script. See Installation paths for derived paths. Note If a virtual environment is in effect, this value will be changed in site.py to point to the virtual environment. The value for the Python installation will still be available, via base_prefix. |
No, what I mean is that sometimes sys.prefix is intended to be more specific than the user-level directory (for example, if a single person is using multiple virtual environments), and other times sys.prefix is intended to be less specific than the user-level directory (for example, if you are not using a virtual environment, sys.prefix points to a system location like One heuristic is to check if sys.prefix and the user-level directory share a common prefix, i.e., see if the sys.prefix points to a directory inside the user's home directory. That heuristic assumes the path location indicates the precedence. I don't know if that heuristic would be reliable enough to make default. For example, a user might change the user-level path to something outside their home directory, or might be using virtual environments based out of a directory outside the home directory. |
From the docs you quoted, another heuristic might be to examine sys.base_prefix and sys.prefix. If they are different, assume we are running in a virtual environment and make sys.prefix more specific than user-level directories. This has the following issues:
|
I guess I am not understanding what the problem is. If someone is not using a virtual environment, sys.prefix should point to the proper directory to look for things in. If they are using a virtual environment it should point to the proper directory. Is the issue how to climb the tree above that looking for things?
Even in this case, I do not understand the problem. This probably means I do not understand what the code is doing with the directory tree. |
I also note this issue about platform dependent directories #234. Does part of the problem surround trying to account for platform dependent differences in the jupyter_core? |
@gutow, as I understand it, the issue is that on a shared system, the order of specificity is: system, user, virtual/conda env. One thing we could do to detect if we are in a virtual/conda env is look for |
That sounds like a reasonable default for the virtual env solutions we know about. I assume mamba sets the CONDA_PREFIX env variable? |
Yes, I only use |
If this works, I think that would provide the behavior most would expect of their virtual environments. |
And inside a virtual env: >>> import sys
>>> sys.prefix
'/private/tmp/foo'
>>> sys.base_prefix
'/Users/steve.silvester/miniconda' |
That looks like a venv inside a conda virtual env :) |
We probably also want to check that sys.prefix starts with CONDA_PREFIX, since we might be in a conda env without a python interpreter (like an R conda env, etc.). |
Oh, interesting, yeah, that makes sense. |
…edence accordingly. We make the assumption that if we are in a virtual environment, we should prioritize the environment-level sys.prefix over the user-level paths. The user can opt out of this behavior by setting JUPYTER_PREFER_ENV_PATH, which takes precedence over our autodetection. Fixes jupyter#283
I took a preliminary stab at this in #286 - anyone feel free to take over it. |
I propose that
JUPYTER_PREFER_ENV_PATH=1
be made the default behavior.I cannot imagine a case where someone would set up a virtual environment and not expect the python based software such as Jupyter lab to use what is installed in the virtual environment first. Thus, I believe Jupyter lab should never use versions of extensions, etc outside of the virtual environment in preference to those inside. I can imagine people wanting to set up some utilities that work in all environments. So, can see there might be issues surrounding that. I suggest the following logic:
I encountered an unexpected issue with ipywidgets because of this (see jupyter-widgets/ipywidgets#3559).
The original implementation was discussed in #199
Thanks for a great tool.
Jonathan
The text was updated successfully, but these errors were encountered: