Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

install cache, not symlinking nvidia cuda related packages #2445

Closed
1 task done
sawyerzheng opened this issue Nov 29, 2023 · 1 comment
Closed
1 task done

install cache, not symlinking nvidia cuda related packages #2445

sawyerzheng opened this issue Nov 29, 2023 · 1 comment
Labels
🐛 bug Something isn't working

Comments

@sawyerzheng
Copy link

  • I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.
there is not error during installation, with command pdm add torch==2.1.0 -v

Steps to reproduce

  1. pdm config install.cache on
  2. pdm init
  3. pdm add torch==2.1.0
  4. pdm run python -c 'import torch; print(torch.version)'

pdm run python -c 'import torch; print(torch.__version__)'

Actual behavior

got error as following

Traceback (most recent call last):
  File "/home/sawyer/.local/share/pdm/venvs/test-install-cache-0hLEHEEK-test/lib/python3.11/site-packages/torch/__init__.py", line 174, in _load_global_deps
    ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
  File "/usr/lib/python3.11/ctypes/__init__.py", line 376, in __init__
    self._handle = _dlopen(self._name, mode)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: libcufft.so.11: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/sawyer/.local/share/pdm/venvs/test-install-cache-0hLEHEEK-test/lib/python3.11/site-packages/torch/__init__.py", line 234, in <module>
    _load_global_deps()
  File "/home/sawyer/.local/share/pdm/venvs/test-install-cache-0hLEHEEK-test/lib/python3.11/site-packages/torch/__init__.py", line 195, in _load_global_deps
    _preload_cuda_deps(lib_folder, lib_name)
  File "/home/sawyer/.local/share/pdm/venvs/test-install-cache-0hLEHEEK-test/lib/python3.11/site-packages/torch/__init__.py", line 160, in _preload_cuda_deps
    raise ValueError(f"{lib_name} not found in the system path {sys.path}")
ValueError: libcublas.so.*[0-9] not found in the system path ['', '/usr/lib/python311.zip', '/usr/lib/python3.11', '/usr/lib/python3.11/lib-dynload', '/home/sawyer/.local/share/pdm/venvs/test-install-cache-0hLEHEEK-test/lib/python3.11/site-packages']

Expected behavior

❯ pdm run python -c 'import torch; print(torch.__version__)'
2.1.0+cu121

Environment Information

# Paste the output of `pdm info && pdm info --env` below:

PDM version:
  2.10.3
Python Interpreter:
  /home/sawyer/.local/share/pdm/venvs/test-install-cache-0hLEHEEK-test/bin/python3.11 (3.11)
Project Root:
  /home/sawyer/programs/pdm/test-install-cache
Local Packages:

{
  "implementation_name": "cpython",
  "implementation_version": "3.11.5",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.15.90.1-microsoft-standard-WSL2",
  "platform_system": "Linux",
  "platform_version": "#1 SMP Fri Jan 27 02:56:13 UTC 2023",
  "python_full_version": "3.11.5",
  "platform_python_implementation": "CPython",
  "python_version": "3.11",
  "sys_platform": "linux"
}

my analysis

pdm do no symlinked cuda related packages, as in following picture, eg: nvidia-cublas package, there just a dist-info folder, not symlinked the lib folder

image

@sawyerzheng sawyerzheng added the 🐛 bug Something isn't working label Nov 29, 2023
@frostming
Copy link
Collaborator

See #1732 , this will be resolved in 2.11.0, by introducing new cache methods.

@frostming frostming closed this as not planned Won't fix, can't repro, duplicate, stale Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants