Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pillow v9.0.0 conda-forge package causes "OSError: [WinError 182] [...] error loading 鈥榗affe2_detectron_ops.dll'" #72293

Closed
joshuacwnewton opened this issue Feb 3, 2022 · 17 comments
Assignees
Labels
module: build Build system issues module: windows Windows support for PyTorch triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@joshuacwnewton
Copy link

joshuacwnewton commented Feb 3, 2022

馃悰 Describe the bug

The conda-forge package for Pillow updated from 8.4.0 to 9.0.0 today (conda-forge/pillow-feedstock#104), and as a result we began receiving the following error when trying to import torch:

C:\Miniconda\envs\ads_venv\lib\site-packages\ivadomed\inference.py:4: in <module>
    import torch
C:\Miniconda\envs\ads_venv\lib\site-packages\torch\__init__.py:123: in <module>
    raise err
E   OSError: [WinError 182] The operating system cannot run %1. Error loading "C:\Miniconda\envs\ads_venv\lib\site-packages\torch\lib\caffe2_detectron_ops.dll" or one of its dependencies.

(Sample CI run here.)

We can also confirm that pinning Pillow<9.0.0 resolves the issue. (axondeepseg/axondeepseg@370c666 and corresponding CI run)

Possibly related issues: #67145, #35803, discuss.pytorch.org#111995

NB: I imagine it's quite difficult for PyTorch to address a problem involving another package's conda release, but at the very least, I wanted to make this issue for others to find when web searching. I figured it would be better here than on the Pillow end of things? But please let me know if there's a better place for this. 馃檪

Versions

Since the error happened in our CI, the quickest way for me to report versions is by copying and pasting the versions from that run. (Apologies that I can't easily run collect_env.py.)

Output of conda info
Run conda info

     active environment : ads_venv
    active env location : C:\Miniconda\envs\ads_venv
            shell level : 2
       user config file : C:\Users\runneradmin\.condarc
 populated config files : C:\Users\runneradmin\.condarc
          conda version : 4.10.3
    conda-build version : not installed
         python version : 3.9.5.final.0
       virtual packages : __win=0=0
                          __archspec=1=x86_64
       base environment : C:\Miniconda  (writable)
      conda av data dir : C:\Miniconda\etc\conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\Users\runneradmin\conda_pkgs_dir
       envs directories : C:\Miniconda\envs
                          C:\Users\runneradmin\.conda\envs
                          C:\Users\runneradmin\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.10.3 requests/2.25.1 CPython/3.9.5 Windows/10 Windows/10.0.20348
          administrator : True
             netrc file : None
           offline mode : False

# packages in environment at C:\Miniconda\envs\ads_venv:
#
# Name                    Version                   Build  Channel
absl-py                   1.0.0                    pypi_0    pypi
alabaster                 0.7.12                     py_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
argon2-cffi               21.3.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0           py38h294d835_1    conda-forge
astor                     0.8.1                    pypi_0    pypi
asttokens                 2.0.5              pyhd8ed1ab_0    conda-forge
atomicwrites              1.4.0              pyh9f0ad1d_0    conda-forge
attrs                     21.4.0             pyhd8ed1ab_0    conda-forge
axondeepseg               4.0.0                     dev_0    <develop>
babel                     2.9.1              pyh44b312d_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
bids-validator            1.8.9                    pypi_0    pypi
black                     22.1.0             pyhd8ed1ab_0    conda-forge
bleach                    4.1.0              pyhd8ed1ab_0    conda-forge
blosc                     1.21.0               h0e60522_0    conda-forge
brotlipy                  0.7.0           py38h294d835_1003    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
c-blosc2                  2.0.4                h09319c2_1    conda-forge
ca-certificates           2021.10.8            h5b45459_0    conda-forge
cachetools                5.0.0                    pypi_0    pypi
certifi                   2021.10.8        py38haa244fe_1    conda-forge
cffi                      1.15.0           py38hd8c33c5_0    conda-forge
cfitsio                   4.0.0                hd67004f_0    conda-forge
charls                    2.2.0                h39d44d4_0    conda-forge
charset-normalizer        2.0.11             pyhd8ed1ab_0    conda-forge
click                     8.0.3            py38haa244fe_1    conda-forge
cloudpickle               2.0.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
contextlib2               21.6.0             pyhd8ed1ab_0    conda-forge
coverage                  6.3.1            py38h294d835_0    conda-forge
cryptography              36.0.1           py38hb7941b4_0    conda-forge
csv-diff                  1.1                      pypi_0    pypi
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
cytoolz                   0.11.2           py38h294d835_1    conda-forge
dask-core                 2022.1.1           pyhd8ed1ab_0    conda-forge
dataclasses               0.8                pyhc8e2a94_3    conda-forge
dcm2niix                  1.0.20211006         h5362a0b_0    conda-forge
debugpy                   1.5.1            py38h885f38d_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
deprecated                1.2.13                   pypi_0    pypi
dictdiffer                0.9.0                    pypi_0    pypi
docopt                    0.6.2                    pypi_0    pypi
docutils                  0.17.1           py38haa244fe_1    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
et_xmlfile                1.0.1                   py_1001    conda-forge
executing                 0.8.2              pyhd8ed1ab_0    conda-forge
file-tree                 0.4.1              pyhd8ed1ab_0    conda-forge
file-tree-fsl             0.2.1              pyhd8ed1ab_0    conda-forge
flit-core                 3.6.0              pyhd8ed1ab_0    conda-forge
formulaic                 0.2.4                    pypi_0    pypi
freetype                  2.10.4               h546665d_1    conda-forge
fribidi                   1.0.10               h8d14728_0    conda-forge
fsleyes                   1.3.3            py38haa244fe_0    conda-forge
fsleyes-props             1.7.3              pyhd8ed1ab_0    conda-forge
fsleyes-widgets           0.12.2             pyhd8ed1ab_0    conda-forge
fslpy                     3.8.1              pyh8a188c0_0    conda-forge
fsspec                    2022.1.0           pyhd8ed1ab_0    conda-forge
giflib                    5.2.1                h8d14728_2    conda-forge
google-auth               2.6.0                    pypi_0    pypi
google-auth-oauthlib      0.4.6                    pypi_0    pypi
grpcio                    1.43.0                   pypi_0    pypi
h5py                      2.10.0          nompi_py38he6c2248_106    conda-forge
hdf5                      1.10.6          nompi_h5268f04_1114    conda-forge
humanize                  3.14.0                   pypi_0    pypi
icu                       68.2                 h0e60522_0    conda-forge
idna                      3.3                pyhd8ed1ab_0    conda-forge
imagecodecs               2021.11.20       py38h84a8deb_1    conda-forge
imageio                   2.14.1             pyh239f2a4_0    conda-forge
imagesize                 1.3.0              pyhd8ed1ab_0    conda-forge
importlib-metadata        4.10.1           py38haa244fe_0    conda-forge
importlib_metadata        4.10.1               hd8ed1ab_0    conda-forge
importlib_resources       5.4.0              pyhd8ed1ab_0    conda-forge
indexed_gzip              1.6.4            py38h3ec5bfc_1    conda-forge
iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
intel-openmp              2022.0.0          h57928b3_3663    conda-forge
interface-meta            1.2.4                    pypi_0    pypi
ipykernel                 6.8.0            py38h595d716_0    conda-forge
ipython                   8.0.1            py38haa244fe_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.6.5              pyhd8ed1ab_0    conda-forge
isodate                   0.6.1              pyhd8ed1ab_0    conda-forge
ivadomed                  2.9.3                    pypi_0    pypi
jbig                      2.1               h8d14728_2003    conda-forge
jedi                      0.18.1           py38haa244fe_0    conda-forge
jinja2                    3.0.3              pyhd8ed1ab_0    conda-forge
joblib                    1.1.0              pyhd8ed1ab_0    conda-forge
jpeg                      9e                   h8ffe710_0    conda-forge
jsonschema                4.4.0              pyhd8ed1ab_0    conda-forge
jupyter                   1.0.0            py38haa244fe_7    conda-forge
jupyter_client            7.1.2              pyhd8ed1ab_0    conda-forge
jupyter_console           6.4.0              pyhd8ed1ab_0    conda-forge
jupyter_core              4.9.1            py38haa244fe_1    conda-forge
jupyterlab_pygments       0.1.2              pyh9f0ad1d_0    conda-forge
jupyterlab_widgets        1.0.2              pyhd8ed1ab_0    conda-forge
jxrlib                    1.1                  h8ffe710_2    conda-forge
kiwisolver                1.3.2            py38hbd9d945_1    conda-forge
krb5                      1.19.2               h20d022d_3    conda-forge
lcms2                     2.12                 h2a16943_0    conda-forge
lerc                      3.0                  h0e60522_0    conda-forge
libaec                    1.0.6                h39d44d4_0    conda-forge
libblas                   3.9.0              13_win64_mkl    conda-forge
libbrotlicommon           1.0.9                h8ffe710_6    conda-forge
libbrotlidec              1.0.9                h8ffe710_6    conda-forge
libbrotlienc              1.0.9                h8ffe710_6    conda-forge
libcblas                  3.9.0              13_win64_mkl    conda-forge
libclang                  11.1.0          default_h5c34c98_1    conda-forge
libcurl                   7.81.0               h789b8ee_0    conda-forge
libdeflate                1.8                  h8ffe710_0    conda-forge
libimagequant             2.17.0               hcfcfb64_1    conda-forge
liblapack                 3.9.0              13_win64_mkl    conda-forge
libpng                    1.6.37               h1d00b33_2    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libspatialindex           1.9.3                h39d44d4_4    conda-forge
libssh2                   1.10.0               h680486a_2    conda-forge
libtiff                   4.3.0                hd413186_2    conda-forge
libwebp                   1.2.2                h57928b3_0    conda-forge
libwebp-base              1.2.2                h8ffe710_1    conda-forge
libxcb                    1.13              hcd874cb_1004    conda-forge
libzlib                   1.2.11            h8ffe710_1013    conda-forge
libzopfli                 1.0.3                h0e60522_0    conda-forge
llvm-openmp               12.0.1               h2d74725_1    conda-forge
locket                    0.2.0                      py_2    conda-forge
loguru                    0.6.0                    pypi_0    pypi
lz4-c                     1.9.3                h8ffe710_1    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
markdown                  3.3.6                    pypi_0    pypi
markupsafe                2.0.1            py38h294d835_1    conda-forge
matplotlib                3.3.4            py38haa244fe_0    conda-forge
matplotlib-base           3.3.4            py38h34ddff4_0    conda-forge
matplotlib-inline         0.1.3              pyhd8ed1ab_0    conda-forge
mistune                   0.8.4           py38h294d835_1005    conda-forge
mkl                       2022.0.0           h0e2418a_796    conda-forge
mpld3                     0.5.7              pyhd8ed1ab_0    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
mypy_extensions           0.4.3            py38haa244fe_4    conda-forge
nbclient                  0.5.10             pyhd8ed1ab_1    conda-forge
nbconvert                 6.4.1            py38haa244fe_0    conda-forge
nbformat                  5.1.3              pyhd8ed1ab_0    conda-forge
nest-asyncio              1.5.4              pyhd8ed1ab_0    conda-forge
networkx                  2.6.3              pyhd8ed1ab_1    conda-forge
nibabel                   3.2.1              pyhd8ed1ab_0    conda-forge
notebook                  6.4.8              pyha770c72_0    conda-forge
num2words                 0.5.10                   pypi_0    pypi
numpy                     1.22.1           py38hcf66579_0    conda-forge
oauthlib                  3.2.0                    pypi_0    pypi
onnxruntime               1.7.0                    pypi_0    pypi
opencv-contrib-python     4.5.5.62                 pypi_0    pypi
opencv-python-headless    4.5.5.62                 pypi_0    pypi
openjpeg                  2.4.0                hb211442_1    conda-forge
openpyxl                  3.0.9              pyhd8ed1ab_0    conda-forge
openssl                   1.1.1l               h8ffe710_0    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pandas                    1.4.0            py38h5d928e2_0    conda-forge
pandoc                    2.17.1.1             h57928b3_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parse                     1.19.0             pyh44b312d_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
partd                     1.2.0              pyhd8ed1ab_0    conda-forge
pathlib2                  2.3.6            py38haa244fe_1    conda-forge
pathspec                  0.9.0              pyhd8ed1ab_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    9.0.0            py38hd8e0db4_0    conda-forge
pip                       22.0.3             pyhd8ed1ab_0    conda-forge
platformdirs              2.3.0              pyhd8ed1ab_0    conda-forge
pluggy                    1.0.0            py38haa244fe_2    conda-forge
pooch                     1.6.0              pyhd8ed1ab_0    conda-forge
prettytable               3.0.0              pyhd8ed1ab_0    conda-forge
progressbar2              4.0.0              pyhd8ed1ab_0    conda-forge
prometheus_client         0.13.1             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.26             pyha770c72_0    conda-forge
prompt_toolkit            3.0.26               hd8ed1ab_0    conda-forge
protobuf                  3.19.4                   pypi_0    pypi
pthread-stubs             0.4               hcd874cb_1001    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
py                        1.11.0             pyh6c4a22f_0    conda-forge
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pybids                    0.14.0                   pypi_0    pypi
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pydicom                   2.2.2              pyh6c4a22f_0    conda-forge
pygments                  2.11.2             pyhd8ed1ab_0    conda-forge
pyopengl                  3.1.5                      py_0    conda-forge
pyopenssl                 22.0.0             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7              pyhd8ed1ab_1    conda-forge
pypubsub                  4.0.3                      py_0    conda-forge
pyqt                      5.12.3           py38haa244fe_8    conda-forge
pyqt-impl                 5.12.3           py38h885f38d_8    conda-forge
pyqt5-sip                 4.19.18          py38h885f38d_8    conda-forge
pyqtchart                 5.12             py38h885f38d_8    conda-forge
pyqtwebengine             5.12.1           py38h885f38d_8    conda-forge
pyreadline                2.1             py38haa244fe_1005    conda-forge
pyrsistent                0.18.1           py38h294d835_0    conda-forge
pysocks                   1.7.1            py38haa244fe_4    conda-forge
pytest                    6.2.5            py38haa244fe_2    conda-forge
pytest-cov                3.0.0              pyhd8ed1ab_0    conda-forge
python                    3.8.12          h7840368_2_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-utils              3.1.0              pyhd8ed1ab_0    conda-forge
python_abi                3.8                      2_cp38    conda-forge
pytz                      2021.3             pyhd8ed1ab_0    conda-forge
pywavelets                1.2.0            py38h6f4d8f0_1    conda-forge
pywin32                   303              py38h294d835_0    conda-forge
pywinpty                  2.0.1            py38hd3f51b4_0    conda-forge
pyyaml                    6.0              py38h294d835_3    conda-forge
pyzmq                     22.3.0           py38h09162b1_1    conda-forge
qt                        5.12.9               h5909a2a_4    conda-forge
qtconsole                 5.2.2              pyhd8ed1ab_1    conda-forge
qtconsole-base            5.2.2              pyhd8ed1ab_1    conda-forge
qtpy                      2.0.1              pyhd8ed1ab_0    conda-forge
raven                     6.10.0                     py_0    conda-forge
requests                  2.27.1             pyhd8ed1ab_0    conda-forge
requests-oauthlib         1.3.1                    pypi_0    pypi
rsa                       4.8                      pypi_0    pypi
rtree                     0.9.7            py38h8b54edf_3    conda-forge
scikit-image              0.18.3           py38h5d928e2_1    conda-forge
scikit-learn              1.0.1            py38hb60ee80_3    conda-forge
scipy                     1.7.3            py38ha1292f7_0    conda-forge
seaborn                   0.11.2                   pypi_0    pypi
send2trash                1.8.0              pyhd8ed1ab_0    conda-forge
setuptools                60.7.1           py38haa244fe_0    conda-forge
simpleitk                 2.1.1                    pypi_0    pypi
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.8                ha925a31_3    conda-forge
snowballstemmer           2.2.0              pyhd8ed1ab_0    conda-forge
sphinx                    4.4.0              pyh6c4a22f_1    conda-forge
sphinx_rtd_theme          1.0.0              pyhd8ed1ab_0    conda-forge
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    2.0.0              pyhd8ed1ab_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.5              pyhd8ed1ab_1    conda-forge
sqlalchemy                1.3.24                   pypi_0    pypi
sqlite                    3.37.0               h8ffe710_0    conda-forge
stack_data                0.1.4              pyhd8ed1ab_0    conda-forge
tabulate                  0.8.9              pyhd8ed1ab_0    conda-forge
tbb                       2021.5.0             h2d74725_0    conda-forge
tensorboard               2.8.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.1                    pypi_0    pypi
terminado                 0.13.1           py38haa244fe_0    conda-forge
testpath                  0.5.0              pyhd8ed1ab_0    conda-forge
threadpoolctl             3.1.0              pyh8a188c0_0    conda-forge
tifffile                  2021.11.2          pyhd8ed1ab_0    conda-forge
tk                        8.6.11               h8ffe710_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.0              pyhd8ed1ab_1    conda-forge
toolz                     0.11.2             pyhd8ed1ab_0    conda-forge
torch                     1.8.1+cpu                pypi_0    pypi
torchio                   0.18.73                  pypi_0    pypi
torchvision               0.9.1+cpu                pypi_0    pypi
tornado                   6.1              py38h294d835_2    conda-forge
tqdm                      4.62.3             pyhd8ed1ab_0    conda-forge
traitlets                 5.1.1              pyhd8ed1ab_0    conda-forge
trimesh                   3.9.43             pyh6c4a22f_0    conda-forge
typed-ast                 1.5.2            py38h294d835_0    conda-forge
typing_extensions         4.0.1              pyha770c72_0    conda-forge
ucrt                      10.0.20348.0         h57928b3_0    conda-forge
urllib3                   1.26.8             pyhd8ed1ab_1    conda-forge
vc                        14.2                 hb210afc_6    conda-forge
vs2015_runtime            14.29.30037          h902a5da_6    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
webencodings              0.5.1                      py_1    conda-forge
werkzeug                  2.0.2                    pypi_0    pypi
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
widgetsnbextension        3.5.2            py38haa244fe_1    conda-forge
win32-setctime            1.1.0                    pypi_0    pypi
win_inet_pton             1.1.0            py38haa244fe_3    conda-forge
winpty                    0.4.3                         4    conda-forge
wrapt                     1.13.3                   pypi_0    pypi
wxnatpy                   0.4.0              pyhd8ed1ab_0    conda-forge
wxpython                  4.1.1            py38heb73c8a_3    conda-forge
xarray                    0.21.1             pyhd8ed1ab_0    conda-forge
xnat                      0.4.0              pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.9                hcd874cb_0    conda-forge
xorg-libxdmcp             1.1.3                hcd874cb_0    conda-forge
xz                        5.2.5                h62dcd97_1    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
zeromq                    4.3.4                h0e60522_1    conda-forge
zfp                       0.5.5                h0e60522_8    conda-forge
zipp                      3.7.0              pyhd8ed1ab_1    conda-forge
zlib                      1.2.11            h8ffe710_1013    conda-forge
zstd                      1.5.2                h6255e5f_0    conda-forge

We also diffed this output with the last passing run, and the majority of the packages versions are unchanged aside from Pillow.

Lastly, here is the Windows CI runner information:

Current runner version: '2.287.1'
Operating System
  Microsoft Windows Server 2022
  10.0.20348
  Datacenter
Virtual Environment
  Environment: windows-2022
  Version: 20220131.1
  Included Software: https://github.com/actions/virtual-environments/blob/win22/20220131.1/images/win/Windows2022-Readme.md
  Image Release: https://github.com/actions/virtual-environments/releases/tag/win22%2F20220131.1

cc @malfet @seemethere @peterjc123 @mszhanyi @skyline75489 @nbcsm

@h-vetinari
Copy link

Hey, sorry to hear that our packaging broke something.

I added a bunch of bindings for 9.0.0 that pillow offers support for (like libimagequant), but that weren't previously built-in for the conda-forge packages.

I'm surprised that these relatively minor changes seem to have broken pytorch. Do we know what caffe2_detectron_ops.dll depends on?

@metab0t
Copy link

metab0t commented Feb 4, 2022

This is caused by incompatible libiomp5md.dll from llvm-openmp and intel-openmp.
See conda-forge/conda-forge.github.io#1597 (comment)
In short, Pillow v9.0.0 depends on libimagequant which depends on llvm-openmp but pytorch depends on intel-openmp.
You can use Dependencies to compare libiomp5md.dll from llvm-openmp and intel-openmp.
caffe2_detectron_ops.dll links to some _vcomp_xxx functions which only exist in libiomp5md.dll from intel-openmp but do not exist in libiomp5md.dll from llvm-openmp, so after installing llvm-openmp and overwriting libiomp5md.dll, caffe2_detectron_ops.dll cannot be loaded correctly.
Pic

@h-vetinari
Copy link

Thanks for the investigation! I'll remove libimagequant on windows for pillow 9.0.1 for the time being (before the larger issues around llvm-openmp vs. intel-openmp can be resolved)

@metab0t
Copy link

metab0t commented Feb 4, 2022

Thanks for the investigation! I'll remove libimagequant on windows for pillow 9.0.1 for the time being (before the larger issues around llvm-openmp vs. intel-openmp can be resolved)

As a side note, don't forget to reinstall intel-openmp to retrieve its original libiomp5md.dll after llvm-openmp is uninstalled.

conda install intel-openmp --force-reinstall

@jbschlosser jbschlosser added module: build Build system issues module: windows Windows support for PyTorch triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Feb 4, 2022
@graingert
Copy link

should be fixed by conda-forge/pillow-feedstock#107 ?

joshuacwnewton added a commit to axondeepseg/axondeepseg that referenced this issue Feb 8, 2022
The issue with v9.0.0 seems to have been fixed in v9.0.1:

pytorch/pytorch#72293 (comment)
@joshuacwnewton
Copy link
Author

joshuacwnewton commented Feb 8, 2022

should be fixed by conda-forge/pillow-feedstock#107 ?

Thanks for the reminder ping! I've tried pinning pillow so that 9.0.1 will get installed (see above commit/PR), but our project is still encountering the same issues:

# Output from `conda list`
pillow                    9.0.1            py38hd8e0db4_0    conda-forge
# Identical test failures
C:\Miniconda\envs\ads_venv\lib\site-packages\ivadomed\inference.py:4: in <module>
    import torch
C:\Miniconda\envs\ads_venv\lib\site-packages\torch\__init__.py:123: in <module>
    raise err
E   OSError: [WinError 182] The operating system cannot run %1. Error loading "C:\Miniconda\envs\ads_venv\lib\site-packages\torch\lib\caffe2_detectron_ops.dll" or one of its dependencies.

CI run: https://github.com/neuropoly/axondeepseg/runs/5110622161?check_suite_focus=true

EDIT: Just to be a little more thorough, here's the output of conda info and conda list:

Run conda info

     active environment : ads_venv
    active env location : C:\Miniconda\envs\ads_venv
            shell level : 2
       user config file : C:\Users\runneradmin\.condarc
 populated config files : C:\Users\runneradmin\.condarc
          conda version : 4.10.3
    conda-build version : not installed
         python version : 3.9.5.final.0
       virtual packages : __win=0=0
                          __archspec=1=x86_64
       base environment : C:\Miniconda  (writable)
      conda av data dir : C:\Miniconda\etc\conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\Users\runneradmin\conda_pkgs_dir
       envs directories : C:\Miniconda\envs
                          C:\Users\runneradmin\.conda\envs
                          C:\Users\runneradmin\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.10.3 requests/2.25.1 CPython/3.9.5 Windows/10 Windows/10.0.20348
          administrator : True
             netrc file : None
           offline mode : False

# packages in environment at C:\Miniconda\envs\ads_venv:
#
# Name                    Version                   Build  Channel
absl-py                   1.0.0                    pypi_0    pypi
alabaster                 0.7.12                     py_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
argon2-cffi               21.3.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0           py38h294d835_1    conda-forge
astor                     0.8.1                    pypi_0    pypi
asttokens                 2.0.5              pyhd8ed1ab_0    conda-forge
atomicwrites              1.4.0              pyh9f0ad1d_0    conda-forge
attrs                     21.4.0             pyhd8ed1ab_0    conda-forge
axondeepseg               4.0.0                     dev_0    <develop>
babel                     2.9.1              pyh44b312d_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
bids-validator            1.8.9                    pypi_0    pypi
black                     22.1.0             pyhd8ed1ab_0    conda-forge
bleach                    4.1.0              pyhd8ed1ab_0    conda-forge
blosc                     1.21.0               h0e60522_0    conda-forge
brotlipy                  0.7.0           py38h294d835_1003    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
c-blosc2                  2.0.4                h09319c2_1    conda-forge
ca-certificates           2021.10.8            h5b45459_0    conda-forge
cachetools                5.0.0                    pypi_0    pypi
certifi                   2021.10.8        py38haa244fe_1    conda-forge
cffi                      1.15.0           py38hd8c33c5_0    conda-forge
cfitsio                   4.0.0                hd67004f_0    conda-forge
charls                    2.2.0                h39d44d4_0    conda-forge
charset-normalizer        2.0.11             pyhd8ed1ab_0    conda-forge
click                     8.0.3            py38haa244fe_1    conda-forge
cloudpickle               2.0.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
contextlib2               21.6.0             pyhd8ed1ab_0    conda-forge
coverage                  6.3.1            py38h294d835_0    conda-forge
cryptography              36.0.1           py38hb7941b4_0    conda-forge
csv-diff                  1.1                      pypi_0    pypi
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
cytoolz                   0.11.2           py38h294d835_1    conda-forge
dask-core                 2022.1.1           pyhd8ed1ab_0    conda-forge
dataclasses               0.8                pyhc8e2a94_3    conda-forge
dcm2niix                  1.0.20211006         h5362a0b_0    conda-forge
debugpy                   1.5.1            py38h885f38d_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
deprecated                1.2.13                   pypi_0    pypi
dictdiffer                0.9.0                    pypi_0    pypi
docopt                    0.6.2                    pypi_0    pypi
docutils                  0.17.1           py38haa244fe_1    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
et_xmlfile                1.0.1                   py_1001    conda-forge
executing                 0.8.2              pyhd8ed1ab_0    conda-forge
file-tree                 0.4.1              pyhd8ed1ab_0    conda-forge
file-tree-fsl             0.2.1              pyhd8ed1ab_0    conda-forge
flit-core                 3.6.0              pyhd8ed1ab_0    conda-forge
formulaic                 0.2.4                    pypi_0    pypi
freetype                  2.10.4               h546665d_1    conda-forge
fribidi                   1.0.10               h8d14728_0    conda-forge
fsleyes                   1.3.3            py38haa244fe_0    conda-forge
fsleyes-props             1.7.3              pyhd8ed1ab_0    conda-forge
fsleyes-widgets           0.12.2             pyhd8ed1ab_0    conda-forge
fslpy                     3.8.1              pyh8a188c0_0    conda-forge
fsspec                    2022.1.0           pyhd8ed1ab_0    conda-forge
giflib                    5.2.1                h8d14728_2    conda-forge
google-auth               2.6.0                    pypi_0    pypi
google-auth-oauthlib      0.4.6                    pypi_0    pypi
grpcio                    1.43.0                   pypi_0    pypi
h5py                      2.10.0          nompi_py38he6c2248_106    conda-forge
hdf5                      1.10.6          nompi_h5268f04_1114    conda-forge
humanize                  3.14.0                   pypi_0    pypi
icu                       68.2                 h0e60522_0    conda-forge
idna                      3.3                pyhd8ed1ab_0    conda-forge
imagecodecs               2021.11.20       py38h84a8deb_1    conda-forge
imageio                   2.14.1             pyh239f2a4_0    conda-forge
imagesize                 1.3.0              pyhd8ed1ab_0    conda-forge
importlib-metadata        4.10.1           py38haa244fe_0    conda-forge
importlib_metadata        4.10.1               hd8ed1ab_0    conda-forge
importlib_resources       5.4.0              pyhd8ed1ab_0    conda-forge
indexed_gzip              1.6.4            py38h3ec5bfc_1    conda-forge
iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
intel-openmp              2022.0.0          h57928b3_3663    conda-forge
interface-meta            1.2.4                    pypi_0    pypi
ipykernel                 6.9.0            py38h595d716_0    conda-forge
ipython                   8.0.1            py38haa244fe_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.6.5              pyhd8ed1ab_0    conda-forge
isodate                   0.6.1              pyhd8ed1ab_0    conda-forge
ivadomed                  2.9.3                    pypi_0    pypi
jbig                      2.1               h8d14728_2003    conda-forge
jedi                      0.18.1           py38haa244fe_0    conda-forge
jinja2                    3.0.3              pyhd8ed1ab_0    conda-forge
joblib                    1.1.0              pyhd8ed1ab_0    conda-forge
jpeg                      9e                   h8ffe710_0    conda-forge
jsonschema                4.4.0              pyhd8ed1ab_0    conda-forge
jupyter                   1.0.0            py38haa244fe_7    conda-forge
jupyter_client            7.1.2              pyhd8ed1ab_0    conda-forge
jupyter_console           6.4.0              pyhd8ed1ab_0    conda-forge
jupyter_core              4.9.1            py38haa244fe_1    conda-forge
jupyterlab_pygments       0.1.2              pyh9f0ad1d_0    conda-forge
jupyterlab_widgets        1.0.2              pyhd8ed1ab_0    conda-forge
jxrlib                    1.1                  h8ffe710_2    conda-forge
kiwisolver                1.3.2            py38hbd9d945_1    conda-forge
krb5                      1.19.2               h20d022d_3    conda-forge
lcms2                     2.12                 h2a16943_0    conda-forge
lerc                      3.0                  h0e60522_0    conda-forge
libaec                    1.0.6                h39d44d4_0    conda-forge
libblas                   3.9.0              13_win64_mkl    conda-forge
libbrotlicommon           1.0.9                h8ffe710_6    conda-forge
libbrotlidec              1.0.9                h8ffe710_6    conda-forge
libbrotlienc              1.0.9                h8ffe710_6    conda-forge
libcblas                  3.9.0              13_win64_mkl    conda-forge
libclang                  11.1.0          default_h5c34c98_1    conda-forge
libcurl                   7.81.0               h789b8ee_0    conda-forge
libdeflate                1.8                  h8ffe710_0    conda-forge
libimagequant             2.17.0               hcfcfb64_1    conda-forge
liblapack                 3.9.0              13_win64_mkl    conda-forge
libpng                    1.6.37               h1d00b33_2    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libspatialindex           1.9.3                h39d44d4_4    conda-forge
libssh2                   1.10.0               h680486a_2    conda-forge
libtiff                   4.3.0                hd413186_2    conda-forge
libwebp                   1.2.2                h57928b3_0    conda-forge
libwebp-base              1.2.2                h8ffe710_1    conda-forge
libxcb                    1.13              hcd874cb_1004    conda-forge
libzlib                   1.2.11            h8ffe710_1013    conda-forge
libzopfli                 1.0.3                h0e60522_0    conda-forge
llvm-openmp               12.0.1               h2d74725_1    conda-forge
locket                    0.2.0                      py_2    conda-forge
loguru                    0.6.0                    pypi_0    pypi
lz4-c                     1.9.3                h8ffe710_1    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
markdown                  3.3.6                    pypi_0    pypi
markupsafe                2.0.1            py38h294d835_1    conda-forge
matplotlib                3.3.4            py38haa244fe_0    conda-forge
matplotlib-base           3.3.4            py38h34ddff4_0    conda-forge
matplotlib-inline         0.1.3              pyhd8ed1ab_0    conda-forge
mistune                   0.8.4           py38h294d835_1005    conda-forge
mkl                       2022.0.0           h0e2418a_796    conda-forge
mpld3                     0.5.7              pyhd8ed1ab_0    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
mypy_extensions           0.4.3            py38haa244fe_4    conda-forge
nbclient                  0.5.10             pyhd8ed1ab_1    conda-forge
nbconvert                 6.4.1            py38haa244fe_0    conda-forge
nbformat                  5.1.3              pyhd8ed1ab_0    conda-forge
nest-asyncio              1.5.4              pyhd8ed1ab_0    conda-forge
networkx                  2.6.3              pyhd8ed1ab_1    conda-forge
nibabel                   3.2.2              pyhd8ed1ab_0    conda-forge
notebook                  6.4.8              pyha770c72_0    conda-forge
num2words                 0.5.10                   pypi_0    pypi
numpy                     1.22.2           py38hcf66579_0    conda-forge
oauthlib                  3.2.0                    pypi_0    pypi
onnxruntime               1.7.0                    pypi_0    pypi
opencv-contrib-python     4.5.5.62                 pypi_0    pypi
opencv-python-headless    4.5.5.62                 pypi_0    pypi
openjpeg                  2.4.0                hb211442_1    conda-forge
openpyxl                  3.0.9              pyhd8ed1ab_0    conda-forge
openssl                   1.1.1l               h8ffe710_0    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pandas                    1.4.0            py38h5d928e2_0    conda-forge
pandoc                    2.17.1.1             h57928b3_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parse                     1.19.0             pyh44b312d_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
partd                     1.2.0              pyhd8ed1ab_0    conda-forge
pathlib2                  2.3.6            py38haa244fe_1    conda-forge
pathspec                  0.9.0              pyhd8ed1ab_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    9.0.1            py38hd8e0db4_0    conda-forge
pip                       22.0.3             pyhd8ed1ab_0    conda-forge
platformdirs              2.4.1              pyhd8ed1ab_1    conda-forge
pluggy                    1.0.0            py38haa244fe_2    conda-forge
pooch                     1.6.0              pyhd8ed1ab_0    conda-forge
prettytable               3.0.0              pyhd8ed1ab_0    conda-forge
progressbar2              4.0.0              pyhd8ed1ab_0    conda-forge
prometheus_client         0.13.1             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.26             pyha770c72_0    conda-forge
prompt_toolkit            3.0.26               hd8ed1ab_0    conda-forge
protobuf                  3.19.4                   pypi_0    pypi
pthread-stubs             0.4               hcd874cb_1001    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
py                        1.11.0             pyh6c4a22f_0    conda-forge
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pybids                    0.14.0                   pypi_0    pypi
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pydicom                   2.2.2              pyh6c4a22f_0    conda-forge
pygments                  2.11.2             pyhd8ed1ab_0    conda-forge
pyopengl                  3.1.5                      py_0    conda-forge
pyopenssl                 22.0.0             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7              pyhd8ed1ab_1    conda-forge
pypubsub                  4.0.3                      py_0    conda-forge
pyqt                      5.12.3           py38haa244fe_8    conda-forge
pyqt-impl                 5.12.3           py38h885f38d_8    conda-forge
pyqt5-sip                 4.19.18          py38h885f38d_8    conda-forge
pyqtchart                 5.12             py38h885f38d_8    conda-forge
pyqtwebengine             5.12.1           py38h885f38d_8    conda-forge
pyreadline                2.1             py38haa244fe_1005    conda-forge
pyrsistent                0.18.1           py38h294d835_0    conda-forge
pysocks                   1.7.1            py38haa244fe_4    conda-forge
pytest                    7.0.0            py38haa244fe_0    conda-forge
pytest-cov                3.0.0              pyhd8ed1ab_0    conda-forge
python                    3.8.12          h7840368_2_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-utils              3.1.0              pyhd8ed1ab_0    conda-forge
python_abi                3.8                      2_cp38    conda-forge
pytz                      2021.3             pyhd8ed1ab_0    conda-forge
pywavelets                1.2.0            py38h6f4d8f0_1    conda-forge
pywin32                   303              py38h294d835_0    conda-forge
pywinpty                  2.0.2            py38hd3f51b4_0    conda-forge
pyyaml                    6.0              py38h294d835_3    conda-forge
pyzmq                     22.3.0           py38h09162b1_1    conda-forge
qt                        5.12.9               h5909a2a_4    conda-forge
qtconsole                 5.2.2              pyhd8ed1ab_1    conda-forge
qtconsole-base            5.2.2              pyhd8ed1ab_1    conda-forge
qtpy                      2.0.1              pyhd8ed1ab_0    conda-forge
raven                     6.10.0                     py_0    conda-forge
requests                  2.27.1             pyhd8ed1ab_0    conda-forge
requests-oauthlib         1.3.1                    pypi_0    pypi
rsa                       4.8                      pypi_0    pypi
rtree                     0.9.7            py38h8b54edf_3    conda-forge
scikit-image              0.18.3           py38h5d928e2_1    conda-forge
scikit-learn              1.0.1            py38hb60ee80_3    conda-forge
scipy                     1.8.0            py38ha1292f7_0    conda-forge
seaborn                   0.11.2                   pypi_0    pypi
send2trash                1.8.0              pyhd8ed1ab_0    conda-forge
setuptools                60.8.1           py38haa244fe_0    conda-forge
simpleitk                 2.1.1                    pypi_0    pypi
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.8                ha925a31_3    conda-forge
snowballstemmer           2.2.0              pyhd8ed1ab_0    conda-forge
sphinx                    4.4.0              pyh6c4a22f_1    conda-forge
sphinx_rtd_theme          1.0.0              pyhd8ed1ab_0    conda-forge
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    2.0.0              pyhd8ed1ab_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.5              pyhd8ed1ab_1    conda-forge
sqlalchemy                1.3.24                   pypi_0    pypi
sqlite                    3.37.0               h8ffe710_0    conda-forge
stack_data                0.1.4              pyhd8ed1ab_0    conda-forge
tabulate                  0.8.9              pyhd8ed1ab_0    conda-forge
tbb                       2021.5.0             h2d74725_0    conda-forge
tensorboard               2.8.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.1                    pypi_0    pypi
terminado                 0.13.1           py38haa244fe_0    conda-forge
testpath                  0.5.0              pyhd8ed1ab_0    conda-forge
threadpoolctl             3.1.0              pyh8a188c0_0    conda-forge
tifffile                  2022.2.2           pyhd8ed1ab_0    conda-forge
tk                        8.6.11               h8ffe710_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.0              pyhd8ed1ab_1    conda-forge
toolz                     0.11.2             pyhd8ed1ab_0    conda-forge
torch                     1.8.1+cpu                pypi_0    pypi
torchio                   0.18.73                  pypi_0    pypi
torchvision               0.9.1+cpu                pypi_0    pypi
tornado                   6.1              py38h294d835_2    conda-forge
tqdm                      4.62.3             pyhd8ed1ab_0    conda-forge
traitlets                 5.1.1              pyhd8ed1ab_0    conda-forge
trimesh                   3.9.43             pyh6c4a22f_0    conda-forge
typed-ast                 1.5.2            py38h294d835_0    conda-forge
typing_extensions         4.0.1              pyha770c72_0    conda-forge
ucrt                      10.0.20348.0         h57928b3_0    conda-forge
urllib3                   1.26.8             pyhd8ed1ab_1    conda-forge
vc                        14.2                 hb210afc_6    conda-forge
vs2015_runtime            14.29.30037          h902a5da_6    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
webencodings              0.5.1                      py_1    conda-forge
werkzeug                  2.0.3                    pypi_0    pypi
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
widgetsnbextension        3.5.2            py38haa244fe_1    conda-forge
win32-setctime            1.1.0                    pypi_0    pypi
win_inet_pton             1.1.0            py38haa244fe_3    conda-forge
winpty                    0.4.3                         4    conda-forge
wrapt                     1.13.3                   pypi_0    pypi
wxnatpy                   0.4.0              pyhd8ed1ab_0    conda-forge
wxpython                  4.1.1            py38heb73c8a_3    conda-forge
xarray                    0.21.1             pyhd8ed1ab_0    conda-forge
xnat                      0.4.0              pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.9                hcd874cb_0    conda-forge
xorg-libxdmcp             1.1.3                hcd874cb_0    conda-forge
xz                        5.2.5                h62dcd97_1    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
zeromq                    4.3.4                h0e60522_1    conda-forge
zfp                       0.5.5                h0e60522_8    conda-forge
zipp                      3.7.0              pyhd8ed1ab_1    conda-forge
zlib                      1.2.11            h8ffe710_1013    conda-forge
zstd                      1.5.2                h6255e5f_0    conda-forge

Looks like llvm-openmp is still getting installed somehow.

@graingert
Copy link

graingert commented Feb 8, 2022

@joshuacwnewton thanks I'm also getting the same issue still: dask/distributed#5775 (review)

@graingert
Copy link

cc @h-vetinari

@h-vetinari
Copy link

Yeah, that was my bad, I changed the dependency specification only for the host and not runtime. Fix is here. Currently azure is unhappy for some reason in all of conda-forge, but once that is figured out, the new builds will be up in a matter of hours.

@h-vetinari
Copy link

h-vetinari commented Feb 9, 2022

I'll put an automerge label on it, so you'll even be able to retrigger it in my absence, by the posting magic invocation of
@conda-forge-admin, please restart ci
(without formatting) in a comment.

@malfet
Copy link
Contributor

malfet commented Feb 16, 2022

Please note, that caffe2 support is disabled by default, i.e. there would be no "caffe2_detectron_ops.dll" in next release of pytorch

@joshuacwnewton
Copy link
Author

joshuacwnewton commented Feb 24, 2022

Small update: I tried re-running our CI, and managed to get a passing run (i.e. no OSError).

The pillow version (v9.0.1) remains unchanged, but the conda build ID has updated from py38hd8e0db4_0 to py38hd8e0db4_2. (See: diff between packages for failing vs. passing run --> the passing run has pillow==9.0.1 but no libimagequant or llvm-openmp).

I presume the updated fix was quietly folded into the same patch version of pillow? (Please correct me if I'm wrong, as perhaps the fix came from somewhere else in the conda ecosystem.) Regardless, this issue seems to be resolved on our end.

mathieuboudreau pushed a commit to axondeepseg/axondeepseg that referenced this issue Mar 2, 2022
The issue with v9.0.0 seems to have been fixed in v9.0.1:

pytorch/pytorch#72293 (comment)
@mantaionut
Copy link
Collaborator

mantaionut commented Dec 12, 2022

When using Conda PyTorch version with CAFFE2 the libiomp5md.dll is not included in the torch\lib while its included in the pip package. So this issue is only happening when using Conda. To reproduce this:

conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cpuonly -c pytorch
conda install llvm-openmp
python -c "import torch"

@h-vetinari
Copy link

the libiomp5md.dll is not included in the torch\lib while its included in the pip package.

That's because conda does not stuff non-python libraries into python-specific folders. That's a core part of the design, and intentionally different from pip (which has no concept of non-python dependencies).

So this issue is only happening when using Conda. To reproduce this:

This issue only happens when mixing pip with conda, or conda-forge with other channels (e.g. your -c pytorch). Due to untestable nature brought about by the combinatorial explosion of installing different packages in a variety of ways, conda-forge only really supports taking all (available) packages from conda-forge (i.e. --channel-priority=strict) - all other combinations work at best coincidentally.

@mantaionut
Copy link
Collaborator

@joshuacwnewton I suspect this is a conda issue since it's about a conflict between intel and llvm openmp.
Could you open an issue in the conda repo?
I think this is the best place where you could open it https://github.com/ContinuumIO/anaconda-issues/issues.

@joshuacwnewton
Copy link
Author

@mantaionut I'm not sure if I'm the best person to re-report? I'm just a downstream user, and admittedly I don't have a good understanding of the underlying problem here, since I moved on after the resolution from #72293 (comment).

I also don't have any insight into e.g. why this was moved to "Blocked" on the PyTorch On Windows project board? So, it's hard for me to summarize what has happened here in the year since filing this issue.

@mantaionut
Copy link
Collaborator

I've created an issue in conda ContinuumIO/anaconda-issues#13146. So I'll close this issue here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: build Build system issues module: windows Windows support for PyTorch triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
Status: Done
Development

No branches or pull requests

8 participants