[Bug]: Errors when trying to install on m1 max

[Roblouisck]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What happened?

Getting ERROR: Failed building wheel for basicsr

Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting oauthlib>=3.0.0
  Using cached oauthlib-3.2.2-py3-none-any.whl (151 kB)
Building wheels for collected packages: basicsr
  Building wheel for basicsr (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [30 lines of output]
      /Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
        warnings.warn(
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-xty3eb57/basicsr_5e2e5664514d41fea39d216d15a004d2/setup.py", line 147, in <module>
          setup(
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup
          _install_setup_requires(attrs)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/__init__.py", line 80, in _install_setup_requires
          dist.fetch_build_eggs(dist.setup_requires)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/dist.py", line 875, in fetch_build_eggs
          resolved_dists = pkg_resources.working_set.resolve(
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/pkg_resources/__init__.py", line 789, in resolve
          dist = best[req.key] = env.best_match(
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1075, in best_match
          return self.obtain(req, installer)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1087, in obtain
          return installer(requirement)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/dist.py", line 945, in fetch_build_egg
          return fetch_build_egg(self, req)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/installer.py", line 87, in fetch_build_egg
          wheel.install_as_egg(dist_location)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/wheel.py", line 110, in install_as_egg
          self._install_as_egg(destination_eggdir, zf)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/wheel.py", line 118, in _install_as_egg
          self._convert_metadata(zf, destination_eggdir, dist_info, egg_info)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/wheel.py", line 162, in _convert_metadata
          os.rename(dist_info, egg_info)
      OSError: [Errno 66] Directory not empty: '/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-xty3eb57/basicsr_5e2e5664514d41fea39d216d15a004d2/.eggs/torch-1.13.0-py3.10-macosx-11.1-arm64.egg/torch-1.13.0.dist-info' -> '/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-xty3eb57/basicsr_5e2e5664514d41fea39d216d15a004d2/.eggs/torch-1.13.0-py3.10-macosx-11.1-arm64.egg/EGG-INFO'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for basicsr
  Running setup.py clean for basicsr
  error: subprocess-exited-with-error
  
  × python setup.py clean did not run successfully.
  │ exit code: 1
  ╰─> [30 lines of output]
      /Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
        warnings.warn(
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-xty3eb57/basicsr_5e2e5664514d41fea39d216d15a004d2/setup.py", line 147, in <module>
          setup(
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup
          _install_setup_requires(attrs)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/__init__.py", line 80, in _install_setup_requires
          dist.fetch_build_eggs(dist.setup_requires)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/dist.py", line 875, in fetch_build_eggs
          resolved_dists = pkg_resources.working_set.resolve(
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/pkg_resources/__init__.py", line 789, in resolve
          dist = best[req.key] = env.best_match(
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1075, in best_match
          return self.obtain(req, installer)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1087, in obtain
          return installer(requirement)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/dist.py", line 945, in fetch_build_egg
          return fetch_build_egg(self, req)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/installer.py", line 87, in fetch_build_egg
          wheel.install_as_egg(dist_location)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/wheel.py", line 110, in install_as_egg
          self._install_as_egg(destination_eggdir, zf)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/wheel.py", line 118, in _install_as_egg
          self._convert_metadata(zf, destination_eggdir, dist_info, egg_info)
        File "/Users/robert/miniconda/envs/web-ui/lib/python3.10/site-packages/setuptools/wheel.py", line 162, in _convert_metadata
          os.rename(dist_info, egg_info)
      OSError: [Errno 66] Directory not empty: '/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-xty3eb57/basicsr_5e2e5664514d41fea39d216d15a004d2/.eggs/torch-1.13.0-py3.10-macosx-11.1-arm64.egg/torch-1.13.0.dist-info' -> '/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-xty3eb57/basicsr_5e2e5664514d41fea39d216d15a004d2/.eggs/torch-1.13.0-py3.10-macosx-11.1-arm64.egg/EGG-INFO'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed cleaning build dir for basicsr
Failed to build basicsr
Installing collected packages: yapf, tokenizers, tensorboard-plugin-wit, rfc3986, resize-right, pytz, pydub, pyasn1, mpmath, lmdb, lark, fonts, font-roboto, flatbuffers, ffmpy, antlr4-python3-runtime, addict, zipp, websockets, urllib3, uc-micro-py, typing-extensions, tqdm, tensorboard-data-server, sympy, sniffio, smmap, six, rsa, regex, pyyaml, pyrsistent, pyparsing, pyDeprecate, pycryptodome, pycparser, pyasn1-modules, protobuf, Pillow, piexif, orjson, oauthlib, numpy, networkx, multidict, mdurl, MarkupSafe, markdown, llvmlite, kiwisolver, inflection, idna, humanfriendly, h11, future, fsspec, frozenlist, fonttools, filelock, einops, cycler, click, charset-normalizer, chardet, cachetools, bcrypt, attrs, async-timeout, absl-py, yarl, werkzeug, uvicorn, torch, tifffile, scipy, requests, PyWavelets, python-multipart, python-dateutil, pydantic, packaging, opencv-python, onnx, omegaconf, numba, markdown-it-py, linkify-it-py, jsonschema, jinja2, importlib-metadata, imageio, grpcio, google-auth, gitdb, contourpy, coloredlogs, cffi, anyio, aiosignal, torchvision, torchmetrics, torchdiffeq, starlette, scikit-image, requests-oauthlib, pynacl, pandas, onnxruntime, mdit-py-plugins, matplotlib, kornia, jsonmerge, huggingface-hub, httpcore, GitPython, fairscale, cryptography, aiohttp, transformers, timm, paramiko, invisible-watermark, httpx, google-auth-oauthlib, filterpy, fastapi, diffusers, clean-fid, tensorboard, tb-nightly, gradio, facexlib, pytorch_lightning, basicsr, gfpgan, realesrgan

Followed by

=============================================
Traceback (most recent call last):
  File "/Users/robert/Documents/stable-diffusion-webui/webui.py", line 13, in <module>
    from modules import devices, sd_samplers, upscaler, extensions, localization
  File "/Users/robert/Documents/stable-diffusion-webui/modules/sd_samplers.py", line 8, in <module>
    import k_diffusion.sampling
  File "/Users/robert/Documents/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/__init__.py", line 1, in <module>
    from . import augmentation, config, evaluation, external, gns, layers, models, sampling, utils
  File "/Users/robert/Documents/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 6, in <module>
    from . import sampling, utils
  File "/Users/robert/Documents/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/sampling.py", line 7, in <module>
    import torchsde
ModuleNotFoundError: No module named 'torchsde'
robert@Roberts-MacBook-Pro Documents % 

Steps to reproduce the problem

run setup_mac.sh on mac m1 max

What should have happened?

should have finished downloading

Commit where the problem happens

4bbe535

What platforms do you use to access UI ?

No response

What browsers do you use to access the UI ?

No response

Command Line Arguments

No response

Additional information, context and logs

No response

[ivan-volchenskov]

This worked for me
#4536 (comment)

[bad1313]

I had the same error.
Seems like torchsde is not installed by the setup_mac.sh
I can't open an issue with dylancl unfortunately.

What I did to solve the issue was the following:
In terminal:
conda activate web-ui
pip install torchsde

Then start as usual using run_webui_mac.sh. Maybe this works. Maybe we need to explicitly choose the version for torchsde.

[Roblouisck]

Getting some new errors now after using that solution

My python version is 3.9.12, yet it's still using python 2. Even after I switched all the lines in setup_mac.sh from python --> python3.

All files are in place. Continuing installation.
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting basicsr
  Using cached basicsr-1.4.2.tar.gz (172 kB)
    ERROR: Command errored out with exit status 1:
     command: /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-0Kaqq2/basicsr/setup.py'"'"'; __file__='"'"'/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-0Kaqq2/basicsr/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-pip-egg-info-OnC1QT
         cwd: /private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-0Kaqq2/basicsr/
    Complete output (6 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-0Kaqq2/basicsr/setup.py", line 69
        VERSION_INFO = ', '.join([x if x.isdigit() else f'"{x}"' for x in SHORT_VERSION.split('.')])
                                                               ^
    SyntaxError: invalid syntax
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting git+https://github.com/openai/CLIP.git@d50d76daa670286dd6cacf3bcd80b5e4823fc8e1
  Cloning https://github.com/openai/CLIP.git (to revision d50d76daa670286dd6cacf3bcd80b5e4823fc8e1) to /private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-req-build-OVqS1k
  Running command git clone -q https://github.com/openai/CLIP.git /private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-req-build-OVqS1k
  Running command git rev-parse -q --verify 'sha^d50d76daa670286dd6cacf3bcd80b5e4823fc8e1'
  Running command git fetch -q https://github.com/openai/CLIP.git d50d76daa670286dd6cacf3bcd80b5e4823fc8e1
Collecting ftfy
  Using cached ftfy-5.3.0.tar.gz (53 kB)
    ERROR: Command errored out with exit status 1:
     command: /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-LTmTbQ/ftfy/setup.py'"'"'; __file__='"'"'/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-LTmTbQ/ftfy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-pip-egg-info-BtyMKd
         cwd: /private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-install-LTmTbQ/ftfy/
    Complete output (13 lines):
    
    Sorry, this version of ftfy is no longer written for Python 2.
    
    It is much easier to provide consistent handling of Unicode when developing
    only for Python 3. ftfy is exactly the kind of library that Python 3 is
    designed for, and now there is enough Python 3 usage that we can take advantage
    of it and write better, simpler code.
    
    The older version of ftfy, version 4.4, is still available and can run on
    Python 2. Try this:
    
        pip install ftfy==4.4.3
    
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting git+https://github.com/TencentARC/GFPGAN.git@8d2447a2d918f8eba5a4a01463fd48e45126a379
  Cloning https://github.com/TencentARC/GFPGAN.git (to revision 8d2447a2d918f8eba5a4a01463fd48e45126a379) to /private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-req-build-qnWjfl
  Running command git clone -q https://github.com/TencentARC/GFPGAN.git /private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-req-build-qnWjfl
  Running command git rev-parse -q --verify 'sha^8d2447a2d918f8eba5a4a01463fd48e45126a379'
  Running command git fetch -q https://github.com/TencentARC/GFPGAN.git 8d2447a2d918f8eba5a4a01463fd48e45126a379
  Running command git checkout -q 8d2447a2d918f8eba5a4a01463fd48e45126a379
    ERROR: Command errored out with exit status 1:
     command: /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-req-build-qnWjfl/setup.py'"'"'; __file__='"'"'/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-req-build-qnWjfl/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-pip-egg-info-kEZtYm
         cwd: /private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-req-build-qnWjfl/
    Complete output (6 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/m1/t1dmhj111cg3h9sn1thdcf4w0000gn/T/pip-req-build-qnWjfl/setup.py", line 62
        VERSION_INFO = ', '.join([x if x.isdigit() else f'"{x}"' for x in SHORT_VERSION.split('.')])
                                                               ^
    SyntaxError: invalid syntax
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
ERROR: Could not find a version that satisfies the requirement torch==1.12.1 (from versions: 1.0.0, 1.0.1, 1.0.1.post2, 1.1.0, 1.1.0.post2, 1.2.0, 1.3.0.post2, 1.3.1, 1.4.0, 1.5.0)
ERROR: No matching distribution found for torch==1.12.1
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named torch
patch: **** can't cd to /nn: No such file or directory
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting gdown
  Using cached gdown-4.5.3.tar.gz (14 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
ERROR: Could not find a version that satisfies the requirement fastapi (from versions: none)
ERROR: No matching distribution found for fastapi
To make your changes take effect please reactivate your environment
WARNING: overwriting environment variables set in the machine
overwriting variable PYTORCH_ENABLE_MPS_FALLBACK
=============================================
=============================================
==============MORE INFORMATION===============
=============================================
=============================================
If you want to run the web UI again, you can run the following command:
./stable-diffusion-webui/run_webui_mac.sh
or
cd stable-diffusion-webui && ./run_webui_mac.sh
=============================================
=============================================
=============================================
=============================================
  File "webui.py", line 94
    print(f"Invalid path to TLS certfile: '{cmd_opts.tls_certfile}'")
                                                                   ^
SyntaxError: invalid syntax
(base) robert@Roberts-MacBook-Pro ~ % 

[bad1313]

The reference to Python 2.7 shows that somehow your Mac has big but typical issues.
There are hints in the internet how to remove the standard python 2.7 from Mac.
In my case, I completely switched to homebrew for installing Python on Mac.
For Stable diffusion, your goal is to run python 3.10.x

[Roblouisck]

@bad1313 I ended up doing that as well, but ran into another set of problems after that: #4594

[0xdevalias]

Not sure if this will work for your specific issue here, but I suspect that it possibly will:

You can modify your run_webui_mac.sh to add a line that tries to install the required dependencies with pip install -r requirements.txt each time it starts, after it pulls the latest code. This would probably make a sensible default, and is fairly quick to run, so it might be worth someone telling the original author to include it in their setup script to avoid this sort of error for end users in future:

Here is what mine looks like:

#!/usr/bin/env bash -l

pyenv local anaconda3-2022.05

# This should not be needed since it's configured during installation, but might as well have it here.
conda env config vars set PYTORCH_ENABLE_MPS_FALLBACK=1

# Activate conda environment
conda activate web-ui

# Pull the latest changes from the repo
git pull --rebase

+ # Update the dependencies if needed
+ pip install -r requirements.txt

# Run the web ui
python webui.py --deepdanbooru --precision full --no-half --use-cpu Interrogate GFPGAN CodeFormer $@

# Deactivate conda environment
conda deactivate

Originally posted by @0xdevalias in #4109 (comment)

---

For the others who said that pip install -r requirements.txt didn't work for them, or finding that despite pip installing the individual requirements they still don't seem to 'be there', it might be an issue with your conda environment, and which pip is being used. Sometimes the version you're calling doesn't actually install the packages to the correct place, and so they can't be found later.

I have a new theory for you, based on this StackOverflow:

• https://stackoverflow.com/questions/41060382/using-pip-to-install-packages-to-anaconda-environment

What do you see when you activate your conda environment, then run which -a pip?

If it's only something like:

/opt/conda/bin/pip

And not something like:

/opt/conda/envs/web-ui/bin/pip
/opt/conda/bin/pip

Then you can likely fix it by either doing a conda install pip then using pip as normal, or using python -m pip install FOO instead of using pip directly.

Originally posted by @0xdevalias in #4109 (comment)

I did a conda install pip and then used python -m, it looked like it wasn't going to work but then I changed my run script a tiny bit,

Curious, what made you think it wasn't going to work? And what was the specific change to the run script that made it work for you? Was it using requirements_versions.txt rather than requirements.txt?

python -m pip install -r requirements_versions.txt

Originally posted by @0xdevalias in #4109 (comment)

---

Hopefully the above helps people who are still running into this issue 🖤

Originally posted by @0xdevalias in #4061 (comment)