dpt_beit_large_512 (midas 3.1) - ERROR

[web2299]

When i try to use dpt_beit_large_512 (midas 3.1) i receive this error:

ERROR: Error(s) in loading state_dict for DPTDepthModel:
Unexpected key(s) in state_dict: "pretrained.model.blocks.0.attn.relative_position_index", "pretrained.model.blocks.1.attn.relative_position_index", "pretrained.model.blocks.2.attn.relative_position_index", "pretrained.model.blocks.3.attn.relative_position_index", "pretrained.model.blocks.4.attn.relative_position_index", "pretrained.model.blocks.5.attn.relative_position_index", "pretrained.model.blocks.6.attn.relative_position_index", "pretrained.model.blocks.7.attn.relative_position_index", "pretrained.model.blocks.8.attn.relative_position_index", "pretrained.model.blocks.9.attn.relative_position_index", "pretrained.model.blocks.10.attn.relative_position_index", "pretrained.model.blocks.11.attn.relative_position_index", "pretrained.model.blocks.12.attn.relative_position_index", "pretrained.model.blocks.13.attn.relative_position_index", "pretrained.model.blocks.14.attn.relative_position_index", "pretrained.model.blocks.15.attn.relative_position_index", "pretrained.model.blocks.16.attn.relative_position_index", "pretrained.model.blocks.17.attn.relative_position_index", "pretrained.model.blocks.18.attn.relative_position_index", "pretrained.model.blocks.19.attn.relative_position_index", "pretrained.model.blocks.20.attn.relative_position_index", "pretrained.model.blocks.21.attn.relative_position_index", "pretrained.model.blocks.22.attn.relative_position_index", "pretrained.model.blocks.23.attn.relative_position_index".

Please report this issue here. Make sure to provide the full stacktrace:
Traceback (most recent call last):
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\src\common_ui.py", line 532, in run_generate
    input_i, type, result = next(gen_obj)
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\src\core.py", line 322, in core_generation_funnel
    raise e
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\src\core.py", line 124, in core_generation_funnel
    model_holder.ensure_models(inp[go.MODEL_TYPE], device, inp[go.BOOST])
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\src\depthmap_generation.py", line 62, in ensure_models
    self.load_models(model_type, device, boost)
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\src\depthmap_generation.py", line 111, in load_models
    model = DPTDepthModel(
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\dmidas\dpt_depth.py", line 163, in __init__
    self.load(path)
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\dmidas\base_model.py", line 16, in load
    self.load_state_dict(parameters)
  File "C:\SD_Forge\system\python\lib\site-packages\torch\nn\modules\module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for DPTDepthModel:
	Unexpected key(s) in state_dict: "pretrained.model.blocks.0.attn.relative_position_index", "pretrained.model.blocks.1.attn.relative_position_index", "pretrained.model.blocks.2.attn.relative_position_index", "pretrained.model.blocks.3.attn.relative_position_index", "pretrained.model.blocks.4.attn.relative_position_index", "pretrained.model.blocks.5.attn.relative_position_index", "pretrained.model.blocks.6.attn.relative_position_index", "pretrained.model.blocks.7.attn.relative_position_index", "pretrained.model.blocks.8.attn.relative_position_index", "pretrained.model.blocks.9.attn.relative_position_index", "pretrained.model.blocks.10.attn.relative_position_index", "pretrained.model.blocks.11.attn.relative_position_index", "pretrained.model.blocks.12.attn.relative_position_index", "pretrained.model.blocks.13.attn.relative_position_index", "pretrained.model.blocks.14.attn.relative_position_index", "pretrained.model.blocks.15.attn.relative_position_index", "pretrained.model.blocks.16.attn.relative_position_index", "pretrained.model.blocks.17.attn.relative_position_index", "pretrained.model.blocks.18.attn.relative_position_index", "pretrained.model.blocks.19.attn.relative_position_index", "pretrained.model.blocks.20.attn.relative_position_index", "pretrained.model.blocks.21.attn.relative_position_index", "pretrained.model.blocks.22.attn.relative_position_index", "pretrained.model.blocks.23.attn.relative_position_index". 
Time taken: 13.1 sec.

A: 0.32 GB, R: 0.34 GB, Sys: 1.5/15.9961 GB (9.3%)

I tried:

pip install timm==0.6.7

but this didnt solve the problem !

📂

[zzcxxoo]

same question

[NicoSteyl]

Same issue, pip didn't fix anything.

[web2299]

any support ?

[graemeniedermayer]

Is this problem specific to midas?

A similar issue a while ago there was a problem with timm library. What's you version of timm?

#323

#319

[web2299]

as already said i tried to install timm (many different versions) but this didnt solve the problem . Yes i think the problem is specific to midas because other models work fine .

[graemeniedermayer]

Oh yup sorry missed that. It does look like a library issue. Like the weight file and the config are mismatched for some reason.

It's working for me.

So the file size for me is
1.5gb dpt_beit_large_512.pt (if it's different that's a problem)

Some of my versions of relevant libraries are
python3.10
triton==2.1.0
torch==2.1.0+cu121
timm==0.9.2
torchvision==0.16.0

[NicoSteyl]

I got it working but I don't know what did lol

In summary, I upgraded pip first. Then I opened a command shell and activated venv, I installed triton, then I launched the webui and it failed so I uninstalled triton and it looks like cmake got left behind. It's working for me now.

Powershell Log (incase a real programmer can make sense of it):

`Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows

PS G:\AI\stable-diffusion-webui-1.8.0\venv> .\scripts\activate
(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install triton
WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
ERROR: Could not find a version that satisfies the requirement triton (from versions: none)
ERROR: No matching distribution found for triton

[notice] A new release of pip is available: 22.2.1 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip
(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> python.exe -m pip install --upgrade pip
WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
Requirement already satisfied: pip in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (22.2.1)
Collecting pip
Downloading pip-24.0-py3-none-any.whl.metadata (3.6 kB)
Downloading pip-24.0-py3-none-any.whl (2.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 884.2 kB/s eta 0:00:00
WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
DEPRECATION: torchsde 0.2.5 has a non-standard dependency specifier numpy>=1.19.*; python_version >= "3.7". pip 24.1 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of torchsde or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at pypa/pip#12063
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 22.2.1
Uninstalling pip-22.2.1:
Successfully uninstalled pip-22.2.1
Successfully installed pip-24.0
(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install triton
WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
ERROR: Could not find a version that satisfies the requirement triton (from versions: none)
ERROR: No matching distribution found for triton
(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install https://huggingface.co/r4ziel/xformers_pre_built/resolve/main/triton-2.0.0-cp310-cp310-win_amd64.whl
https://oo.pe/https://huggingface.co/r4ziel/xformers_pre_built/resolve/main/triton-2.0.0-cp310-cp310-win_amd64.whl :
The term
'https://oo.pe/https://huggingface.co/r4ziel/xformers_pre_built/resolve/main/triton-2.0.0-cp310-cp310-win_amd64.whl'
is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name,
or if a path was included, verify that the path is correct and try again.
At line:1 char:116

• ... amd64.whl](https://oo.pe/https://huggingface.co/r4ziel/xformers_pre ...

•             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  

  • CategoryInfo : ObjectNotFound: (https://oo.pe/h...0-win_amd64.whl:String) [], CommandNotFoundException
  • FullyQualifiedErrorId : CommandNotFoundException

(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install https://huggingface.co/r4ziel/xformers_pre_built/resolve/main/triton-2.0.0-cp310-cp310-win_amd64.whl
WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
Collecting triton==2.0.0
Downloading https://huggingface.co/r4ziel/xformers_pre_built/resolve/main/triton-2.0.0-cp310-cp310-win_amd64.whl (12.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.6/12.6 MB 1.1 MB/s eta 0:00:00
Collecting cmake (from triton==2.0.0)
Downloading cmake-3.29.0.1-py3-none-win_amd64.whl.metadata (6.1 kB)
Requirement already satisfied: filelock in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from triton==2.0.0) (3.13.1)
Requirement already satisfied: torch in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from triton==2.0.0) (2.0.1+cu118)
Requirement already satisfied: typing-extensions in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from torch->triton==2.0.0) (4.10.0)
Requirement already satisfied: sympy in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from torch->triton==2.0.0) (1.12)
Requirement already satisfied: networkx in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from torch->triton==2.0.0) (3.2.1)
Requirement already satisfied: jinja2 in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from torch->triton==2.0.0) (3.1.2)
Requirement already satisfied: MarkupSafe>=2.0 in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from jinja2->torch->triton==2.0.0) (2.1.3)
Requirement already satisfied: mpmath>=0.19 in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from sympy->torch->triton==2.0.0) (1.3.0)
Downloading cmake-3.29.0.1-py3-none-win_amd64.whl (36.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 36.2/36.2 MB 1.0 MB/s eta 0:00:00
WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
DEPRECATION: torchsde 0.2.5 has a non-standard dependency specifier numpy>=1.19.*; python_version >= "3.7". pip 24.1 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of torchsde or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at pypa/pip#12063
Installing collected packages: cmake, triton
Successfully installed cmake-3.29.0.1 triton-2.0.0
(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip uninstall triton
Found existing installation: triton 2.0.0
Uninstalling triton-2.0.0:
Would remove:
g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages\triton-2.0.0.dist-info*
g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages\triton*
Proceed (Y/n)? y
Successfully uninstalled triton-2.0.0
(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip cache purge
Files removed: 1293
(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install triton-2.1.0-cp310-cp310-win_amd64.whl
WARNING: Requirement 'triton-2.1.0-cp310-cp310-win_amd64.whl' looks like a filename, but the file does not exist
WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
Processing g:\ai\stable-diffusion-webui-1.8.0\venv\triton-2.1.0-cp310-cp310-win_amd64.whl
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: 'G:\AI\stable-diffusion-webui-1.8.0\venv\triton-2.1.0-cp310-cp310-win_amd64.whl'

(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip uninstall itsandbytes -y
WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Skipping itsandbytes as it is not installed.
(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip uninstall numpy -y
Found existing installation: numpy 1.26.2
Uninstalling numpy-1.26.2:
ERROR: Operation cancelled by user
(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install numpy -y

Usage:
pip install [options] [package-index-options] ...
pip install [options] -r [package-index-options] ...
pip install [options] [-e] ...
pip install [options] [-e] ...
pip install [options] <archive url/path> ...

no such option: -y
(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install numpy
WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
Requirement already satisfied: numpy in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (1.26.2)
WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip uninstall umpy -y
WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages)
WARNING: Skipping umpy as it is not installed.
(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv>
`

[graemeniedermayer]

Wonderful. Oh also checking if any library has changed after auto1111 launchs is important. Sometimes there is silent requirements conflicts. Trying without any other extensions can be a helpful troubleshooting strategy (particularly newer extensions. Maybe this is a depth-anything conflict?)