HTTP 416 when trying to download model

[martyphee]

I've been having a tough time trying to get sentence-transformers running in a Docker container in Kobe. Locally all good but on kube I keep running into the issue below. We do have disk space allocated and I've set HF_HOME to point to the mount (/data/cache) and we have temp file mount so it can save the files.

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

  File "/venv/lib/python3.12/site-packages/transformers/utils/hub.py", line 460, in cached_file
    raise EnvironmentError(f"There was a specific connection error when trying to load {path_or_repo_id}:\n{err}")
OSError: There was a specific connection error when trying to load sentence-transformers/all-MiniLM-L6-v2:
416 Client Error: Range Not Satisfiable for url: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/config.json (Request ID: Root=1-664638ad-4620ad8d4cb775c724201eca;f61d8e6b-0acf-4445-aeac-ff4029b61956)

I even get this if I go into the pod and use the CLI to download the model. When trying the CLI it looks like it attempts to download 17 file simultaneously and fails on the last one. I can see it did progress a good amount but I can't get it to complete. Is there anyway to throttle how many requests are don't in parallel?`

My workaround for right now is to download and save the model during the docker build but I wondering if anyone has run into this previously and had a workaround?

[tomaarsen]

Hello!

I haven't seen this issue before; paging @Wauplin as he may have encountered this with huggingface_hub before. Truthfully, I've never even seen a 416 Range Not Satisfiable. Do you also experience this with other models, e.g. sentence-transformers/all-MiniLM-L12-v2?

In the meantime, your workaround should indeed work, although I recognize that it's very inconvenient.

• Tom Aarsen

[martyphee]

I haven't tried other models yet but will try today using the cli.

Same issue

search-77f69f566-grwm7:/data/hf_cache$ huggingface-cli download sentence-transformers/all-MiniLM-L12-v2 --cache-dir /data/hf_cache

Traceback (most recent call last):
  File "/venv/bin/huggingface-cli", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/venv/lib/python3.12/site-packages/huggingface_hub/commands/huggingface_cli.py", line 51, in main
    service.run()
  File "/venv/lib/python3.12/site-packages/huggingface_hub/commands/download.py", line 146, in run
    print(self._download())  # Print path to downloaded files
          ^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/huggingface_hub/commands/download.py", line 180, in _download
    return snapshot_download(
           ^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/huggingface_hub/_snapshot_download.py", line 294, in snapshot_download
    thread_map(
  File "/venv/lib/python3.12/site-packages/tqdm/contrib/concurrent.py", line 69, in thread_map
    return _executor_map(ThreadPoolExecutor, fn, *iterables, **tqdm_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/tqdm/contrib/concurrent.py", line 51, in _executor_map
    return list(tqdm_class(ex.map(fn, *iterables, chunksize=chunksize), **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/tqdm/std.py", line 1181, in __iter__
    for obj in iterable:
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 619, in result_iterator
    yield _result_or_cancel(fs.pop())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 317, in _result_or_cancel
    return fut.result(timeout)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/huggingface_hub/_snapshot_download.py", line 268, in _inner_hf_hub_download
    return hf_hub_download(
           ^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download
    return _hf_hub_download_to_cache_dir(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/venv/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 1367, in _hf_hub_download_to_cache_dir
    _download_to_tmp_and_move(
  File "/venv/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 1894, in _download_to_tmp_and_move
    _chmod_and_move(incomplete_path, destination_path)
  File "/venv/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 1922, in _chmod_and_move
    os.chmod(str(src), stat.S_IMODE(cache_dir_mode))
PermissionError: [Errno 1] Operation not permitted: '/data/hf_cache/models--sentence-transformers--all-MiniLM-L12-v2/blobs/2bf99854609538f90af03c921c41a0cbde670b2f.incomplete'