pre-commit autoupdate yields "PermissionError: [WinError 32]"

[lcnittl]

describe your issue

Whenever I try to do an autoupdate of the pre-commit config, I get the following error. I tried to reinstall pre-commit, wipe the venv dir, upgrade pip etc. Running on Win10 and Python 3.9.6 - it has been working since recently, but I can not recall any substantial change on the system that might cause this behavior. Interestingly installing the hooks works fine.

Really no idea what causes this and how to fix it. Did I miss something?

PS C:\Users\myusername\repos\myrepo> pre-commit autoupdate
Updating https://github.com/pre-commit/pre-commit-hooks ... An unexpected error has occurred: PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\myusername\\AppData\\Local\\Temp\\tmpjwm58zbt'
Check the log at C:\Users\myusername\.cache\pre-commit\pre-commit.log

pre-commit --version

pre-commit 2.15.0

.pre-commit-config.yaml

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.0.1
    hooks:
      - id: check-case-conflict
      - id: check-merge-conflict
      - id: trailing-whitespace
        args: [--markdown-linebreak-ext=md]
      - id: end-of-file-fixer
      - id: check-yaml
  - repo: https://github.com/pre-commit/mirrors-prettier
    rev: v2.3.2
    hooks:
      - id: prettier

~/.cache/pre-commit/pre-commit.log (if present)

version information

pre-commit version: 2.15.0
sys.version:
    3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)]
sys.executable: C:\Users\myusername\.local\pipx\venvs\pre-commit\Scripts\python.exe
os.name: nt
sys.platform: win32

error information

An unexpected error has occurred: PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\myusername\\AppData\\Local\\Temp\\tmpjwm58zbt'

Traceback (most recent call last):
  File "c:\program files\python39\lib\shutil.py", line 620, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\myusername\\AppData\\Local\\Temp\\tmpjwm58zbt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\myusername\.local\pipx\venvs\pre-commit\lib\site-packages\pre_commit\error_handler.py", line 65, in error_handler
    yield
  File "C:\Users\myusername\.local\pipx\venvs\pre-commit\lib\site-packages\pre_commit\main.py", line 357, in main
    return autoupdate(
  File "C:\Users\myusername\.local\pipx\venvs\pre-commit\lib\site-packages\pre_commit\commands\autoupdate.py", line 157, in autoupdate
    new_info = info.update(tags_only=tags_only, freeze=freeze)
  File "C:\Users\myusername\.local\pipx\venvs\pre-commit\lib\site-packages\pre_commit\commands\autoupdate.py", line 58, in update
    rev, frozen = exact, rev
  File "c:\program files\python39\lib\contextlib.py", line 124, in __exit__
    next(self.gen)
  File "C:\Users\myusername\.local\pipx\venvs\pre-commit\lib\site-packages\pre_commit\util.py", line 72, in tmpdir
    rmtree(tempdir)
  File "C:\Users\myusername\.local\pipx\venvs\pre-commit\lib\site-packages\pre_commit\util.py", line 265, in rmtree
    shutil.rmtree(path, ignore_errors=False, onerror=handle_remove_readonly)
  File "c:\program files\python39\lib\shutil.py", line 740, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\program files\python39\lib\shutil.py", line 622, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\Users\myusername\.local\pipx\venvs\pre-commit\lib\site-packages\pre_commit\util.py", line 262, in handle_remove_readonly
    func(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\myusername\\AppData\\Local\\Temp\\tmpjwm58zbt'

[asottile]

do you have some sort of virus scanner? or do you have that directory open in some other program?

[lcnittl]

I'd say no to both.

Ad virus scanner: Nothing custom, just what is on Windows 10 by default ("Virus & threat protection" in the settings). No reports of blocked access there.

Ad directory open in some other program: Happens even when I completely remove the ~\.cache\pre-commit directory before.

[asottile]

the error doesn't really have anything to do with the pre-commit cache -- it's creating a temporary directory in your tempdir and then cleaning it up when it is done. there must be some other program running on your machine which notices that directory and holds onto it for some reason -- maybe you could inspect what programs are accessing that directory?

[lcnittl]

You are right! Using "Resource Manager" I was able to spot solely AcroRd64.exe was "listening" to the temp folder itself, some other programs to folders and files within. Though, after terminating each of the processes the error still persists. Very strange.

[asottile]

perhaps try using handle.exe to figure out precisely what is accessing pre-commit's temporary directories? https://docs.microsoft.com/en-us/sysinternals/downloads/handle

[lcnittl]

Thanks for the link - nice utility. It basically gives me the same list as "Resource Manager", but running it in the correct moment reveals the following:

PS C:\Users\myusername\Downloads\Handle> .\handle64.exe Users\myusername\AppData\Local\Temp

Nthandle v4.22 - Handle viewer
Copyright (C) 1997-2019 Mark Russinovich
Sysinternals - www.sysinternals.com

git.exe            pid: 4028   type: File            44: C:\Users\myusername\AppData\Local\Temp\tmpp8tkb524
git.exe            pid: 10736  type: File            48: C:\Users\myusername\AppData\Local\Temp\tmpp8tkb524
git.exe            pid: 17316  type: File            44: C:\Users\myusername\AppData\Local\Temp\tmpp8tkb524
git.exe            pid: 17316  type: File           114: C:\Users\myusername\AppData\Local\Temp\tmpp8tkb524
git.exe            pid: 21328  type: File            44: C:\Users\myusername\AppData\Local\Temp\tmpp8tkb524
git-remote-https.exe pid: 10852  type: File            44: C:\Users\myusername\AppData\Local\Temp\tmpp8tkb524
PS C:\Users\myusername\Downloads\Handle>

So it seems to be just pre-commit? (does it call git in the background - or is this the culprit??)

[asottile]

it calls git indeed -- but it always waits until the process succeeds -- unless there's something fiddly that git broke where it is leaking processes

[asottile]

what git version are you using (I can't reproduce with what I have installed -- though it's fairly old: git version 2.29.2.windows.2

[lcnittl]

git was the key word. I apparently recently upgraded it, and enabled the experimental built-in file system monitor (cf. screenshot).

Removing that setting during a reinstall fixed the issue.

Thank you for the - very quick - support!

Not sure if this is something that has to be fixed here - as the feature is experimental, I think it is rather on git's side. So this issue might be closed as resolved.

[asottile]

I wonder if there's a way to disable it for specific repositories -- that definitely seems like it'll be problematic if it's turned on for everything

[lcnittl]

To me it seems like a yes:

From https://newreleases.io/project/github/git-for-windows/git/release/v2.31.0.windows.1 :

Git for Windows now ships with an experimental built-in file-system monitor, without the need to install Watchman and setting core.fsmonitor. It can be turned on by setting both feature.manyFiles=true and feature.experimental=true (or directly, via core.useBuiltinFSMonitor=true).

So I assume adding those config settings on a repo level with false should disable that feature?