Stalls on hooks checkout

[admorgan]

When trying to use some hook repositories I am getting hangs while fetching hooks. Tried pre-commit clean and rm -rf ~/.cache/pre-commit, but nothing changed. This is happening on both Windows 10 and Ubuntu 18.04.4 LTS.

pre-commit.log

version information

pre-commit version: 2.4.0
sys.version:
    3.6.9 (default, Apr 18 2020, 01:56:04)
    [GCC 8.4.0]
sys.executable: /usr/bin/python3
os.name: posix
sys.platform: linux

error information

Interrupted (^C): KeyboardInterrupt:

Traceback (most recent call last):
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/error_handler.py", line 56, in error_handler
    yield
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/main.py", line 388, in main
    return install_hooks(args.config, store)
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/commands/install_uninstall.py", line 153, in install_hooks
    install_hook_envs(all_hooks(load_config(config_file), store), store)
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/repository.py", line 206, in all_hooks
    for repo in root_config['repos']
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/repository.py", line 207, in <genexpr>
    for hook in _repository_hooks(repo, store, root_config)
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/repository.py", line 182, in _repository_hooks
    return _cloned_repository_hooks(repo_config, store, root_config)
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/repository.py", line 148, in _cloned_repository_hooks
    manifest_path = os.path.join(store.clone(repo, rev), C.MANIFEST_FILE)
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/store.py", line 183, in clone
    return self._new_repo(repo, ref, deps, clone_strategy)
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/store.py", line 140, in _new_repo
    make_strategy(directory)
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/store.py", line 179, in clone_strategy
    self._shallow_clone(ref, _git_cmd)
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/store.py", line 162, in _shallow_clone
    git_cmd('checkout', 'FETCH_HEAD')
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/store.py", line 176, in _git_cmd
    cmd_output_b('git', *args, cwd=directory, env=env)
  File "/home/morgana/.local/lib/python3.6/site-packages/pre_commit/util.py", line 155, in cmd_output_b
    stdout_b, stderr_b = proc.communicate()
  File "/usr/lib/python3.6/subprocess.py", line 863, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.6/subprocess.py", line 1534, in _communicate
    ready = selector.select(timeout)
  File "/usr/lib/python3.6/selectors.py", line 376, in select
    fd_event_list = self._poll.poll(timeout)
KeyboardInterrupt

Added to ~/.local/lib/python3.6/site-packages/pre_commit/util.py line 143

print(cmd)
print(kwargs)

$ pre-commit install-hooks
('git', 'rev-parse', '--show-toplevel')
{'stdin': -1, 'stdout': -1, 'stderr': -1}
[INFO] Initializing environment for https://github.com/psf/black.
('git', 'init', '/home/morgana/.cache/pre-commit/repohg1s5b7b')
{'env': {'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:.xspf=00;36:', 'SSH_CONNECTION': '10.30.48.92 50335 10.30.48.87 22', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'LANG': 'en_US.UTF-8', 'PERL_MB_OPT': '--install_base "/home/morgana/perl5"', 'XDG_SESSION_ID': '10', 'USER': 'morgana', 'PWD': '/home/morgana/projects/git-hooks', 'HOME': '/home/morgana', 'SSH_CLIENT': '10.30.48.92 50335 22', 'XDG_DATA_DIRS': '/usr/local/share:/usr/share:/var/lib/snapd/desktop', 'PERL_MM_OPT': 'INSTALL_BASE=/home/morgana/perl5', 'PERL_LOCAL_LIB_ROOT': '/home/morgana/perl5', 'SSH_TTY': '/dev/pts/0', 'MAIL': '/var/mail/morgana', 'TERM': 'xterm', 'SHELL': '/bin/bash', 'PERL5LIB': '/home/morgana/perl5/lib/perl5', 'SHLVL': '1', 'LANGUAGE': 'en_US:', 'MANPATH': '/home/morgana/.git/subrepo/man:', 'LOGNAME': 'morgana', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/17173/bus', 'XDG_RUNTIME_DIR': '/run/user/17173', 'PATH': '/home/morgana/.local/bin:/home/morgana/bin:/home/morgana/perl5/bin:/home/morgana/.git/subrepo/lib:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'OLDPWD': '/home/morgana', '': '/home/morgana/.local/bin/pre-commit'}, 'stdin': -1, 'stdout': -1, 'stderr': -1}
('git', 'remote', 'add', 'origin', 'https://github.com/psf/black')
{'cwd': '/home/morgana/.cache/pre-commit/repohg1s5b7b', 'env': {'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:.xspf=00;36:', 'SSH_CONNECTION': '10.30.48.92 50335 10.30.48.87 22', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'LANG': 'en_US.UTF-8', 'PERL_MB_OPT': '--install_base "/home/morgana/perl5"', 'XDG_SESSION_ID': '10', 'USER': 'morgana', 'PWD': '/home/morgana/projects/git-hooks', 'HOME': '/home/morgana', 'SSH_CLIENT': '10.30.48.92 50335 22', 'XDG_DATA_DIRS': '/usr/local/share:/usr/share:/var/lib/snapd/desktop', 'PERL_MM_OPT': 'INSTALL_BASE=/home/morgana/perl5', 'PERL_LOCAL_LIB_ROOT': '/home/morgana/perl5', 'SSH_TTY': '/dev/pts/0', 'MAIL': '/var/mail/morgana', 'TERM': 'xterm', 'SHELL': '/bin/bash', 'PERL5LIB': '/home/morgana/perl5/lib/perl5', 'SHLVL': '1', 'LANGUAGE': 'en_US:', 'MANPATH': '/home/morgana/.git/subrepo/man:', 'LOGNAME': 'morgana', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/17173/bus', 'XDG_RUNTIME_DIR': '/run/user/17173', 'PATH': '/home/morgana/.local/bin:/home/morgana/bin:/home/morgana/perl5/bin:/home/morgana/.git/subrepo/lib:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'OLDPWD': '/home/morgana', '': '/home/morgana/.local/bin/pre-commit'}, 'stdin': -1, 'stdout': -1, 'stderr': -1}
('git', '-c', 'protocol.version=2', 'fetch', 'origin', 'stable', '--depth=1')
{'cwd': '/home/morgana/.cache/pre-commit/repohg1s5b7b', 'env': {'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:.xspf=00;36:', 'SSH_CONNECTION': '10.30.48.92 50335 10.30.48.87 22', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'LANG': 'en_US.UTF-8', 'PERL_MB_OPT': '--install_base "/home/morgana/perl5"', 'XDG_SESSION_ID': '10', 'USER': 'morgana', 'PWD': '/home/morgana/projects/git-hooks', 'HOME': '/home/morgana', 'SSH_CLIENT': '10.30.48.92 50335 22', 'XDG_DATA_DIRS': '/usr/local/share:/usr/share:/var/lib/snapd/desktop', 'PERL_MM_OPT': 'INSTALL_BASE=/home/morgana/perl5', 'PERL_LOCAL_LIB_ROOT': '/home/morgana/perl5', 'SSH_TTY': '/dev/pts/0', 'MAIL': '/var/mail/morgana', 'TERM': 'xterm', 'SHELL': '/bin/bash', 'PERL5LIB': '/home/morgana/perl5/lib/perl5', 'SHLVL': '1', 'LANGUAGE': 'en_US:', 'MANPATH': '/home/morgana/.git/subrepo/man:', 'LOGNAME': 'morgana', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/17173/bus', 'XDG_RUNTIME_DIR': '/run/user/17173', 'PATH': '/home/morgana/.local/bin:/home/morgana/bin:/home/morgana/perl5/bin:/home/morgana/.git/subrepo/lib:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'OLDPWD': '/home/morgana', '': '/home/morgana/.local/bin/pre-commit'}, 'stdin': -1, 'stdout': -1, 'stderr': -1}
('git', 'checkout', 'FETCH_HEAD')
{'cwd': '/home/morgana/.cache/pre-commit/repohg1s5b7b', 'env': {'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:.xspf=00;36:', 'SSH_CONNECTION': '10.30.48.92 50335 10.30.48.87 22', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'LANG': 'en_US.UTF-8', 'PERL_MB_OPT': '--install_base "/home/morgana/perl5"', 'XDG_SESSION_ID': '10', 'USER': 'morgana', 'PWD': '/home/morgana/projects/git-hooks', 'HOME': '/home/morgana', 'SSH_CLIENT': '10.30.48.92 50335 22', 'XDG_DATA_DIRS': '/usr/local/share:/usr/share:/var/lib/snapd/desktop', 'PERL_MM_OPT': 'INSTALL_BASE=/home/morgana/perl5', 'PERL_LOCAL_LIB_ROOT': '/home/morgana/perl5', 'SSH_TTY': '/dev/pts/0', 'MAIL': '/var/mail/morgana', 'TERM': 'xterm', 'SHELL': '/bin/bash', 'PERL5LIB': '/home/morgana/perl5/lib/perl5', 'SHLVL': '1', 'LANGUAGE': 'en_US:', 'MANPATH': '/home/morgana/.git/subrepo/man:', 'LOGNAME': 'morgana', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/17173/bus', 'XDG_RUNTIME_DIR': '/run/user/17173', 'PATH': '/home/morgana/.local/bin:/home/morgana/bin:/home/morgana/perl5/bin:/home/morgana/.git/subrepo/lib:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'OLDPWD': '/home/morgana', '': '/home/morgana/.local/bin/pre-commit'}, 'stdin': -1, 'stdout': -1, 'stderr': -1}

.pre-commit-config.yaml

repos:
  - repo: https://github.com/psf/black
    rev: stable
    hooks:
      - id: black
        language_version: python3.6
        stages: [commit]
  - repo: https://github.com/shellcheck-py/shellcheck-py
    rev: v0.7.1.1
    hooks:
    -   id: shellcheck
        stages: [commit]
  - repo: local
    hooks:
      # Add a ChangeID tag to commit messages that don't have one
      -   id: Change-Id
          name: Add a ChangeID tag to commit messages that don't have one
          entry: change-id
          always_run: true
          language: script
          stages: [commit-msg]

[asottile]

have you tried the commands separately? are you behind some sort of proxy?

[admorgan]

I have, and everything clones/checks out normally

[asottile]

on 18.04 it shouldn't be taking the shallow clone, unless you have updated git out-of-band

it's not using git clone so you'll need to replicate the commands it runs exactly. we avoid git clone for performance reasons

can you try this?

git init repo
cd repo
git remote add origin https://github.com/psf/black
git -c protocol.version=2 fetch origin stable --depth=1
git checkout FETCH_HEAD

[asottile]

also check env | grep -i git -- we need to wipe the git environment to avoid breaking the repo-under-test

[admorgan]

I think I potentially see an issue. When I am checking out FETCH_HEAD it is trying to run pre-commit and install flake8 which also hangs.

[admorgan]

git version 2.24.1

[asottile]

huh that's weird, how do you have checkout hooks set up ?

[admorgan]

I have pre-commit using the global templates, therefore it gets applied to all repositories as we have hundreds that will be using it in the near future.

[asottile]

hmmmmmm ok so pre-commit is calling pre-commit is calling pre-commit is calling pre-commit 🤔

[admorgan]

Yes, I just started seeing this when I upgraded to 2.4.0. I don't know what the previous version was.

[admorgan]

If I remove the post-checkout hook from the template dir I am able to prevent this from hanging.

[asottile]

my guess is you were using something <2.2 which didn't have support for post-checkout?

[asottile]

can you try the patch in #1497

[admorgan]

That works for me. I have seen installs that put the hooks in the core.hookspath and it doesn't address that configuration.

[asottile]

core.hooksPath is forbidden in pre-commit so not a problem