Most pre-commit hooks broken after OS upgrade to 20.04

[ahonnecke]

repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v2.5.0
    hooks:
    - id: check-ast

ahonnecke@homonym:~/src/web3/web3-maker-scraper$ pre-commit
Check python ast.........................................................Failed
- hook id: check-ast
- exit code: 1

Traceback (most recent call last):
  File "/home/ahonnecke/.cache/pre-commit/repoliux_i6a/py_env-python3/bin/check-ast", line 5, in <module>
    from pre_commit_hooks.check_ast import main
ModuleNotFoundError: No module named 'pre_commit_hooks'

I get the same behavior from all my hooks (except pyupgrade)

Traceback (most recent call last):
  File "/home/ahonnecke/.cache/pre-commit/repozqy6dpbu/py_env-python3/bin/black", line 5, in <module>
    from black import patched_main
ModuleNotFoundError: No module named 'black'

I tried uninstalling and reinstalling all those pip packages (both globally and with --user):
sudo pip install pylint pytest flake8 mypy pre-commit isort black

[asottile]

that's strange, it worked fine for me when I upgrade from 18.04 -> 20.04 -- pre-commit has code to detect this and recreate environments even

can you poke around in /home/ahonnecke/.cache/pre-commit/repozqy6dpbu and see if things got (partially) deleted? usually when upgrading operating systems virtualenvs need rebuilding

[asottile]

an ls -al in that folder + ls -al in that folder + py_env-python3/bin will probably tell me enough information

[asottile]

and likely pre-commit clean will put you back into a working state, but I do want to figure out what's wrong and if there's anything more that I can do to fix it

[asottile]

oh I know what happened, yeah I'll work on fixing this

the python version changed, but since virtualenv 20.x the executable is just a symlink -- so the symlink was valid and the "healthy" check succeeded but site-packages was at a completely different path resulting in import errors

[ahonnecke]

My apologies, I didn't look at this before I tried blowing away the cache, removing it resolved it:
ahonnecke@homonym:~/.cache$ rm -rf pre-commit
and then
pre-commit run --all-files

Thanks!

[ahonnecke]

I did list the .cache directory if that's of any help:
drwxr-xr-x 12 ahonnecke ahonnecke 4096 May 2 13:07 pre-commit/

[asottile]

ok I'm able to reproduce this in docker, but only after doing a strange thing to teh system python -- but in theory it seems possible to end up in this case in some other way

FROM ubuntu:bionic AS builder
RUN : \
    && apt-get update \
    && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
        ca-certificates \
        curl \
        git \
        python3 \
        python3-distutils \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# I couldn't figure out how to force virtualenv to symlink to python3
# so this hack works :shrugs:
RUN mv /usr/bin/python3.6 /usr/bin/python3

ENV PATH=/venv/bin:$PATH
RUN : \
    && curl https://asottile.github.io/get-virtualenv.py | python3 - /venv \
    && pip install pre-commit

RUN : \
    && git clone https://github.com/asottile/astpretty /astpretty \
    && cd /astpretty \
    && echo 'default_language_version:' >> .pre-commit-config.yaml \
    && echo '  python: python3' >> .pre-commit-config.yaml \
    && pre-commit run --all-files

FROM ubuntu:focal AS final
RUN : \
    && apt-get update \
    && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
        ca-certificates \
        curl \
        gcc \
        g++ \
        git \
        python3-dev \
        python3-distutils \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

ENV PATH=/venv/bin:$PATH
RUN : \
    && curl https://asottile.github.io/get-virtualenv.py | python3 - /venv \
    && pip install pre-commit

COPY --from=builder /root/.cache/pre-commit /root/.cache/pre-commit
COPY --from=builder /astpretty /astpretty
RUN cd /astpretty && pre-commit run --all-files

$ docker build -t test .

...

setup-cfg-fmt............................................................Failed
- hook id: setup-cfg-fmt
- exit code: 1

Traceback (most recent call last):
  File "/root/.cache/pre-commit/repo_vb9lwwz/py_env-python3/bin/setup-cfg-fmt", line 5, in <module>
    from setup_cfg_fmt import main
ModuleNotFoundError: No module named 'setup_cfg_fmt'

mypy.....................................................................Failed
- hook id: mypy
- exit code: 1

Traceback (most recent call last):
  File "/root/.cache/pre-commit/repokpzyv_r7/py_env-python3/bin/mypy", line 5, in <module>
    from mypy.__main__ import console_entry
ModuleNotFoundError: No module named 'mypy'

The command '/bin/sh -c cd /astpretty && pre-commit run --all-files' returned a non-zero code: 1

[ahonnecke]

but only after doing a strange thing to teh system python

That sounds like a thing that I would do. :)

[asottile]

thanks for the issue, this has been released as part of v2.4.0 🎉