Questions regarding contributing guidance

[rosydawn6]

I have a few questions regarding: https://github.com/mitmproxy/mitmproxy/blob/main/CONTRIBUTING.md

1. tox -e py is successful but the next recommended command

cd test/mitmproxy/addons
pytest --cov mitmproxy.addons.anticache --cov-report term-missing --looponfail test_anticache.py

gives me errors:

../../../mitm_venv/lib/python3.11/site-packages/mitmproxy_rs/__init__.py:1: in <module>
    from .mitmproxy_rs import *
E   ImportError: PyO3 modules may only be initialized once per interpreter process

---------- coverage: platform linux, python 3.11.6-final-0 -----------
Name                                                                      Stmts   Miss  Cover   Missing
-------------------------------------------------------------------------------------------------------
./mitmproxy/mitmproxy/mitmproxy/addons/anticache.py       7      6    14%   4-18
-------------------------------------------------------------------------------------------------------
TOTAL                                                                         7      6    14%

=========================== short test summary info ============================
ERROR test_anticache.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.42s ===============================
################################################################### waiting for changes ###################################################################
### Watching:   ./mitmproxy/mitmproxy/test/mitmproxy/addons

Any ideas as to how to fix this?

2. tox -e lint gives many errors but when I made the PR none were found by the automated system:

> Found 8370 errors.
> [*] 3535 fixable with the `--fix` option (182 hidden fixes can be enabled with the `--unsafe-fixes` option).
> ERROR: InvocationError for command ./mitmproxy/mitmproxy/.tox/lint/bin/ruff . (exited with code 1)
> _________________________________________________________________________ summary _________________________________________________________________________
> ERROR:   lint: commands failed
> 

[mhils]

pytest --cov mitmproxy.addons.anticache --cov-report term-missing --looponfail test_anticache.py

It looks like we're running into PyO3/pyo3#3446 here - I'm actually quite surprised this does not happen for all invocations. Anyhow, it looks like this should fix itself once all our PyO3-based dependencies have upgraded to PyO3>=0.20. I've removed the offending parts for now, and opened #6520 to re-add it.

tox -e lint

This works just fine for me:

$ pwd
/Users/user/git/mitmproxy
$ tox -e lint --recreate
lint: remove tox env folder .tox/lint
lint: install_deps> python -I -m pip install 'ruff<0.2,>=0.1.3'
lint: commands[0]> ruff .
  lint: OK (6.34=setup[5.76]+cmd[0.57] seconds)
  congratulations :) (6.36 seconds)

Are you doing anything different? Are you on main?

[rosydawn6]

I was on stream_large_bodies_warn_with_modify_body when I got the error. I switched back to main a few moments ago and first fetched and pulled into main and then reran

tox -e lint

and got a similar error. Then I tried the --recreate option to be sure:

tox -e lint --recreate

and got a similar error.

lint recreate: ./mitmproxy/mitmproxy/.tox/lint
lint installdeps: ruff>=0.1.3,<0.2
lint installed: ruff==0.1.6
lint run-test: commands[0] | ruff .

...
...
...
mitm_venv/lib/python3.11/site-packages/zstandard/__init__.py:52:13: F403 `from .backend_cffi import *` used; unable to detect undefined names
mitm_venv/lib/python3.11/site-packages/zstandard/__init__.py:57:9: F403 `from .backend_c import *` used; unable to detect undefined names
mitm_venv/lib/python3.11/site-packages/zstandard/__init__.py:61:9: F403 `from .backend_cffi import *` used; unable to detect undefined names
mitm_venv/lib/python3.11/site-packages/zstandard/__init__.py:65:5: F403 `from .backend_rust import *` used; unable to detect undefined names
mitm_venv/lib/python3.11/site-packages/zstandard/__init__.py:69:5: F403 `from .backend_c import *` used; unable to detect undefined names
mitm_venv/lib/python3.11/site-packages/zstandard/__init__.py:73:5: F403 `from .backend_cffi import *` used; unable to detect undefined names
mitm_venv/lib/python3.11/site-packages/zstandard/__init__.py:134:24: F405 `ZstdDecompressor` may be undefined, or defined from star imports
mitm_venv/lib/python3.11/site-packages/zstandard/__init__.py:138:24: F405 `ZstdCompressor` may be undefined, or defined from star imports
mitm_venv/lib/python3.11/site-packages/zstandard/__init__.py:190:12: F405 `ZstdCompressor` may be undefined, or defined from star imports
mitm_venv/lib/python3.11/site-packages/zstandard/__init__.py:208:12: F405 `ZstdDecompressor` may be undefined, or defined from star imports
mitm_venv/lib/python3.11/site-packages/zstandard/__init__.pyi:7:1: I001 [*] Import block is un-sorted or un-formatted
mitm_venv/lib/python3.11/site-packages/zstandard/backend_cffi.py:9:1: I001 [*] Import block is un-sorted or un-formatted
mitm_venv/lib/python3.11/site-packages/zstandard/backend_cffi.py:86:1: I001 [*] Import block is un-sorted or un-formatted
mitm_venv/lib/python3.11/site-packages/zstandard/backend_cffi.py:2856:9: E741 Ambiguous variable name: `l`
Found 8370 errors.
[*] 3535 fixable with the `--fix` option (182 hidden fixes can be enabled with the `--unsafe-fixes` option).
ERROR: InvocationError for command ./mitmproxy/mitmproxy/.tox/lint/bin/ruff . (exited with code 1)
_____________________________________________________________________________________ summary ______________________________________________________________________________________
ERROR:   lint: commands failed

[mhils]

mitm_venv looks like a custom virtualenv directory on your mitmproxy directory, tripping up ruff which checks your entire mitmproxy dir. :)

[rosydawn6]

Confirming that when the contributing guidance is followed exactly including naming the virtual environment venv everything works. On the other hand: ./mitmproxy/mitmproxy/.tox/lint/bin/ruff --exclude=mitm_venv is able to exclude custom directory and could be added to a developer/local pyproject.toml config

[rosydawn6]

I would like to put a note in the contributing guidelines briefly mentioning some of these findings. Would you be ok with that PR?

[mhils]

I've already addressed the first point in CONTRIBUTING.md, and I don't feel that mitm_venv crosses the bar in terms of useful information / added amount of text. But please do feel free to open a PR with suggestions if you feel there are other things that should be mentioned! :)