Remove parser reference to break the cyclic reference to protocol

[humrochagf]

This removes the reference to the parser after losing connection so during the teardown of the protocol there will be no cyclic ref to it freeing up the object to be collected by the garbage collector.

Fixes #1564

[Kludex]

What about h11?

[humrochagf]

h11 frees up memory as expected after the TRACE: <IP:PORT> - HTTP connection lost log that's what lead me to see that the problem was in the protocol and not outside it.

[humrochagf]

Btw mypy is not super happy about setting the parser to None
I tested calling del self.parser instead and it also works and apparently mypy is okay about it (maybe because it's an explicit object destruction and not a type change)

[Kludex]

I'm going to review this in a few hours. If confirmed, I'll create a follow-up release.

Thanks @humrochagf 🙏

[Kludex]

I still have the leak. 😗

[humrochagf]

Ok, maybe we have 2 different issues. One that is directly related to httptools and another that could be related to some dependency lib that leaks on both protocol implementation (btw is your h11 also leaking memory?)

I'm attaching here the memory profiles that I made for this fix, with a before fix and after fix profiles (4 in total because I ran in memray in normal mode with pymalloc and another in leak mode with malloc).

And this is my pip freeze output:

anyio==3.6.1
asgiref==3.5.2
asttokens==2.0.5
attrs==21.4.0
autoflake==1.4
backcall==0.2.0
black==22.6.0
bleach==5.0.1
certifi==2022.6.15
cffi==1.15.1
charset-normalizer==2.1.0
click==8.1.3
commonmark==0.9.1
coverage==6.4.1
coverage-conditional-plugin==0.5.0
cryptography==37.0.4
decorator==5.1.1
docutils==0.19
executing==0.8.3
flake8==3.9.2
ghp-import==2.1.0
graphviz==0.20
h11==0.12.0
httpcore==0.15.0
httptools==0.4.0
httpx==0.23.0
idna==3.3
importlib-metadata==4.12.0
iniconfig==1.1.1
ipdb==0.13.9
ipython==8.4.0
isort==5.10.1
jedi==0.18.1
jeepney==0.8.0
Jinja2==3.1.2
keyring==23.7.0
Markdown==3.4.1
MarkupSafe==2.1.1
matplotlib-inline==0.1.3
mccabe==0.6.1
memray==1.2.0
mergedeep==1.3.4
mkdocs==1.3.0
mkdocs-material==8.3.9
mkdocs-material-extensions==1.0.3
mypy==0.961
mypy-extensions==0.4.3
objgraph==3.5.0
packaging==21.3
parso==0.8.3
pathspec==0.9.0
pexpect==4.8.0
pickleshare==0.7.5
pkginfo==1.8.3
platformdirs==2.5.2
pluggy==1.0.0
prompt-toolkit==3.0.30
ptyprocess==0.7.0
pure-eval==0.2.2
py==1.11.0
pycodestyle==2.7.0
pycparser==2.21
pyflakes==2.3.1
Pygments==2.12.0
pymdown-extensions==9.5
pyparsing==3.0.9
pytest==7.1.2
pytest-mock==3.8.2
python-dateutil==2.8.2
python-dotenv==0.20.0
PyYAML==6.0
pyyaml_env_tag==0.1
readme-renderer==35.0
requests==2.28.1
requests-toolbelt==0.9.1
rfc3986==1.5.0
rich==12.5.1
SecretStorage==3.3.2
six==1.16.0
sniffio==1.2.0
stack-data==0.3.0
starlette==0.20.4
toml==0.10.2
tomli==2.0.1
traitlets==5.3.0
trustme==0.9.0
twine==4.0.1
types-click==7.1.8
types-PyYAML==6.0.9
typing_extensions==4.3.0
urllib3==1.26.10
-e git+ssh://git@github.com/humrochagf/uvicorn.git@2245ad70de28d40e89c53012003272344537482b#egg=uvicorn
uvloop==0.16.0
watchdog==2.1.9
watchfiles==0.16.0
watchgod==0.8.2
wcwidth==0.2.5
webencodings==0.5.1
websockets==10.3
wsproto==1.1.0
zipp==3.8.1

I'll try to run with a fresh env to see if I get the same leak that you are still getting. But in case I don't get the leak, could you please send me your pip freeze output to compare them?

[humrochagf]

Here are the files
memray-uvicorn.zip

[humrochagf]

Alright, I found the second point of leak 🎉
When closing the transport in the connection_lost it skips the unset of the keepalive so the reference to the server remains up.
In uvloop, everything seems to get destroyed during the teardown, so this reference was not leaking on my previous tests.

But when I started a new venv without uvloop I noticed it was still leaking.

Now if you check for references to in both protocols at the end of connection_lost you will see that either h11 and httptools have a consistent number of references, even for disconnection by request cancellation 😃

[Kludex]

Thanks @humrochagf :)