Fix memory leak

[sergei-maertens]

There is a memory leak in the /api/ endpoint - it's causing the memory usage of the container to grow and eventually get OOM-killed (because we do set our memory limits).

The /api/ endpoint is used as health check, but of course this should not be memory leaking in the first place.

Possible mitigations:

• cache the API schema and serve the cache
• use another _health endpoint for the health check

[sergei-maertens]

Some debugging with mem_top seems to confirm that drf_spectacular is problematic:

refs:
5742	<class 'list'> ['# module pyparsing.py\n', '#\n', '# Copyright (c) 2003-2018  Paul T. McGuire\n', '#\n', '# Permiss
5404	<class 'dict'> {PosixPath('/home/bbt/.virtualenvs/zac/lib/python3.9/site-packages/elasticsearch/client/rollup.py'):
3364	<class 'dict'> {140013453740288: <weakref at 0x7f576b8d4db0; to 'type' at 0x7f576c2c8900 (type)>, 140013453737696: 
2562	<class 'dict'> {<weakref at 0x7f576512bf90; to 'ActivityViewSet' at 0x7f576a7f40a0>: <drf_spectacular.utils.extend_
2514	<class 'dict'> {(('urn:oasis:names:tc:opendocument:xmlns:animation:1.0', 'audio-level'), None): <function cnv_doubl
2180	<class 'dict'> {'sys': <module 'sys' (built-in)>, 'builtins': <module 'builtins' (built-in)>, '_frozen_importlib': 
2180	<class 'tuple'> (<module 'MarkupPy' from '/home/bbt/.virtualenvs/zac/lib/python3.9/site-packages/MarkupPy/__init__.p
2041	<class 'frozenset'> frozenset({PosixPath('/home/bbt/.virtualenvs/zac/lib/python3.9/site-packages/elasticsearch/client/ro
1526	<class 'list'> ['"""Thread module emulating a subset of Java\'s threading model."""\n', '\n', 'import os as _os\n',
1311	<class 'dict'> {'__name__': 'lib', '__doc__': None, '__package__': None, '__loader__': None, '__spec__': None, '_or

bytes:
147552	 {'sys': <module 'sys' (built-in)>, 'builtins': <module 'builtins' (built-in)>, '_frozen_importlib': 
73816	 {140013453740288: <weakref at 0x7f576b8d4db0; to 'type' at 0x7f576c2c8900 (type)>, 140013453737696: 
73816	 {PosixPath('/home/bbt/.virtualenvs/zac/lib/python3.9/site-packages/elasticsearch/client/rollup.py'):
65752	 frozenset({PosixPath('/home/bbt/.virtualenvs/zac/lib/python3.9/site-packages/elasticsearch/client/ro
47160	 ['# module pyparsing.py\n', '#\n', '# Copyright (c) 2003-2018  Paul T. McGuire\n', '#\n', '# Permiss
36960	 {'__module__': 'lxml.etree', '__doc__': 'Libxml2 error types', '__dict__': <attribute '__dict__' of 
36960	 {'application/javascript': ['.js', '.mjs'], 'application/json': ['.json'], 'application/manifest+jso
36960	 {(('urn:oasis:names:tc:opendocument:xmlns:animation:1.0', 'audio-level'), None): <function cnv_doubl
36960	 {'CRYPTOGRAPHY_PACKAGE_VERSION': <cdata 'char *' 0x7f57665bf000>, 'Cryptography_HAS_EC2M': 1, 'Crypt
36960	 {'__name__': 'lib', '__doc__': None, '__package__': None, '__loader__': None, '__spec__': None, '_or

types:
89589	 <class 'dict'>
45074	 <class 'list'>
35322	 <class 'tuple'>
33857	 <class 'function'>
18048	 <class 'collections.OrderedDict'>
10647	 <class 'weakref'>
9547	 <class 'cell'>
5134	 <class 'type'>
4784	 <class 'pathlib.PosixPath'>
4183	 <class 'django.core.validators.ProhibitNullCharactersValidator'>

some time later

refs:
5742	<class 'list'> ['# module pyparsing.py\n', '#\n', '# Copyright (c) 2003-2018  Paul T. McGuire\n', '#\n', '# Permiss
5404	<class 'dict'> {PosixPath('/home/bbt/.virtualenvs/zac/lib/python3.9/site-packages/elasticsearch/client/rollup.py'):
4354	<class 'dict'> {<weakref at 0x7f576512bf90; to 'ActivityViewSet' at 0x7f576a7f40a0>: <drf_spectacular.utils.extend_
3364	<class 'dict'> {140013453740288: <weakref at 0x7f576b8d4db0; to 'type' at 0x7f576c2c8900 (type)>, 140013453737696: 
2514	<class 'dict'> {(('urn:oasis:names:tc:opendocument:xmlns:animation:1.0', 'audio-level'), None): <function cnv_doubl
2180	<class 'dict'> {'sys': <module 'sys' (built-in)>, 'builtins': <module 'builtins' (built-in)>, '_frozen_importlib': 
2180	<class 'tuple'> (<module 'MarkupPy' from '/home/bbt/.virtualenvs/zac/lib/python3.9/site-packages/MarkupPy/__init__.p
2041	<class 'frozenset'> frozenset({PosixPath('/home/bbt/.virtualenvs/zac/lib/python3.9/site-packages/elasticsearch/client/ro
1526	<class 'list'> ['"""Thread module emulating a subset of Java\'s threading model."""\n', '\n', 'import os as _os\n',
1311	<class 'dict'> {'__name__': 'lib', '__doc__': None, '__package__': None, '__loader__': None, '__spec__': None, '_or

bytes:
147552	 {'sys': <module 'sys' (built-in)>, 'builtins': <module 'builtins' (built-in)>, '_frozen_importlib': 
73816	 {140013453740288: <weakref at 0x7f576b8d4db0; to 'type' at 0x7f576c2c8900 (type)>, 140013453737696: 
73816	 {PosixPath('/home/bbt/.virtualenvs/zac/lib/python3.9/site-packages/elasticsearch/client/rollup.py'):
73816	 {<weakref at 0x7f576512bf90; to 'ActivityViewSet' at 0x7f576a7f40a0>: <drf_spectacular.utils.extend_
65752	 frozenset({PosixPath('/home/bbt/.virtualenvs/zac/lib/python3.9/site-packages/elasticsearch/client/ro
47160	 ['# module pyparsing.py\n', '#\n', '# Copyright (c) 2003-2018  Paul T. McGuire\n', '#\n', '# Permiss
36960	 {'__module__': 'lxml.etree', '__doc__': 'Libxml2 error types', '__dict__': <attribute '__dict__' of 
36960	 {'application/javascript': ['.js', '.mjs'], 'application/json': ['.json'], 'application/manifest+jso
36960	 {(('urn:oasis:names:tc:opendocument:xmlns:animation:1.0', 'audio-level'), None): <function cnv_doubl
36960	 {'CRYPTOGRAPHY_PACKAGE_VERSION': <cdata 'char *' 0x7f57665bf000>, 'Cryptography_HAS_EC2M': 1, 'Crypt

types:
133115	 <class 'dict'>
64550	 <class 'list'>
42050	 <class 'tuple'>
35333	 <class 'function'>
30305	 <class 'collections.OrderedDict'>
12517	 <class 'weakref'>
10749	 <class 'cell'>
6997	 <class 'django.core.validators.ProhibitNullCharactersValidator'>
6967	 <class 'rest_framework.validators.ProhibitSurrogateCharactersValidator'>
6494	 <class 'drf_spectacular.plumbing.ResolvedComponent'>

[sergei-maertens]

526731c is the first step in treating the symptoms, but of course that doesn't fix the memory leak itself yet

[sergei-maertens]

Upstream bug report: tfranzel/drf-spectacular#597