From 5e6413e6ada489afa8b009cc246762c293cbdf8b Mon Sep 17 00:00:00 2001 From: "mergify[bot]" <37929162+mergify[bot]@users.noreply.github.com> Date: Fri, 13 May 2022 10:36:16 -0400 Subject: [PATCH] fix(profiling): do not clear thread-loop link too often (#3668) (#3711) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The current code calls DdtraceProfilerEventLoopPolicy.clear_threads each time a task is resolved, which can be up to 100 times a second. This is way too much and we don't expect for the thread->loop mapping to change that often. Instead we often try to clear the mapping when a new loop is attached to a thread. In a regular application, this is the expected workflow: a thread appears and gets a new loop, so we clear the old ones. The worst case scenario would be an app spawning 100 threads with 100 loops and then not doing that ever again, which would make the profiler keep a reference on the 100 loops — until a new loop is attached, which if never, would kept the reference forever. Since this far from being a common pattern, it should be safe to switch to a simpler model like this. (cherry picked from commit f583fec63c4392a0784b4199b0e20931f9aae9b5) Co-authored-by: Julien Danjou --- ddtrace/profiling/_asyncio.py | 1 - ...profiling-asyncio-less-clear-threads-33d3436ab76ea158.yaml | 4 ++++ 2 files changed, 4 insertions(+), 1 deletion(-) create mode 100644 releasenotes/notes/profiling-asyncio-less-clear-threads-33d3436ab76ea158.yaml diff --git a/ddtrace/profiling/_asyncio.py b/ddtrace/profiling/_asyncio.py index fa177dfd021..e70ed389e97 100644 --- a/ddtrace/profiling/_asyncio.py +++ b/ddtrace/profiling/_asyncio.py @@ -72,5 +72,4 @@ def set_event_loop(self, loop): def _ddtrace_get_loop(self, thread_id): # type: (...) -> typing.Optional[asyncio.AbstractEventLoop] - self._clear_threads() return self.loop_per_thread.get_object(thread_id) diff --git a/releasenotes/notes/profiling-asyncio-less-clear-threads-33d3436ab76ea158.yaml b/releasenotes/notes/profiling-asyncio-less-clear-threads-33d3436ab76ea158.yaml new file mode 100644 index 00000000000..1b9d17863da --- /dev/null +++ b/releasenotes/notes/profiling-asyncio-less-clear-threads-33d3436ab76ea158.yaml @@ -0,0 +1,4 @@ +--- +fixes: + - | + Fixes a performance issue with the profiler when used in an asyncio application.