RuntimeWarnings in IPython when asyncio integration is active

[iamlikeme]

Summary of problem

IPython produces RuntimeWarning on each tab-completion and enter when ddtrace's asyncio integration is active.

Which version of dd-trace-py are you using?

2.8.0

Which version of pip are you using?

24.0

Which libraries and their versions are you using?

ipython==8.23.0
ddtrace==2.8.0

`pip freeze`

asttokens==2.4.1 attrs==23.2.0 bytecode==0.15.1 cattrs==23.2.3 ddsketch==3.0.1 ddtrace==2.8.0 decorator==5.1.1 Deprecated==1.2.14 envier==0.5.1 executing==2.0.1 importlib-metadata==7.0.0 ipython==8.23.0 jedi==0.19.1 matplotlib-inline==0.1.6 opentelemetry-api==1.24.0 parso==0.8.4 pexpect==4.9.0 prompt-toolkit==3.0.43 protobuf==5.26.1 ptyprocess==0.7.0 pure-eval==0.2.2 Pygments==2.17.2 setuptools==69.2.0 six==1.16.0 sqlparse==0.4.4 stack-data==0.6.3 traitlets==5.14.2 typing_extensions==4.11.0 wcwidth==0.2.13 wrapt==1.16.0 xmltodict==0.13.0 zipp==3.18.1

How can we reproduce your problem?

In an IPython REPL:

In [1]: import ddtrace

In [2]: ddtrace.patch_all()

In [3]: # press enter
/home/piotr/repos/ddtrace-bug/.venv/lib/python3.12/site-packages/prompt_toolkit/application/application.py:1207: RuntimeWarning: coroutine 'Application.run_async.<locals>._run_async.<locals>.auto_flush_input' was never awaited
  await asyncio.wait(
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

/home/piotr/repos/ddtrace-bug/.venv/lib/python3.12/site-packages/prompt_toolkit/key_binding/key_processor.py:413: RuntimeWarning: coroutine 'KeyProcessor._start_timeout.<locals>.wait' was never awaited
  self._flush_wait_task = app.create_background_task(wait())
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

In [3]:

The problem disappears when asyncio integration is deactivated:

In [1]: import ddtrace

In [2]: ddtrace.patch_all(asyncio=False)

In [3]: # press enter

In [3]:

What is the result that you get?

I see RuntimeWarnings on every enter and tab-completion in IPython.

What is the result that you expected?

I expected to see no RuntimeWarnings 🙂

[emmettbutler]

Interesting! Thanks for the report, @iamlikeme. We'll check it out.

[darkwind-noveo]

I encountered same issue when navigating with tmate in Ipython shell console

In [1]: ls
/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py:1207: RuntimeWarning: coroutine 'Application.run_async.<locals>._run_async.<locals>.auto_flush_input' was never awaited
  await asyncio.wait(
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
CONTRIBUTING.md  Pipfile.lock  certificates/    scripts/    tests/
Dockerfile       README.md     docker/        locale/       pylintrc    setup.cfg   uwsgi-json.ini
Pipfile            manage.py*    pytest.ini  templates/  uwsgi.ini

/usr/local/lib/python3.11/site-packages/prompt_toolkit/key_binding/key_processor.py:413: RuntimeWarning: coroutine 'KeyProcessor._start_timeout.<locals>.wait' was never awaited
  self._flush_wait_task = app.create_background_task(wait())
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

When i downgraded ddtrace from 2.8.2 to 2.7.2 it resolved my issues. (2.7.10 is good too, 2.8.0 breaks)

I tried turning on python tracemalloc stuff via env var, then written error changed itself to

In [1]: 
/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py:1207: RuntimeWarning: coroutine 'Application.run_async.<locals>._run_async.<locals>.auto_flush_input' was never awaited
  await asyncio.wait(
Object allocated at (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py", lineno 704
    flush_task = self.create_background_task(auto_flush_input())

/usr/local/lib/python3.11/site-packages/prompt_toolkit/key_binding/key_processor.py:413: RuntimeWarning: coroutine 'KeyProcessor._start_timeout.<locals>.wait' was never awaited
  self._flush_wait_task = app.create_background_task(wait())
Object allocated at (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/key_binding/key_processor.py", lineno 413
    self._flush_wait_task = app.create_background_task(wait())

My pip freeze

aiohttp==3.9.5
aioresponses==0.7.6
aiosignal==1.3.1
aiostream==0.5.2
amqp==5.2.0
annotated-types==0.6.0
appdirs==1.4.4
asgiref==3.8.1
astroid==3.1.0
asttokens==2.4.1
async-timeout==4.0.3
asyncpg==0.29.0
attrs==23.2.0
Babel==2.14.0
bcrypt==4.1.2
beautifulsoup4==4.12.3
billiard==4.2.0
boto3==1.28.80
botocore==1.31.80
braintree==3.35.0
Brotli==1.1.0
bytecode==0.15.1
cached-property==1.5.2
cartomancer==0.1.4
cattrs==23.2.3
celery==5.3.1
certifi==2024.2.2
cffi==1.16.0
charset-normalizer==3.3.2
click==8.1.7
click-didyoumean==0.3.1
click-plugins==1.1.1
click-repl==0.3.0
colorama==0.4.6
coverage==6.5.0
coveralls==3.3.1
cron-descriptor==1.4.3
cryptography==42.0.5
cssselect2==0.7.0
cssutils==2.10.2
ddsketch==3.0.1
ddtrace==2.8.2
debugpy==1.8.1
decorator==5.1.1
defusedxml==0.7.1
Deprecated==1.2.14
dill==0.3.8
distlib==0.3.8
Django==4.1.13
django-add-default-value==0.10.0
django-appconf==1.0.6
django-cassette==0.8.1
django-celery-beat==2.5.0
django-chunkator==2.0.0
django-cors-headers==4.3.1
django-countries==7.6.1
django-crispy-forms==2.0
django-datadog-logger==0.7.1
django-extensions==3.2.3
django-filter==22.1
django-fsm==3.0.0
django-fsm-log==3.1.0
django-localflavor==4.0
django-model-utils==4.5.0
django-ratelimit==4.1.0
django-redis==5.4.0
django-relativedelta==2.0.0
django-request-id==1.0.0
django-ses==4.0.0
django-storages==1.14.2
django-timezone-field==6.1.0
django-tqdm==1.3.1
djangorestframework==3.14.0
djangorestframework-filters==1.0.0.dev2
docker==7.0.0
docopt==0.6.2
drf-flex-fields==1.0.2
elastic-apm==2.1.1
elasticsearch==5.5.3
elasticsearch-dsl==5.4.0
elementpath==4.4.0
envier==0.5.1
et-xmlfile==1.1.0
execnet==2.1.1
executing==2.0.1
factory-boy==3.2.1
Faker==24.11.0
filelock==3.14.0
flake8==7.0.0
flake8-tidy-imports==4.10.0
flowey==1.4.5
fonttools==4.51.0
freezegun==1.4.0
frozenlist==1.4.1
getconf==1.11.1
googleapis-common-protos==1.63.0
grpcio==1.62.2
grpcio-tools==1.62.2
html2text==2024.2.26
html5lib==1.1
idna==3.7
importlib-metadata==7.0.0
iniconfig==2.0.0
ipaddress==1.0.23
ipython==8.23.0
isodate==0.6.1
isort==5.13.2
jdcal==1.4.1
jedi==0.19.1
jmespath==1.0.1
JSON-log-formatter==1.0
jsonfield==3.1.0
junitparser==3.1.2
jwcrypto==0.8
kombu==5.3.1
lark==1.1.9
lxml==5.2.1
Markdown==3.6
MarkupSafe==2.1.5
matplotlib-inline==0.1.7
mccabe==0.7.0
mistql==0.4.11
multidict==6.0.5
oauthlib==3.2.2
openpyxl==3.0.6
opentelemetry-api==1.24.0
opentelemetry-exporter-otlp==1.24.0
opentelemetry-exporter-otlp-proto-common==1.24.0
opentelemetry-exporter-otlp-proto-grpc==1.24.0
opentelemetry-exporter-otlp-proto-http==1.24.0
opentelemetry-instrumentation==0.45b0
opentelemetry-instrumentation-asyncpg==0.45b0
opentelemetry-instrumentation-boto==0.45b0
opentelemetry-instrumentation-boto3sqs==0.45b0
opentelemetry-instrumentation-celery==0.45b0
opentelemetry-instrumentation-dbapi==0.45b0
opentelemetry-instrumentation-django==0.45b0
opentelemetry-instrumentation-elasticsearch==0.45b0
opentelemetry-instrumentation-psycopg2==0.45b0
opentelemetry-instrumentation-redis==0.45b0
opentelemetry-instrumentation-requests==0.45b0
opentelemetry-instrumentation-wsgi==0.45b0
opentelemetry-proto==1.24.0
opentelemetry-sdk==1.24.0
opentelemetry-semantic-conventions==0.45b0
opentelemetry-util-http==0.45b0
packaging==24.0
parso==0.8.4
pexpect==4.9.0
pillow==10.3.0
pipenv==2023.12.1
platformdirs==4.2.0
pluggy==1.5.0
prompt-toolkit==3.0.43
protobuf==4.25.3
psutil==5.9.8
psycopg2==2.9.9
psycopg2-binary==2.9.9
ptyprocess==0.7.0
pure-eval==0.2.2
py==1.11.0
pycodestyle==2.11.1
pycparser==2.22
pycurl==7.45.3
pydantic==2.7.0
pydantic-settings==2.2.1
pydantic_core==2.18.1
pydyf==0.9.0
pyflakes==3.2.0
Pygments==2.17.2
PyJWT==2.8.0
pylint==3.1.0
PyNaCl==1.5.0
pynliner==0.8.0
pyphen==0.14.0
pypng==0.20220715.0
pytest==7.4.4
pytest-asyncio==0.23.6
pytest-celery==1.0.0
pytest-cov==4.1.0
pytest-django==4.8.0
pytest-docker-tools==3.1.3
pytest-lazy-fixture==0.6.3
pytest-split==0.8.2
pytest-watch==4.2.0
pytest-xdist==3.3.1
python-crontab==3.0.0
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
python-stdnum==1.20
pytz==2024.1
PyYAML==6.0.1
qrcode==7.4.2
redis==5.0.3
requests==2.31.0
requests-aws4auth==1.2.3
requests-mock==1.12.1
requests-oauthlib==2.0.0
requests-toolbelt==1.0.0
responses==0.25.0
retry==0.9.2
s3transfer==0.7.0
sentry-sdk==1.45.0
sepaxml==2.1.1
simple-term-menu==1.6.4
six==1.16.0
soupsieve==2.5
spider==1.0.34
sqlparse==0.5.0
stack-data==0.6.3
suds-community==1.1.2
symba==0.36
tablib==3.6.1
text-unidecode==1.3
tinycss2==1.2.1
tokipona==1.4.0
tomlkit==0.12.4
tqdm==4.66.2
traitlets==5.14.3
typeguard==2.13.3
typelog==0.3.0
typing_extensions==4.11.0
tzdata==2024.1
unittest-xml==0.2.2
unittest-xml-reporting==3.2.0
urllib3==2.0.7
uWSGI==2.0.25.1
vine==5.1.0
virtualenv==20.26.2
watchdog==4.0.0
wcwidth==0.2.13
weasyprint==61.2
webencodings==0.5.1
Werkzeug==3.0.2
wrapt==1.16.0
xlrd==2.0.1
XlsxWriter==3.2.0
xlwt==1.3.0
xmlschema==3.3.0
xmltodict==0.13.0
yarl==1.9.4
zeep==3.4.0
zipp==3.18.1
zopfli==0.2.3

pip version 24.0

Confirming that solution DD_TRACE_ASYNCIO_ENABLED=false python3 manage.py shell helped me too.