Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix "coroutine was never awaited" warnings #929

Draft
wants to merge 3 commits into
base: branch-0.31
Choose a base branch
from

Conversation

pentschev
Copy link
Member

@pentschev pentschev commented Feb 14, 2023

Fix warnings such as the one below:

sys:1: RuntimeWarning: coroutine 'BlockingMode._arm_worker' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Task was destroyed but it is pending!
task: <Task pending name='Task-22' coro=<BlockingMode._arm_worker() running at /opt/conda/envs/rn-230213/lib/python3.10/site-packages/ucp/continuous_ucx_progress.py:88>>

The fix requires introducing a fairly complex mechanism that checks for references to the UCP context and preemptively stop continuous progress when no listeners or endpoints are alive anymore. This is necessary because it's impossible to cancel a future once the event loop that owns it stops, which will happen in third-party projects where UCX-Py doesn't own the event loop, such as Distributed.

TODO: Verify how to still provide continuous progress for workers that do not make use of listeners, instead relying on ucp.recv(), which receives messages directly on the worker.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant