Commits on Mar 16, 2020

1. Add cooperative task yielding …

   A single call to `poll` on a top-level task may potentially do a lot of
   work before it returns `Poll::Pending`. If a task runs for a long period
   of time without yielding back to the executor, it can starve other tasks
   waiting on that executor to execute them, or drive underlying resources.
   See for example rust-lang/futures-rs#2047, rust-lang/futures-rs#1957,
   and rust-lang/futures-rs#869. Since Rust does not have a runtime, it is
   difficult to forcibly preempt a long-running task.
   
   Consider a future like this one:
   
   ```rust
   use tokio::stream::StreamExt;
   async fn drop_all<I: Stream>(input: I) {
       while let Some(_) = input.next().await {}
   }
   ```
   
   It may look harmless, but consider what happens under heavy load if the
   input stream is _always_ ready. If we spawn `drop_all`, the task will
   never yield, and will starve other tasks and resources on the same
   executor.
   
   This patch adds a `coop` module that provides an opt-in mechanism for
   futures to cooperate with the executor to avoid starvation. This
   alleviates the problem above:
   
   ```
   use tokio::stream::StreamExt;
   async fn drop_all<I: Stream>(input: I) {
       while let Some(_) = input.next().await {
           tokio::coop::proceed().await;
       }
   }
   ```
   
   The call to [`proceed`] will coordinate with the executor to make sure
   that every so often control is yielded back to the executor so it can
   run other tasks.
   
   The implementation uses a thread-local counter that simply counts how
   many "cooperation points" we have passed since the task was first
   polled. Once the "budget" has been spent, any subsequent points will
   return `Poll::Pending`, eventually making the top-level task yield. When
   it finally does yield, the executor resets the budget before
   running the next task.
   
   The budget per task poll is currently hard-coded to 128. Eventually, we
   may want to make it dynamic as more cooperation points are added. The
   number 128 was chosen more or less arbitrarily to balance the cost of
   yielding unnecessarily against the time an executor may be "held up".
   
   At the moment, all the tokio leaf futures ("resources") call into coop,
   but external futures have no way of doing so. We probably want to
   continue limiting coop points to leaf futures in the future, but may
   want to also enable third-party leaf futures to cooperate to benefit the
   ecosystem as a whole. This is reflected in the methods marked as `pub`
   in `mod coop` (even though the module is only `pub(crate)`). We will
   likely also eventually want to expose `coop::limit`, which enables
   sub-executors and manual `impl Future` blocks to avoid one sub-task
   spending all of their poll budget.
   
   Benchmarks (see #2160) suggest that the overhead of `coop`
   is marginal.

   

   jonhoo committed Mar 16, 2020
   Configuration menu

   • View commit details
   • Copy full SHA for 3e5cfe4
   • Browse repository at this point

   Copy the full SHA

   3e5cfe4 View commit details

   Browse the repository at this point in the history