Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elastic Dispatchers.IO unbounded for limitedParallelism #2943

Closed
qwwdfsad opened this issue Sep 20, 2021 · 0 comments
Closed

Elastic Dispatchers.IO unbounded for limitedParallelism #2943

qwwdfsad opened this issue Sep 20, 2021 · 0 comments

Comments

@qwwdfsad
Copy link
Member

Dispatchers.IO legacy

Prior to #2919 and upcoming release 1.6.0, kotlinx.coroutines had no API to limit the parallelism without creating excessive threads. Dispatchers.IO meant to be a dispatcher for "any IO", similar to Rx-like io and newCachedThreadPool.
But having an unlimited number of threads has brittle and dangerous pitfalls: the performance of an overloaded application degrades in a slow, but unlimited manner, eventually causing out-of-memory errors.
Because of that, it was decided to have a "good enough" upper limit (64 seemed to be a reasonable constant at that moment, especially for Android).

After #2919, along with newFixedThreadPoolContext becoming @DelicateCoroutinesApi, we would advocate for using e.g. Dispatchers.IO.limitedParallelism(myDbConnectionPoolSize) instead of newFixedThreadPoolContext(myDbConnectionPoolSize).

Such recommendation has a fatal flaw: -- nowadays the number of threads allocated for DB connections, blocking IO, and networks is measured in hundreds, so as soon as the user has enough "limited" slices of Dispatchers.IO, the system will start starving only during peak loads. It's almost impossible to catch such behavior during code-review, (regular) testing, and regular operational mode. Shortly speaking, this behavior is just a timebomb that will explode at the least expected moment.

Proposed change

To avoid degradations during peak loads, while still providing fluent and hard-to-misuse API, the following changes will be implemented:

  • Dispatchers.IO will still be limited to 64 threads with the limit being a fail-safe mechanism
  • Dispatchers.IO.limitedParallelism(n) won't be limited by the hard limit of the original Dispatchers.IO, but instead will be limited by the given parallelism
  • Multiple slices of Dispatchers.IO will still be conceptually backed by the same thread-pool, being able to leverage both reduced number of threads and reduced number of context-switches due to its built-in integration with Dispatchers.Defailt

Basically, the mental model (but not the implementation!) is the following:

private val unlimitedThreadsDispatcherIO = ...

Dispatchers.IO = unlimitedThreadsDispatcherIO.limitParallelism(64)
Dispatchers.IO.limitParallelism(x) = unlimitedThreadsDispatcherIO.limitParallelism(x)
qwwdfsad added a commit that referenced this issue Sep 21, 2021
    * Extract Ktor-obsolete API to a separate file for backwards compatibility
    * Make Dispatchers.IO being a slice of unlimited blocking scheduler
    * Make Dispatchers.IO.limitParallelism take slices from the same internal scheduler

Fixes #2943
yorickhenning pushed a commit to yorickhenning/kotlinx.coroutines that referenced this issue Oct 14, 2021
….IO unbounded for limited parallelism (Kotlin#2918)

* Introduce CoroutineDispatcher.limitedParallelism for granular concurrency control

* Elastic Dispatchers.IO:

    * Extract Ktor-obsolete API to a separate file for backwards compatibility
    * Make Dispatchers.IO being a slice of unlimited blocking scheduler
    * Make Dispatchers.IO.limitParallelism take slices from the same internal scheduler

Fixes Kotlin#2943
Fixes Kotlin#2919
pablobaxter pushed a commit to pablobaxter/kotlinx.coroutines that referenced this issue Sep 14, 2022
….IO unbounded for limited parallelism (Kotlin#2918)

* Introduce CoroutineDispatcher.limitedParallelism for granular concurrency control

* Elastic Dispatchers.IO:

    * Extract Ktor-obsolete API to a separate file for backwards compatibility
    * Make Dispatchers.IO being a slice of unlimited blocking scheduler
    * Make Dispatchers.IO.limitParallelism take slices from the same internal scheduler

Fixes Kotlin#2943
Fixes Kotlin#2919
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant