Skip to content

Introduce CoroutineDispatcher.limitedParallelism as an alternative to standalone executors and slices #2919

Closed
@qwwdfsad

Description

@qwwdfsad
Member

See the original idea behind slices here: #261
Since then, a lot of API was introduced and refined, so the proposed API no longer fits kotlinx.coroutines.

Use-cases

newFixedThreadPoolContext and newSingleThreadContext, as well as regular Java executors, are often used as the only way to control concurrency. Various factors contribute to that -- limited external resources (e.g. pool of database connections), functional restrictions (e.g. CPU consumption or rate-limiting), or need to have an isolated serializable execution flow (an extreme example would be IDEA code-base that contains 62 single-threaded dispatchers).

As a result, an average application creates unnecessary many threads, most of which are idle, consuming the memory, CPU, and device battery.

Design overview

To address such use-cases, we aim to provide API to take a view of the dispatcher. The dispatcher's view does not create any additional threads and does not allocate any resources. Instead, it piggybacks the original dispatcher but limits the parallelism of submitted tasks and coroutines. Views themselves are independent and it is possible to have views that have more declared parallelism in total than the original dispatchers. It saves users from meticulous counting of resources, especially when views are used along with global Dispatchers.Default and Dispatchers.IO.

With that API, it is possible to have fine-grained control over the parallelism and leverage already existing dispatchers and executors, saving resources and the number of threads.

The API changes are minimal, i.e. instead of

private val myDatabaseDispatcher = newFixedThreadPool(10, "DB")  // At most 10 connections to the database
private val myCpuHeavyImageProcessingDispatcher = newFixedThreadPool(2, "Image rescaling") // Prevent more than 200% of CPU consumption on images

it is possible to write this instead:

private val myDatabaseDispatcher = Dispatchers.IO.limitedParallelism(10)  // At most 10 connections to the database
private val myCpuHeavyImageProcessingDispatcher = Dispatchers.Default.limitedParallelism(2) // Prevent more than 200% of CPU consumption on images

API changes

Conceptually, any coroutine dispatcher can limit its own parallelism. If we are allowing the sum of views to be greater than the original dispatcher, nothing prevents users to take a view of a single-threaded dispatcher (e.g. Dispatchers.Main) as long as its contract allows it to short-circuit the implementation to return this.

As a result, a single public signature will be added to CoroutineDispatcher class

public abstract class CoroutineDispatcher {

    public open fun limitedParallelism(parallelism: Int): CoroutineDispatcher
   
    ...the rest of the methods...
}

with the default implementation provided.

Rejected alternative namings

Dispatchers.Default.slice(parallelism = 2)
Dispatchers.Default.view(2)
Dispatchers.Default.limited(parallelism = 2)
Dispatchers.Default.subset(2)
Dispatchers.Default.dedicated(2)

While being short and visually attractive, it's easy to misinterpret their semantics.

  • For slice, subset, and view it can be reasonable to assume that the sum of all the slices/subsets/views cannot be greater than the original
  • For dedicated it can be reasonable to assume that dedicated threads are used for such dispatcher
  • The biggest concern for limited is its name. We cannot enforce the usage of named parameters on the language level, and seeing myDispatcher.limited(2) or myDispatcher.limited(myLimit) in the wild may be quite misleading.

Activity

self-assigned this
on Sep 6, 2021
changed the title [-]Introduce CoroutineDispatcher.limitedParallelism as an alternative to standalone executors and slices[/-] [+]Introduce `CoroutineDispatcher.limitedParallelism` as an alternative to standalone executors and slices[/+] on Sep 6, 2021
LouisCAD

LouisCAD commented on Sep 6, 2021

@LouisCAD
Contributor

Very nice API, I like the interesting capabilities it can give with such simplicity.

qwwdfsad

qwwdfsad commented on Sep 20, 2021

@qwwdfsad
MemberAuthor

The complementary proposal: #2943

fvasco

fvasco commented on Sep 20, 2021

@fvasco
Contributor

This proposal is a great improvement of newFixedThreadPoolContext, are you considered to use simply Dispatchers.limitedParallelism(10) or similar?

added a commit that references this issue on Oct 12, 2021
1df0be5

11 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @qwwdfsad@ultraon@patkujawa-wf@LouisCAD@fvasco

      Issue actions

        Introduce `CoroutineDispatcher.limitedParallelism` as an alternative to standalone executors and slices · Issue #2919 · Kotlin/kotlinx.coroutines