API to work with event loop in native while we wait for #462

[benasher44]

Hi there! I'm the maintainer for https://github.com/autodesk/coroutineworker. This allows us to do multi-threaded-like coroutine-based work on native, while we (the community) wait on #462. We have an API that allows suspending the current coroutine on a thread while we wait for work in a coroutine on another thread to finish, which allows some level of cross-thread communication. However due to current API limitations, we're stuck implementing a hacked together suspend-y future centered around a while loop and an atomic reference: https://github.com/Autodesk/coroutineworker/blob/master/src/nativeMain/kotlin/com/autodesk/coroutineworker/CoroutineUtils.kt#L32.

I'm curious if you all have any ideas around improving this. My first thought is that it would be great if there could be a native-only API (could even be marked as experimental or something, since #462 would obsolete it) that would allow adding a future event (maybe just calls a closure) to the event loop, which would would start in a not-ready state. Then later on, when the work we were waiting on has finished, we could set it to a ready state across threads to signal that the work we were waiting on has finished, thus avoiding the current delay-based busy wait.

Does this make sense what I'm asking? I'm also open to other suggestions here. Thanks for your consideration! :)

[zach-klippenstein]

Are you only using your worker to do IO? If so, another approach could be for whatever IO library you're using to provide a non-blocking API - e.g. kotlinx.io has this API, but as far as I can tell does not actually implement it on Native. Okio doesn't have a non-blocking API (yet, see square/okio#501), but there's a proposal (square/okio#531). On native, these libraries could either use a worker trick like you're doing, or use non-blocking platform APIs - but either way would allow consumers to run completely on the main event loop and not need to be concerned with threading at all.

[benasher44]

The primary use case is emulating withContext(Dispatchers.IO) on native, but it's more generally about being able to push off a long-running task to another thread. So for me? Yes. For others? Not sure! :)

run completely on the main event loop and not need to be concerned with threading at all.

IMO, it's important to note that Dispatchers.Main isn't implemented on native in the way one might think. See this note in the comments for the main dispatcher:

* Depending on platform and classpath it can be mapped to different dispatchers:
     * - On JS and Native it is equivalent to the [Default] dispatcher.

This means that you're not going to have a usable event loop unless you call runBlocking. If you want a "real main dispatcher" like you have on Android/JVM, there's a suggestion on how to implement one in #462. But, that means your application needs to be aware of it's existence (i.e. it'll be separate from Dispatchers.Main), and someone (your application or okio) will have to provide it.

Anyway, just some notes about what it means for main dispatcher on native.