Add virtual-threads support in Quarkus #24942

anavarr · 2022-04-14T15:39:46Z

when an endpoint has the annotation @RunOnVirtualThread its blockingHandler will use a special Excutor spawing a new virtual threads for every request
added isRunOnVirtualThread method to check that @RunOnVirtualThread is possible (must be on a @Blocking endpoint, the user is warned if the jdk they use to compile is not loom-compliant)
an endpoint with the @Transactional annotation will override the @RunOnVirtualThread annotation -> arbitrary choice of caution (@Transactional might use thread locals for instance ?)
the loom-related pieces are called by reflective calls to avoid dependencies on java 19 (if the runtime is not compliant, the endpoint falls back to traditional workers)

Add a new extension : netty-loom-adaptor that performs bytecode manipulation on netty's PooledByteBufAllocator to use a concurrentHashMap instead of threadLocals

when creating a DirectBuffer, we check if the current Thread is virtual or not
if it is, we get its carrier to check if the threadCaches hashMap has the carrier's name as a key
if so, we return the PoolThreadCache associated to the carrier
else, we create a new PoolThreadCache (wip) and return it
after that, we will use this cache instead of the PoolThreadLocalCache
added static fields to cache isVirtual() method (instead of using reflection to get it for every request)
added static fields to cache currentCarrierThread method (same goal as for isVirtualMethod field)
tested quarkus-loom with the application --> from 20 000 reqs during benchmark to ~ 160 000 requests during benchmark (9.5% quarkus perf to ~71% quarkus perf)
netty-loom-adaptor extension will do its magic only if @RunOnVirtualThread annotations are present in the combinedIndexBuildItem, else it won't do anything

When using quarkus dev services with loom code, the user must specify -DopenLoom. If they don't the reflective operations will throw exceptions

geoand

Great stuff!

I've added some comments

devtools/maven/src/main/java/io/quarkus/maven/DevMojo.java

...daptor/deployment/src/main/java/io/quarkus/netty/loom/adaptor/NettyLoomAdaptorProcessor.java

geoand · 2022-05-20T08:27:23Z

...daptor/deployment/src/main/java/io/quarkus/netty/loom/adaptor/NettyLoomAdaptorProcessor.java

+import io.quarkus.netty.deployment.MinNettyAllocatorMaxOrderBuildItem;
+import io.smallrye.common.annotation.RunOnVirtualThread;
+
+public class NettyLoomAdaptorProcessor {


I think we'll need some comments on what this does, otherwise it will be hard to maintain this ASM :)

thx so much for your comments @geoand
I'm trying to comment this mess, would it make sense to display the actual Netty method and the desired one in order for the reader to see where we're going ?

It would definitely make sense to display the actual result of the transformation in addition to why we need it.

hey @geoand, I added some comments explaining what we do
If you find it satisfying I'll just squash everything and rebase to make it merge-ready

Great, thanks

...time/src/main/java/io/quarkus/resteasy/reactive/server/runtime/ResteasyReactiveRecorder.java

...ain/java/org/jboss/resteasy/reactive/server/processor/ResteasyReactiveDeploymentManager.java

...c/main/java/org/jboss/resteasy/reactive/server/handlers/VirtualThreadNonBlockingHandler.java

nipafx · 2022-05-22T13:27:34Z

Short note: I recommend not to use "Loom" for class or option names. While very prominent now, the name is just an artifact of the OpenJDK development process and doesn't exist "in code". Also, it will soon just be an obscure historic details (c.f. "Coin" or "Jigsaw"). I'd propose useVirtualThreads over openLoom.

cescoffier · 2022-05-22T17:22:22Z

@nipafx Thanks! Yes, Definitely. It's actually not about using loom, it's to open some modules we need (JPMS stuff). The name was rushed and does not convey what it does (and fortunately, we will make this obscure option goes away.

@anavarr what about open-xxx-package (I can't remember the one we need, so replace xxx). Also, can't we detect and pass the option without having the user explicitly doing it?

anavarr · 2022-05-23T13:19:50Z

Quick disclaimer and questions about the "openLoom" flag

It is used in reflection : we open the java.lang module to be able to access the VirtualThreadBuilder class and use a custom executor (was possible before, is impossible now) to dispatch virtual threads on carrier threads

Our hypothesis was that the current ForkJoinPool-like executor was going to spawn too many carrier threads and increase the memory footprint of the application.
We hence replaced it with a custom executor: a fixedThreadPoolExecutor with 2*CPU_cores carrier threads.

Recent benchmarks we did indicate that this choice didn't affect the memory footprint of the application but reduced its scalability

We were thinking about switching back to the "default" executor but I am afraid the API will change, shall we keep the FixedThreadPool executor for now or switch to NewVirtualThreadPerTaskExecutor ?

nipafx · 2022-05-23T14:39:57Z

I'm neither a virtual thread expert nor a Quarkus expert, but I'd recommend not to hack into these systems on the unreported hypothesis that there may be a memory issue. Particularly not since some of these things can be controlled by the user with command line flags. Form the Thread Javadoc:

In the JDK Reference Implementation, the virtual thread scheduler may be configured with the following system properties:

jdk.virtualThreadScheduler.parallelism: The number of platform threads available for scheduling virtual threads. It defaults to the number of available processors.

jdk.virtualThreadScheduler.maxPoolSize: The maximum number of platform threads available to the scheduler. It defaults to 256.

I hope we can start the journey towards virtual threads with small steps, each one allowing time for feedback to come in, instead of taking several steps at once.

anavarr · 2022-05-23T14:52:31Z

@nipafx you're right, I guess it was useful hack to explore different leads but our experiments didn't give results good enough for us to continue down this path ?

geoand · 2022-05-24T09:01:49Z

This is good to go from my perspective.

If everything is also good for @cescoffier, let's squash and merge it

cescoffier · 2022-05-24T09:04:34Z

All good for me!
It's a first step, and for sure experimental.

@Blocking

- when an endpoint has the annotation @RunOnVirtualThread its blockingHandler will use a special Excutor spawing a new virtual threads for every request - added isRunOnVirtualThread method to check that @RunOnVirtualThread is possible (must be on a @Blocking endpoint, the user is warned if the jdk they use to compile is not loom-compliant) - an endpoint with the @transactional annotation will override the @RunOnVirtualThread annotation -> arbitrary choice of caution (@transactional might use thread locals for instance ?) - the loom-related pieces are called by reflective calls to avoid dependencies on java 19 (if the runtime is not compliant, the endpoint falls back to traditional workers) Add a new extension : netty-loom-adaptor that performs bytecode manipulation on netty's PooledByteBufAllocator to use a concurrentHashMap instead of threadLocals - when creating a DirectBuffer, we check if the current Thread is virtual or not - if it is, we get its carrier to check if the threadCaches hashMap has the carrier's name as a key - if so, we return the PoolThreadCache associated to the carrier - else, we create a new PoolThreadCache (wip) and return it - after that, we will use this cache instead of the PoolThreadLocalCache - added static fields to cache isVirtual() method (instead of using reflection to get it for every request) - added static fields to cache currentCarrierThread method (same goal as for isVirtualMethod field) - tested quarkus-loom with the application --> from 20 000 reqs during benchmark to ~ 160 000 requests during benchmark (9.5% quarkus perf to ~71% quarkus perf) - netty-loom-adaptor extension will do its magic only if @RunOnVirtualThread annotations are present in the combinedIndexBuildItem, else it won't do anything When using quarkus dev services with loom code, the user must specify -Dopen-lang-package. If they don't the reflective operations will throw exceptions

anavarr · 2022-05-24T09:25:24Z

squashed and rebased, checked diff with main, should be good to go

geoand

🎉

jponge · 2022-05-24T09:30:02Z

Nice to see this coming disciple @anavarr 👍

tigerinus · 2022-06-15T15:13:30Z

any official doc regarding how to use this in Quarkus?

geoand · 2022-06-15T15:17:40Z

Not yet, it's as it's very experimental and subject to change

JackyAnn · 2022-09-26T03:39:08Z

is there a global config run all endpoint on virtual thread or platform thread？

cescoffier · 2022-09-26T05:42:33Z

@JackyAnn No, and the reason is that there are too many possibilities of pinning or monopolizing the carrier threads, which would be terrible. Thus, switching to a virtual thread is something you need to be aware of for now. Later, once Loom improves, and the Java ecosystem adapts, it will become the default. But for now, the risk is too high.

apatrida · 2023-01-15T21:15:47Z

This could also allow dispatching Kotlin coroutines onto LOOM VirtualThreads, by having that annotation cooperate with suspend functions

quarkus-bot bot added area/devtools Issues/PR related to maven, gradle, platform and cli tooling/plugins area/maven area/resteasy-reactive labels Apr 14, 2022

cescoffier marked this pull request as ready for review May 20, 2022 08:18

cescoffier requested a review from geoand May 20, 2022 08:19

cescoffier added the release/noteworthy-feature label May 20, 2022

geoand reviewed May 20, 2022

View reviewed changes

This comment has been minimized.

Sign in to view

anavarr force-pushed the quarkus-loom branch from 1b8394e to e9f7b16 Compare May 23, 2022 09:47

anavarr force-pushed the quarkus-loom branch from 035ea51 to 0639068 Compare May 24, 2022 09:20

anavarr force-pushed the quarkus-loom branch from 671b2f3 to 8da31fd Compare May 24, 2022 09:24

geoand approved these changes May 24, 2022

View reviewed changes

geoand added the triage/waiting-for-ci Ready to merge when CI successfully finishes label May 24, 2022

cescoffier merged commit 925df4a into quarkusio:main May 24, 2022

quarkus-bot bot added this to the 2.10 - main milestone May 24, 2022

quarkus-bot bot removed the triage/waiting-for-ci Ready to merge when CI successfully finishes label May 24, 2022

anavarr deleted the quarkus-loom branch February 13, 2023 17:25

anavarr restored the quarkus-loom branch February 13, 2023 17:25

anavarr deleted the quarkus-loom branch February 13, 2023 17:25

imperatorx mentioned this pull request Jun 26, 2023

@Transactional should not override @RunOnVirtualThread in reasteasy reactive #34319

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add virtual-threads support in Quarkus #24942

Add virtual-threads support in Quarkus #24942

anavarr commented Apr 14, 2022 •

edited by geoand

geoand left a comment

geoand May 20, 2022

anavarr May 23, 2022

geoand May 23, 2022

anavarr May 24, 2022

geoand May 24, 2022

This comment has been minimized.

nipafx commented May 22, 2022

cescoffier commented May 22, 2022

anavarr commented May 23, 2022 •

edited

nipafx commented May 23, 2022

anavarr commented May 23, 2022

geoand commented May 24, 2022

cescoffier commented May 24, 2022

anavarr commented May 24, 2022

geoand left a comment

jponge commented May 24, 2022

tigerinus commented Jun 15, 2022

geoand commented Jun 15, 2022

JackyAnn commented Sep 26, 2022

cescoffier commented Sep 26, 2022

apatrida commented Jan 15, 2023 •

edited

Add virtual-threads support in Quarkus #24942

Add virtual-threads support in Quarkus #24942

Conversation

anavarr commented Apr 14, 2022 • edited by geoand

geoand left a comment

Choose a reason for hiding this comment

geoand May 20, 2022

Choose a reason for hiding this comment

anavarr May 23, 2022

Choose a reason for hiding this comment

geoand May 23, 2022

Choose a reason for hiding this comment

anavarr May 24, 2022

Choose a reason for hiding this comment

geoand May 24, 2022

Choose a reason for hiding this comment

This comment has been minimized.

nipafx commented May 22, 2022

cescoffier commented May 22, 2022

anavarr commented May 23, 2022 • edited

nipafx commented May 23, 2022

anavarr commented May 23, 2022

geoand commented May 24, 2022

cescoffier commented May 24, 2022

anavarr commented May 24, 2022

geoand left a comment

Choose a reason for hiding this comment

jponge commented May 24, 2022

tigerinus commented Jun 15, 2022

geoand commented Jun 15, 2022

JackyAnn commented Sep 26, 2022

cescoffier commented Sep 26, 2022

apatrida commented Jan 15, 2023 • edited

anavarr commented Apr 14, 2022 •

edited by geoand

anavarr commented May 23, 2022 •

edited

apatrida commented Jan 15, 2023 •

edited