[Feature Request] Ruby 3.0 Fiber Scheduler support

[jasl]

Is your feature request related to a problem? Please describe.

Ruby 3.0 will release very soon, the new feature Fiber Scheduler may significant improve Ruby server app.

My friend working on the first Fiber Scheduler implementation called EVT https://github.com/dsh0416/evt

The benchmark shows impressive.

Although there's still lacking DB drivers, Redis, etc. to adapt the fiber scheduler so Rack-based app can't get such performance, but I think it is time now.

Describe the solution you'd like

Let Puma support Ruby 3.0 Fiber Scheduler

[leebo]

👍🏻👍🏻👍🏻

[nateberkopec]

For the Reactor, maybe. I want to spike a Ractor-based implementation and could do a Fiber scheduler based one as well.

I don't think I see a big upside for using a Fiber scheduler based worker pool. It's a) getting too far away from the core concept of Puma b) IMO presents very limited latency overhead improvement opportunity, less than a millisecond of latency per request c) I don't think would result in less GVL/shared-resource contention than our current approach.

[jasl]

For the Reactor, maybe. I want to spike a Ractor-based implementation and could do a Fiber scheduler based one as well.

I don't think I see a big upside for using a Fiber scheduler based worker pool. It's a) getting too far away from the core concept of Puma b) IMO presents very limited latency overhead improvement opportunity, less than a millisecond of latency per request c) I don't think would result in less GVL/shared-resource contention than our current approach.

Thanks for your reply!

I agree with you,
In addition, maybe Puma can remove usage of nio4r in Reactor, and I saw Samuel said at #1728 (comment)

Falcon support Fiber scheduler now (actually async gem supports), but lacking support for 3rd scheduler,
Samuel reply me at socketry/async#82 (comment)

[nateberkopec]

I would definitely love getting rid of the nio4r dependency, though we will be stuck with that for quite some time for Rubies less than 3.0.

[jasl]

I would definitely love getting rid of the nio4r dependency, though we will be stuck with that for quite some time for Rubies less than 3.0.

nio4r only use in reactor?
maybe Puma can provide a legacy reactor and a fiber-based reactor? then can configure by Ruby version?

[wjordan]

I want to spike a Ractor-based implementation and could do a Fiber scheduler based one as well.

On this, see commit ce53007 from #2279 (comment) which was a proof-of-concept using async to implement a Fiber-based Puma reactor.

This could be useful as a starting point for experimentation along these lines, but I believe async still uses nio4r to manage the non-blocking I/O in its own reactor, so performance/compatibility probably wouldn't change much. Using the Fiber-scheduler interface for async's reactor itself (and the possibility of custom Fiber-scheduler implementations in place of the nio4r-based one in async) is left to future work as mentioned.

[ioquatix]

Great discussion.

[wjordan]

This is now being worked on in #2601, please take a look and try it out!

[jakitliang]

For the Reactor, maybe. I want to spike a Ractor-based implementation and could do a Fiber scheduler based one as well.

I don't think I see a big upside for using a Fiber scheduler based worker pool. It's a) getting too far away from the core concept of Puma b) IMO presents very limited latency overhead improvement opportunity, less than a millisecond of latency per request c) I don't think would result in less GVL/shared-resource contention than our current approach.

Yeah, I read the source code of Ruby well, the scheduler use a GVL to switch between a global Thread pool.

But server aimed gems developers dont like this design. Instead, we server gem developers can do it in c too.

So I think the ioquatix is doing what we do. But only put himself's design into ruby source code.

It is unfair. And it is not tidy in current design.

In other words, we developers can also make our thread pool and manage our Fibers without ioquatix's intervention.

The worst thing is that: ioquatix put his Fiber.schesuler hooking into Thread. Which is likely to impact the origin Fiber implementation and maybe change the behavior that Fiber did before. And this is very awful for me.

Why not remain the origin Fiber code and write another class some like Coroutine and Coroutine.scheduler.

So I think what ioquatix did was just like ONE MAN SHOW.

Although the Fiber.scheduler improve the multi-thread between Fibers performance, but breaks other developers's work.

And for this reason, I keep stay with Ruby v2.5.8 now.

One day the ruby/Coroutine is removed. And the Guild is announced, I would upgrade my ruby version.

I think Guild is the authetic design rather than Fiber.scheduler.

For other programming languages, like JavaScript never do this (Fiber.scheduler) and use multi-workers instead. And Golang design GMP in the first time, so that users never create the thread directly but manage by GMP scheduler.

And I want to say that the way Fiber.scheduler can go on and be nice to me if Ruby remove Thread and Fiber, and let user use async syntax to create a schedulered Thread.

[eregon]

@jakitliang About everything you said is incorrect. Would you mind take your outdated inaccurate rants/trolling somewhere else, or inform yourself better?
Fiber.scheduler is about Fiber scheduling, not thread scheduling. Guild is a very outdated name, it's called Ractor now and they are orthogonal. Node.js does very much do things like Fiber.scheduler, etc.

Which is likely to impact the origin Fiber implementation and maybe change the behavior that Fiber did before.

It doesn't. Inform yourself before accusing.

Insulting Samuel like that and based on your misunderstandings is unacceptable.

[jasl]

For the Reactor, maybe. I want to spike a Ractor-based implementation and could do a Fiber scheduler based one as well.
I don't think I see a big upside for using a Fiber scheduler based worker pool. It's a) getting too far away from the core concept of Puma b) IMO presents very limited latency overhead improvement opportunity, less than a millisecond of latency per request c) I don't think would result in less GVL/shared-resource contention than our current approach.

Yeah, I read the source code of Ruby well, the scheduler use a GVL to switch between a global Thread pool.

But server aimed gems developers dont like this design. Instead, we server gem developers can do it in c too.

So I think the ioquatix is doing what we do. But only put himself's design into ruby source code.

It is unfair. And it is not tidy in current design.

In other words, we developers can also make our thread pool and manage our Fibers without ioquatix's intervention.

The worst thing is that: ioquatix put his Fiber.schesuler hooking into Thread. Which is likely to impact the origin Fiber implementation and maybe change the behavior that Fiber did before. And this is very awful for me.

Why not remain the origin Fiber code and write another class some like Coroutine and Coroutine.scheduler.

So I think what ioquatix did was just like ONE MAN SHOW.

Although the Fiber.scheduler improve the multi-thread between Fibers performance, but breaks other developers's work.

And for this reason, I keep stay with Ruby v2.5.8 now.

One day the ruby/Coroutine is removed. And the Guild is announced, I would upgrade my ruby version.

I think Guild is the authetic design rather than Fiber.scheduler.

For other programming languages, like JavaScript never do this (Fiber.scheduler) and use multi-workers instead. And Golang design GMP in the first time, so that users never create the thread directly but manage by GMP scheduler.

And I want to say that if the way Fiber.scheduler goes on and be nice to me if Ruby remove Thread and Fiber, and let user use async syntax to create a schedulered Thread.

First, Fiber-scheduler is unrelatesd with multi-thread, it's aim for make IO non-blocking, it hooks IO would made now and future Ruby codes take advantage transparently, and it just an interface, without a scheduler implementation IO should work like claasic.

Second, Guild is a more higher level abstraction, they can work together and MAY gain more performance.
Besides, Guild was already in MRI since 3.0, just rebrand as Ractor, your're not well informed.

Third, I guess you should move your reply here: https://bugs.ruby-lang.org/issues/16786, It was a (nearly) one year dissussion, it's not one person show but get approved by many people including Matz, ko1 and Eregon who are key people for MRI and TruffleRuby.

[jakitliang]

@jakitliang About everything you said is incorrect. Would you mind take your outdated inaccurate rants/trolling somewhere else, or inform yourself better? Fiber.scheduler is about Fiber scheduling, not thread scheduling. Guild is a very outdated name, it's called Ractor now and they are orthogonal. Node.js does very much do things like Fiber.scheduler, etc.

Which is likely to impact the origin Fiber implementation and maybe change the behavior that Fiber did before.

It doesn't. Inform yourself before accusing.

Insulting Samuel like that and based on your misunderstandings is unacceptable.

@eregon

The couroutine has been hooked into pthread:

https://github.com/ruby/ruby/blob/789da481fc59ffb1a0d4e4deb0a17ec4fcbd1d58/cont.c#L780

https://github.com/ruby/ruby/blob/master/coroutine/pthread/Context.c

and the GVL is here:

https://github.com/ruby/ruby/blob/26b4aa853f5b770af03d2d50d7045e04b887b45a/coroutine/pthread/Context.h#L29

[eregon]

@jakitliang Samuel introduced his coroutine library to replace the ad-hoc implementation in CRuby. There are zero semantics change from that (Fibers are still changed only explicitly with resume/yield/transfer as before, and there is no Fiber scheduler by default), except now Fibers are faster to create & switch.
(There is a pthread backend for Fibers for uncommon architectures which are not supported directly, that's a small detail and shouldn't matter on commonly-used platforms. It's unrelated to Ruby Thread which is entirely untouched regarding Fiber scheduling)
So I would like you to appreciate such efforts instead of criticizing them without understanding them.

[jakitliang]

@jakitliang Samuel introduced his coroutine library to replace the ad-hoc implementation in CRuby. There are zero semantics change from that (Fibers are still changed only explicitly with resume/yield/transfer as before, and there is no Fiber scheduler by default), except now Fibers are faster to create & switch. (There is a pthread backend for Fibers for uncommon architectures which are not supported directly, that's a small detail and shouldn't matter on commonly-used platforms. It's unrelated to Ruby Thread which is entirely untouched regarding Fiber scheduling) So I would like you to appreciate such efforts instead of criticizing them without understanding them.

Yeah, I appreciate it. I've asked him about these. And I known he works a lot improve the origin implementation of Fiber.

[MSP-Greg]

Puma hat off

ONE MAN SHOW

Many advancements across many fields are the result of a single person or a small team. If the governance of Ruby is the issue, take that up elsewhere.

Puma hat on
Is everyone done? Whatever makes Puma better for the user base is good. Period.

[jakitliang]

@jakitliang Samuel introduced his coroutine library to replace the ad-hoc implementation in CRuby. There are zero semantics change from that (Fibers are still changed only explicitly with resume/yield/transfer as before, and there is no Fiber scheduler by default), except now Fibers are faster to create & switch.
(There is a pthread backend for Fibers for uncommon architectures which are not supported directly, that's a small detail and shouldn't matter on commonly-used platforms. It's unrelated to Ruby Thread which is entirely untouched regarding Fiber scheduling)
So I would like you to appreciate such efforts instead of criticizing them without understanding them.

I think there would be a better way to improve this.

You can see, in c++, std::thread is easy to manually create and join a new thread.

If someone make a std::thread::scheduler as a constraint into the STL of C++. And let user to implement its std::thread::handler to implement the virtual methods (interface or callback) to use it.

And std::thread::scheduler holds an independent thread pool which is not separated.

What do you think about this std::thread::scheduler? And what do c++ devs like @bkaradzic and @nlohmann think?

What about make a golang's GMP into std::thread or std::coroutine. Why not STL do that?

[bkaradzic]

@jakitliang I'm unrelated to this project, don't know why you invited me here, I have no opinion about it...

[nlohmann]

I have no opinion on this. My work projects use some libraries to organize a thread pool, and most side project I have try to avoid threading in the first place.

[jakitliang]

Thanks a lot @bkaradzic and @nlohmann, and sorry 🤣

In this topic we talk about the concurrency and something about module design.