Please support polling from a single non-Netty application thread?

[gordonbuntting]

What I would like to do is build our low-latency financial trading application with a single busy-spinning thread. It needs to perform various tasks as work is available including polling on network channels. Other middleware such as Aeron and Chronicle Queue support this. However we also need to consume and publish data to other protcols like TCP, HTTP and WebSocket at which Netty excels.

With a single-threaded application, there's no need to for thread-safe datastructures, lock-free techniques or thread-handoff which radically simplifies development and supports low-latency applications. Low Latency means aiming for response times in single digit microseconds @ 99%+.

Netty promises a Highly customizable thread model. However this seems to have an ironclad assumption that threads are owned by Netty. So even if I provided an Executor implementation that is serviced by the single application thread, a quick look at all the Netty implementations shows that SingleThreadEventExecutor#run is expected to block indefinitely until shutdown.

Would you consider a poll method on these Executors that could be invoked from outside? And implementations would use either selectNow or have an upper limit? Of course it might not be that simple as any logic that currently blocks or has state in the run() method would need to be refactored.

Perhaps this would also be a better avenue to explore than Virtual Threads - which don't seem to promise much control of which platform thread tasks are run on AFAIK.

Any comments greatly appreciated.

[franz1981]

Currently you can tweak your own SelectStrategy to always busy spin: need to check if it works and is supported by all transports.

[gordonbuntting]

Thanks @franz1981 . It requires an additional method in the EventLoop api perhaps called runOnce() to do the equivalent of run(). The idea is that this method would be continually invoked by a busy-spinning application thread running a duty cycle on the order of every microsecond. It would have been nice to test this out in one of my projects by creating subclasses but the Netty classes are severely locked down with final and private. I guess I could try forking the repo.

At the simplest the runOnce() method would always 'select now' though it might support the SelectStrategy to allow more refined behaviour - for example busy waiting on the selector for a limited time.

In some ways it would be nice to avoid the EventLoop machinery altogether since that is precisely what I'm trying to avoid as I've got my own application event loop. However a lot of the transport logic is embedded inside the various SingleThreadEventLoop implementations (NioEventLoop etc.) and I would not want to duplicate this.

Is this an idea that the community would entertain?

[normanmaurer]

Let me think a bit about this. But this might be possible by exposing it via SingleThreadedEventLoop.

[gordonbuntting]

Thanks @normanmaurer. Looking at Netty5, this would be the equivalent of exposing SingleThreadEventLoop.runIo() method to be run by the application thread. It's a lot neater than Netty4 as it' avoids duplicating the loop in each transport. This would then tie in with @franz1981 's suggestion that only the SelectStrategy would need to be changed to avoid blocking.

In Netty4, I had in mind to basically do the same thing. That is, refactor the 'meat' inside the outer for loop of NioEventLoop#run and other implementations into a separate method. Could even name this runIo().

Would you consider a PR for this in Netty4 if I raise it as that's what I'm most familiaer with and using at the moment? And then Netty5 in due course. Will look into the process.