Thousand Island vs Ranch results?

[Maria-12648430]

Out of interest... at the top of the README, it says that Ranch vs Thousand Island is also benchmarked. But results for this are neither in the PDF nor in the Results paragraph further down in the README.
I would be interested to see how they perform in comparison to each other 😃

[mtrudel]

I've dug around and the only source material I still have for the benchmarking is at https://github.com/mtrudel/talks/blob/main/2021-10-ElixirConf-bandit-on-the-loose/Benchmarks.numbers - and it only includes the results for Bandit and Cowboy.

If I recall correctly, including the fake HTTP servers that were used to test the raw performance of ranch and Thousand Island was too much data for me to effectively deal with / present, and just complicated the core narrative ('Bandit is up to 5x faster!'). I do recall that the the numbers for both were:

1. Surprisingly low, lower than the numbers for either Bandit or Cowboy (the fake servers don't support keepalive, so it was a new TCP connection for every HTTP request)
2. Almost identical to one another. IIRC Ranch won out at the low end and Thousand Island at the high end of concurrency, but it was within a few percent in all cases.

Once the work in phoenixframework/phoenix#5003 lands, I'm planning on doing a large performance push on both Bandit and Thousand Island as part of the 0.7.0 release train, covering:

• Adding comprehensive telemetry span support to the entire stack
• Adding support for the new socket API in OTP 24+
• Looking at process models in Thousand Island whereby acceptors handle connections directly (this would likely trade concurrency for responsiveness, which may be a useful tradeoff for some)
• Profiling common HTTP code paths. I've come across a few places where I think we can squeeze a bit more performance out of things (lots of contributions have been coming in to Bandit lately in this vein as well)
• Adding reproducible micro benchmarks to the Bandit CI pipeline to identify performance regressions, including on WebSocket connections
• Re-running the benchmarks herein to see how we're progressing

I'll be sure to publish a full set of Ranch vs. Thousand Island stats as part of that workup, even if the numbers are near identical.

Thanks for the note!