Replies: 7 comments 18 replies
-
|
Wow that's an awesome summary with really great diagrams. My main focus right now is Rack 3, after that I'll move on to Rails and we will solve any of the issues that still exist there. The recent work with rails/rails#43596 will help set the stage for connection per fiber. Right now with AR, the worst case is you will have one connection per thread which means every incoming request (fiber) on the thread would contend with each other (over a single AR connection). While it works, if you are database bound, it is less than ideal. That's based on last time I checked the state of Rails, but it has changed, I don't know if #43596 made it into Rails 7 - and I haven't evaluated the impact of it yet. There is also work going on in the database backends, e.g. The MySQL gem is a little harder because the actual C interface is insufficient, but |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for taking the time to explain and illustrate the bottleneck currently in ActiveRecord. As I mentioned in my original post, we run a Falcon (proxy server) for communicating with external I/O and Rails on Puma for internal (ActiveRecord) I/O. It's clear that if I run ActiveRecord on Falcon, I would gain anything. However, I'd like to understand what happens when I run both internal (ActiveRecord) and external I/O on a single Rails server? In other words a single Rails request first communicates with an external API and handles an ActiveRecord query (or the reverse).
Will either setup use a Fiber/Thread/Fiber/Thread pattern, or is the second setup all Thread? In other words, while the ActiveRecord query still consumes 1 thread, when the request moves on to the external request does it use an async fiber and you see the same performance as if the 3rd party request was on a proxy server? Or, does ActiveRecord somehow tie up the entire end-to-end request as a thread and does not use any fibers? |
Beta Was this translation helpful? Give feedback.
-
|
Annoying how github discussions hides additional comments of an answer behind a "show more" link. |
Beta Was this translation helpful? Give feedback.
-
|
Agreed. Not sure why they did that. Same with long comment threads, too, imo :/ I think you can wait for Samuel to chime in, but also might be worth testing this yourself. I think you could use the benchmark stuff mentioned above (https://github.com/socketry/db/tree/main/benchmark/compare) -- I suppose you'd just add some non-database work the happens on either side of the database work to see if that work scales up linearly. Looking at the benchmark and trying to suss it out, my guess for the "does ActiveRecord somehow tie up the entire end-to-end request as a thread and does not use any fibers" question is "no" -- rather, ActiveRecord will use a single connection, so any single ActiveRecord query will block any other ActiveRecord query as well, but not necessarily all other work. All ActiveRecord work would be waiting on a single connection. So, if you have super fast queries, oh well, but if you have some slow queries it could be bad, because only one ActiveRecord query could happen at once, so the other fibers will be held up waiting for the single connection to be freed up. So, I suppose it's a bit more complicated than a simple "no" but the practical impact would depend on your specific app. In my case, I'm thinking of either using the db gem until the ActiveRecord connection pool stuff is sorted out, or using caching around everything. That being said, to complicate things further, I'm using Rails.cache with a Redis backend, and I'm really not sure if that suffers from the same issue as ActiveRecord here, where everything would be sharing a single connection. My Redis work is so quick, though, if I am encountering a bottleneck it hasn't been severe enough to be noticed. Similar to the db gem, perhaps I should be using https://github.com/socketry/async-redis for now, or I should figure out how to run the benchmark with a Redis sleep or w/e to prove if we have the same issue or not. I confess things are working well enough for me (so far) that I haven't dug into this too deeply (yet). |
Beta Was this translation helpful? Give feedback.
-
|
@trevorturk is largely correct, the contention is only with the single ActiveRecord connection and so you'd be fine to use Rails provided you don't contend on the connection. Option 2 should be fine, and will only get better as we wrap up some of the downstream implementation details. IF you use Ruby 3.1+ (and disable zeitwerk/autoload because it's buggy in 3.1.x, fixed in 3.2), you can wrap your database queries in |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the I wanted to note, I think (?) it's fair to handle the "disable zeitwerk/autoload" gotcha by using Zeitwerk's global eager load: https://github.com/fxn/zeitwerk#global-eager-load -- it also looks like there's a "Rails way" to do this as well, documented here: https://guides.rubyonrails.org/classic_to_zeitwerk_howto.html#check-zeitwerk-compliance-in-the-test-suite -- this is written as though you'd only use it for testing purposes, but if autoload is buggy in 3.1.x, doing the global eager load either way should avoid the bugs. I'm doing this in an initializer in my Falcon/Rails 3.1.x app now on @ioquatix's advice. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for this great discussion. I wanted to chime in to say that I'm very happily using Rails + Falcon in production, but without a database. IIRC @ioquatix had suggested considering https://github.com/socketry/db but I understand that some work has gone into making ActiveRecord work better when running with Falcon, and sounds like more is coming, which is great news. I wanted to point out also some helpful discussion here: rails/rails#44219 (comment) I'm hoping @ioquatix can clarify the "entire request" vs "part of the request" question, but my (naive) understanding is that ActiveRecord would be operating a single connection, as opposed to a connection pool. If that's the case, it's not really a question of "consuming a fiber" vs not using a connection pool. In my case, I'm just about to add a database to my app, but I was thinking of using |
Beta Was this translation helpful? Give feedback.
-
|
Thank you. The discussion at rails/rails#44219 (comment) was helpful in understanding all the Rails components tied to threads and how big the task is to convert Rails to be fiber friendly. |
Beta Was this translation helpful? Give feedback.
-
|
@tleish Can you provide some more details about the asynchronous Ruby proxy server?
Any code samples you can share will be of great help. |
Beta Was this translation helpful? Give feedback.
-
|
Q: Is it using any framework? Q: Does it proxy all routes or selected routes? Q: Are the calls from proxy server to Rails app through HTTP? If so, how is auth handled? Both servers can access Redis (sessions), but only the Rails server accesses the DB. Rails server never sends requests to Falcon or 3rd Party server to maintain low latency.
|
Beta Was this translation helpful? Give feedback.
-
|
@tleish This has been of great help. Thank you. |
Beta Was this translation helpful? Give feedback.
-
|
👋 I would like to understand if there is any update. |
Beta Was this translation helpful? Give feedback.
-
|
Soooo I just replaced I'm stoked! But also I have.... questions:
|
Beta Was this translation helpful? Give feedback.
-
|
Great news! I found the rails example and copied the I'm going to forge ahead. I'm already using |
Beta Was this translation helpful? Give feedback.
-
|
ActiveRecord won't be fully compatible until Rails 7.1 is released. However, IIRC, it's not totally broken, it's just essentially limited to "1 connection per thread", which since Falcon only uses a single thread, is a bit limiting. You could try using rails head if you want to forge ahead with something. I believe it all works quite okay. |
Beta Was this translation helpful? Give feedback.
-
|
Yes I’m using Rails 7.0 and it works well, with the connection limitation caveat mentioned. Rails 7.1 isolation_state config should help, but if you search the Rails issues you’ll come across some other relatively minor inefficiencies that may want to be addressed eventually. |
Beta Was this translation helpful? Give feedback.
-
What's the upshot of this? Is it essentially that only one request can be served concurrently? Does falcon offer forking or threading to mix concurrency models at all? |
Beta Was this translation helpful? Give feedback.
-
|
I run one Falcon worker per Heroku Dyno so it means one connection is shared between all requests on the same Dyno. Since I’m using the pg gem which is fiber-aware it basically seems fine, but I’m sure it’ll be optimized better when switch to isolation_level :fiber in Rails 7.1. I’d note though that I’m a bit nervous about swapping things in production since I believe it’ll open a connection per fiber, which can be a lot of connections. I believe Samuel’s advice is to create your own pool using an Async Semaphore with a limit, but I haven’t gotten that far yet. When we get there let’s all agree to pitch in with docs and best practices please! :) |
Beta Was this translation helpful? Give feedback.
-
|
FWIW I was running falcon on fly.io and I noticed 2 things:
I can't help but wonder if a forking + evented model like rainbows would be worth trying. not sure what to make of the memory footprint |
Beta Was this translation helpful? Give feedback.
-
|
Interesting! FWIW I'm using Falcon on Heroku and it's been great, much more efficient than Puma, because I'm waiting on a lot of upstream API responses, Fibers are a lighter solution than threads, allowing me to handle more concurrent requests with fewer dynos. Re: memory usage, did you tweak the Re: 429s, I'm not sure if that would be Falcon -- wonder if it could be fly.io limiting the number of concurrent requests per dyno (or whatever their "dyno" equivalent thing is called) |
Beta Was this translation helpful? Give feedback.
-
|
@sdhull how many processors are available on Fly.io? At a guess, you could be comparing multi-process falcon with multi-thread puma. If you didn't use Not sure about 429s, never seen that before, but I've definitely seen issues with proxies in front of Falcon being unable to handle the load correctly. I don't think Falcon itself can return 429s :) It would be nice to understand a bit more about the environment. |
Beta Was this translation helpful? Give feedback.
-
|
@sdhull Hi, I'm interested to know if you ended up resolving your memory issues with Falcon? Did you put it back in or did you go back to sticking with Puma? |
Beta Was this translation helpful? Give feedback.
-
QUESTIONS:
DETAILS:
A recent article on ideas for improving the Ruby JIT compiler sparked a conversation between devs in Hackernews discussing different approaches to making a Ruby on Rails application run faster. If a Rails app is CPU bound, then improving the JIT might help. However, if the Rails app is primarily I/O bound, then this approach might not make much of a difference.
A former Shopify developer wrote:
We avoid I/O delay by deploying two servers. A Rails server and an asynchronous Ruby proxy server (falcon).
Standard Request
An example of a standard internal request looks like this.
sequenceDiagram Browser->>Rails: Submit Form activate Rails Rails->>MySQL: Query User Info activate MySQL MySQL-->>Rails: User Info deactivate MySQL Rails->>MySQL: Update User Info activate MySQL MySQL-->>Rails: Update Status deactivate MySQL deactivate Rails Rails-->>Browser: Response3rd-Party Request
If a user interaction requires I/O delay from a 3rd-party API, the sequence looks like this.
sequenceDiagram Browser->>Falcon: Submit Form activate Falcon Falcon->>Rails: Get User Info (validate user) activate Rails Rails->>MySQL: Query User Info activate MySQL MySQL-->>Rails: User Info deactivate MySQL Rails-->>Falcon: Return User Info deactivate Rails Falcon->>3rd Party API: Submit Form w/ User Info activate 3rd Party API 3rd Party API-->>Falcon: API Response deactivate 3rd Party API Falcon->>Rails: Update Record activate Rails Rails->>MySQL: Update User Info activate MySQL MySQL-->>Rails: Update Status deactivate MySQL Rails-->>Falcon: Response deactivate Rails Falcon-->>Browser: Response deactivate FalconUsing Falcon as an asynchronous proxy prevents our Rails server resources from getting tied up while waiting for 3rd party I/O. For us, this has scaled very well. As you can see, the main downside is the additional complication. I'd love to combine Falcon/Rails into a single server to simplify this. However, it's unclear to me the current status/roadmap for making this work, especially for a legacy app.
In researching this I've found the following:
In response to the above article, @ioquatix responded:
What does "hugely limited" mean? Does it just mean that ActiveRecord would be synchronous, while a 3rd party request could be asynchronous and release the fiber while waiting for response? This is how our stack works today, but this would just mean we could consolidate to a single server.
sequenceDiagram Browser->>Falcon/Rails: Submit Form activate Falcon/Rails Falcon/Rails->>MySQL: Query User Info <br>(use fiber resource) activate MySQL MySQL-->>Falcon/Rails: User Info deactivate MySQL Falcon/Rails->>3rd Party API: Submit Form w/ User Info <br>(wait, release fiber resource) activate 3rd Party API 3rd Party API-->>Falcon/Rails: API Response deactivate 3rd Party API Falcon/Rails->>MySQL: Update User Info <br>(use fiber resource) activate MySQL MySQL-->>Falcon/Rails: Update Status deactivate MySQL Falcon/Rails-->>Browser: Response deactivate Falcon/RailsI also found Falcon Guides > Rails Integration which does not mention any gotchas or limitations of ActiveRecord.
I know there's been some recent changes to Ruby 3 and fibers, is it advised to wait for these to be deployed?
In my research I also remember reading a comment by @ioquatix suggesting Falcon as experimental and not to be used for production (which we currently are).
Beta Was this translation helpful? Give feedback.
All reactions