Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Connection pool issue when using Blazer with a read replica and Rails automatic connection switching #469

Open
weilandia opened this issue Mar 19, 2024 · 3 comments · May be fixed by #470
Open

Connection pool issue when using Blazer with a read replica and Rails automatic connection switching #469

weilandia opened this issue Mar 19, 2024 · 3 comments · May be fixed by #470

Comments

@weilandia
Copy link

weilandia commented Mar 19, 2024 •
edited

After moving to a multidb setup in our app with automatic role switching, we're running into an issue where /queries/tables?data_source=main intermittently returns a 500 with No connection pool for 'Blazer::Connection::Adapter96700' found..

NOTE: We were also getting an ActiveRecord::ConnectionNotEstablished error intermittently when running queries after the switch, but this seems to have been fixed by increasing our db pool size.

BACKTRACE:

lib/active_record/connection_adapters/abstract/connection_handler.rb:243 retrieve_connection  
lib/active_record/connection_handling.rb:287 retrieve_connection  
lib/active_record/connection_handling.rb:254 connection  
lib/active_record/sanitization.rb:199 replace_bind_variables  
lib/active_record/sanitization.rb:168 sanitize_sql_array  
lib/blazer/adapters/sql_adapter.rb:289 add_schemas  
lib/blazer/adapters/sql_adapter.rb:54 tables  
/usr/local/lib/ruby/3.3.0/forwardable.rb:240 tables  
app/controllers/blazer/queries_controller.rb:206 tables  
lib/action_controller/metal/basic_implicit_render.rb:6 send_action  
lib/abstract_controller/base.rb:224 process_action  
lib/action_controller/metal/rendering.rb:165 process_action  
lib/abstract_controller/callbacks.rb:259 block in process_action  
lib/active_support/callbacks.rb:110 run_callbacks  
lib/abstract_controller/callbacks.rb:258 process_action  
lib/action_controller/metal/rescue.rb:25 process_action  
lib/action_controller/metal/instrumentation.rb:74 block in process_action  
lib/appsignal/hooks/active_support_notifications.rb:19 block in instrument  
lib/active_support/notifications/instrumenter.rb:58 instrument  
lib/appsignal/hooks/active_support_notifications.rb:18 instrument  
lib/action_controller/metal/instrumentation.rb:73 process_action  
lib/action_controller/metal/params_wrapper.rb:261 process_action  
lib/searchkick/controller_runtime.rb:15 process_action  
lib/active_record/railties/controller_runtime.rb:32 process_action  
lib/abstract_controller/base.rb:160 process  
lib/action_view/rendering.rb:40 process  
lib/action_controller/metal.rb:227 dispatch  
lib/action_controller/metal.rb:309 dispatch  
lib/action_dispatch/routing/route_set.rb:49 dispatch  
lib/action_dispatch/routing/route_set.rb:32 serve  
lib/action_dispatch/journey/router.rb:51 block in serve  
lib/action_dispatch/journey/router.rb:131 block in find_routes  
lib/action_dispatch/journey/router.rb:124 each  
lib/action_dispatch/journey/router.rb:124 find_routes  
lib/action_dispatch/journey/router.rb:32 serve  
lib/action_dispatch/routing/route_set.rb:882 call  
lib/rails/engine.rb:536 call  
lib/rails/railtie.rb:226 public_send  
lib/rails/railtie.rb:226 method_missing  
lib/action_dispatch/routing/mapper.rb:22 block in class:Constraints  
lib/action_dispatch/routing/mapper.rb:51 serve  
lib/action_dispatch/journey/router.rb:51 block in serve  
lib/action_dispatch/journey/router.rb:131 block in find_routes  
lib/action_dispatch/journey/router.rb:124 each  
lib/action_dispatch/journey/router.rb:124 find_routes  
lib/action_dispatch/journey/router.rb:32 serve  
lib/action_dispatch/routing/route_set.rb:882 call  
lib/rack/attack.rb:127 call  
lib/prerender_rails.rb:124 call  
lib/active_record/middleware/database_selector.rb:67 block in call  
lib/active_support/notifications/instrumenter.rb:58 instrument  
lib/active_record/middleware/database_selector/resolver.rb:58 block in read_from_primary  
lib/active_record/connection_handling.rb:361 with_role_and_shard  
lib/active_record/connection_handling.rb:147 connected_to  
lib/active_record/middleware/database_selector/resolver.rb:57 read_from_primary  
lib/active_record/middleware/database_selector/resolver.rb:37 read  
lib/active_record/middleware/database_selector.rb:77 select_database  
lib/active_record/middleware/database_selector.rb:66 call
...
@weilandia weilandia linked a pull request Mar 19, 2024 that will close this issue
Open
@bdschultzAU
Copy link

bdschultzAU commented May 7, 2024 •
edited

I ran into this exact problem after updating to updating to use a primary / primary_replica. We're on the default pool size right now. How many did you update to? I'm already running 30

@weilandia
Copy link
Author

weilandia commented May 9, 2024 •
edited

@bdschultzAU it actually ended up not working. We ended up moving Blazer back to the primary for now.

@bdschultzAU
Copy link

@bdschultzAU it actually ended up not working. We ended up moving Blazer back to the primary for now.

thanks for the response @weilandia . I came to that conclusion too. I monkey patched it to force blazer to use the writer node for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

POC to close #469 - Establish connection to the correct pool when using a replica weilandia/blazer
2 participants
@weilandia @bdschultzAU