Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Severe Performance Decrease in 1.3.0 when using set_single_row_mode #442

Closed
jeremyevans opened this issue Mar 7, 2022 · 0 comments · Fixed by #445
Closed

Severe Performance Decrease in 1.3.0 when using set_single_row_mode #442

jeremyevans opened this issue Mar 7, 2022 · 0 comments · Fixed by #445

Comments

@jeremyevans
Copy link
Contributor

Reproducer:

gem 'pg', ARGV[0] if ARGV[0]
require 'pg'
conn = PG::Connection.new(user: "sequel_test")
conn.exec("DROP TABLE IF EXISTS accounts;")
conn.exec("CREATE TABLE accounts (id integer default 0);")

conn.exec("INSERT INTO accounts DEFAULT VALUES;")
18.times do
  conn.exec("INSERT INTO accounts SELECT * FROM accounts;")
end

conn.send_query("SELECT * FROM accounts")
conn.set_single_row_mode
conn.block

t = Time.now
nil while conn.get_result
puts "#{Time.now - t} seconds"

Results (ruby is ruby 3.1.1p18, but this same issue occurs for all Ruby versions tested):

$ ruby t2.rb 1.1.4
4.499579633 seconds
$ ruby t2.rb 1.2.3
4.440106149 seconds
$ ruby t2.rb 1.3.0
72.99373833 seconds
$ ruby t2.rb 1.3.1
72.075049592 seconds

Originally discovered by @janko and reported at jeremyevans/sequel#1846, but determined to be a ruby-pg issue and not a Sequel issue. You can probably mitigate this issue by using stream_each, but currently that cannot be used by sequel_pg. I'll send a pull request for the changes needed to make it usable by sequel_pg.

larskanis added a commit that referenced this issue Mar 9, 2022
conn.block still reads data from the socket, but only after checking the result queue and waiting for readability first.
Reading data from the socket (PQconsumeInput) takes much more time than just checking available results (PQisBusy).
So doing it as a precaution to conn.block is a performance issue, now that we call block before each get_result in pg-1.3.x.

The test was introduced in commit 09bdc16 .
It is not particular specific to conn.block, so that we can check the error class on consume_input equaly.

Fixes #442
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant