New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: support queries with up to 65535 (inclusive) parameters #2525
Conversation
ce40224
to
1fe3a38
Compare
I just pushed revert commits to check if the CI duration improves. In an unlikely case, "fix close refcursors" caused cursor leaks or something like that. |
Rolling back "fix close refcursors" reduces CI duration from 3h50m to 50min. |
Ok, the reason is "org.postgresql.test.jdbc2.RefCursorFetchTest | | PASSED | 2h 57m 24.946s" In other words, it does not look like a code regression, rather it looks like RefCursorFetchTest is extremely slow for unknown reasons. |
that seems rather odd ? |
It all means "fix close refcursors" was OK, and there's something else going slow with CI env. The worst thing is that if I launch just So it looks like some of the tests before RefCursorFetchTest triggers a bad condition, so subsequent tests become slow. |
we can try to ask for support from github. They have been helpful in the past on security isssues. |
ae847cd
to
9c85185
Compare
Finally, I nailed it: 82dbbe4#diff-dd8064c49349abfe7ba0b6e2d65ebb7d89600239c982dd0556bdace3765f751cL24-L25
That caused all the test after Just in case you wonder, the trace log does show buffer sizes:
|
Previously the execution failed with "Tried to send an out-of-range integer as a 2-byte value" when the user attempted executing a query with more than 32767 parameters. Technically speaking, the wire protocol limit is 2-byte-unsigned-int, so we should support 65535 parameters. In practice, simple mode (preferQueryMode=simple) allows executing queries with an arbitrary number of parameters, however, that escape hatch is not recommended as it still has limits on the SQL length, and it would likely be slow. fixes pgjdbc#1311
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error message as it stands prints the wrong thing:
org.postgresql.util.PSQLException: PreparedStatement can have at most 65,535 parameters. Please consider using arrays, or splitting the query in several ones, or using COPY. Given query has 65,535 parameters
at org.postgresql.jdbc.PgPreparedStatement.<init>(PgPreparedStatement.java:102)
at org.postgresql.jdbc.PgPreparedStatement.<init>(PgPreparedStatement.java:88)
Previously the execution failed with "Tried to send an out-of-range integer as a 2-byte value"
when the user attempted executing a query with more than 32767 parameters.
Technically speaking, the wire protocol limit is 2-byte-unsigned-int,
so we should support 65535 parameters.
In practice, simple mode (preferQueryMode=simple) allows executing queries
with an arbitrary number of parameters, however, that escape hatch is not recommended
as it still has limits on the SQL length, and it would likely be slow.
fixes #1311