Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pgx): Add support for batch operations #1437

Merged
merged 10 commits into from Feb 15, 2022

Conversation

jrperritt
Copy link
Contributor

This change offers one possible implementation of batch operations via the pgx driver. A full example exercising all of the new API surface can be found in examples/batch/postgresql/db_test.go.

This implementation adds 3 new commands: batchexec, batchmany, and batchone.
Each of them returns a batch object. The batch object has 2 methods:

  • a method corresponding to one of the methods on the pgx.BatchResults object (Exec, Query, QueryRow, respectively).
    Each of the methods takes a callback as a parameter as follows:
Exec(func(int, error))
Query(func(int, []T, error))
QueryRow(func(int, T, error))

The first parameter in the callback is the index of the result from the batch result. That is, if the batch result is completely consumed, i will range from 0 to len(input_params_slice) - 1.

  • a Close method for closing the batch result early and returning the underlying connection.

closes #1275

@jrperritt
Copy link
Contributor Author

I was surprised to see the error in CI, but I see it was recently added in pgx.
I also noticed a place where I didn't update the interface generation code to use batchmany and batchone, only batchexec.
Other than that, only rebased on main.

Copy link
Collaborator

@kyleconroy kyleconroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you have time, a few follow-on asks after this is merged.

  • Can you update query-annotations.md to make it obvious that this only works for the PostgreSQL / Go / pgx combo?
  • Can you rename the batch import test folder to batch_import? Our naming convention doesn't include spaces.

@kyleconroy kyleconroy changed the title Add support for pgx batch operations feat(pgx): Add support for batch operations Feb 15, 2022
Copy link
Collaborator

@kyleconroy kyleconroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it looks like that space issues with batch import is causing the build to fail, so you'll need to fix that before I can merge it in.

@jrperritt
Copy link
Contributor Author

Can you update query-annotations.md to make it obvious that this only works for the PostgreSQL / Go / pgx combo?

Yes, good idea.

Can you rename the batch import test folder to batch_import? Our naming convention doesn't include spaces.

Mine neither! Fixed.

@kyleconroy kyleconroy merged commit 5c5ae9b into sqlc-dev:main Feb 15, 2022
@jrperritt jrperritt deleted the pgx-batch branch February 15, 2022 22:53
@danielbprice
Copy link
Contributor

This is really nice. Thanks for doing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature request: Support for pgx batching
3 participants