Add support for batch operations (R2DBC)

[MarkMarkyMarkus]

Add support for batch operations to the DefaultDatabaseClient:

databaseClient.sql("INSERT INTO legoset (id, name, manual) VALUES(:id, :name, :manual)")
				.bind("id", 42055)
				.bind("name", "SCHAUFELRADBAGGER")
				.bindNull("manual", Integer.class)
				.add()
				.bind("id", 2021)
				.bind("name", "TOM")
				.bindNull("manual", Integer.class)
				.fetch().rowsUpdated()

Closes #259

Demo app: springR2dbcBatchExample.tar.gz

[patrickhuy]

@sbrannen @mp911de this looks very intresting for me, could you share some insights what the next steps for this PR should be?

[mp911de]

Batching through .add() would only properly work if the SQL doesn't change. DatabaseClient uses named parameters and expands named parameter markers using collection arguments into native bind markers. The number of bind markers depends on the actual collection size. This is useful when using IN clauses. I'm not quite sure about the direction.

We could follow the idea of NamedParameter of expanding the SQL using the first bindings and raising an exception later on if the expanded parameter count doesn't match the number of parameters found in a collection argument.

Other than that, please check out the contribution guide to learn how we accept contributions.

[MarkMarkyMarkus]

Thanks for the response.

We could follow the idea of NamedParameter of expanding the SQL using the first bindings and raising an exception later on if the expanded parameter count doesn't match the number of parameters found in a collection argument.

Do you mean that we should check the expanded parameters count in NamedParameterUtils.substituteNamedParameters() or NamedParameter.addPlaceholder()?

[Andro999b]

Is there any plans to add batching?

[gabfssilva]

Batching through .add() would only properly work if the SQL doesn't change.

@mp911de, correct me if I'm wrong, but the SPI has two ways of batching, right? One that allows you to send a single command with multiple parameters (via parameter binding) & another that allows you to pass multiple commands and, although both are sent in a single roundtrip, the latter one is slower since the database will have to parse every single command, so if your command won't change, you should stick with the first approach.

I'm asking because the implementation of this PR is probably the most optimized (since it uses the first approach), although less flexible. Isn't there a way to add both ways to the DatabaseClient, so the developer would choose which way fits better for them?

[mp911de]

@gabfssilva your perception is correct. Parametrized batching parses the command once and sends a execute command to the database for each binding set. I think it would be possible to bridge the .add(…)-based mechanism into DatabaseClient. The other type of batching, where commands are concatenated with ; can be used already because you're in control of the SQL statement.

[bwhyman]

It works for me now.
spring-data-r2dbc: 6.0.1; jasync-r2dbc-mysql: 2.1.12.

@Autowired
private io.r2dbc.spi.ConnectionFactory connectionFactory;
@Transactional
    public Mono<Void> update(List<User> users) {
        String sql = "update user u set u.address=? where s.id=?";
        return DatabaseClient.create(connectionFactory).sql(sql).filter(statement -> {
            for (User u : users) {
                statement.bind(0, u.getAddress()).bind(1, s.getId()).add();
            }
            return statement;
        }).then();
    }

[snicoll]

@mp911de I am happy helping the merge process but I could use a review on your side first.

[mp911de]

From the spec, we've learned that statement.….add().….add().….execute() (no trailing .add() before .execute()) is a pretty sharp edge leading easily to quite some confusion when using that API.

Considering batching with DatabaseClient would allow us to come up with a smoother API and an approach, that doesn't create DefaultGenericExecuteSpec instances for each bound parameter.

As a starting point, how about a bind (or params as in JdbcClient) method lambda, something along the lines of:

databaseClient.sql("INSERT INTO legoset (id, name, manual) VALUES(:id, :name, :manual)")
				.bind(params -> params.bind("name", name1).bind("manual", manual1))
				.bind(params -> params.bind("name", name2).bind("manual", manual2))
				.fetch();

The lambda parameter would be a simple interface carrying bind methods only to avoid confusion with Statement:

interface Params {
  Params bind(String name, Object value);
  Params bind(int index, Object value);
  Params bindNull(String name, Class<?> type);
  Params bindNull(int index, Class<?> type);
}

[snicoll]

@MarkMarkyMarkus would you have some cycle to amend your PR with the review of Mark? If not, that's cool we can take over when time permits.

[MarkMarkyMarkus]

Yes, I will take a look.

[jhoeller]

Note that we intentionally designed our new JdbcClient without batch operations, so it is not a priority for R2DBC either. That said, if we can agree on an implementation style for multiple parameter bindings that is not at odds with the existing API design and that is reasonable to use with named parameters as well as indexed parameters, we may revisit this for both JDBC and R2DBC.

[hantsy]

Hope there is a bind method to accept a list/array as params directly to avoid bind object one by one.

.bind(
  List.of(
    Tuple.of(param1, param2, param3...), 
    Tuple.of(param1, param2, param3...),
    ... 
  )
)