Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DB-side UPSERT optimiziation support #138

Merged
merged 5 commits into from
May 24, 2024
Merged

DB-side UPSERT optimiziation support #138

merged 5 commits into from
May 24, 2024

Conversation

peterbroadhurst
Copy link
Contributor

@peterbroadhurst peterbroadhurst commented May 13, 2024

  • Provides a new feature option for DB providers to build a DB-optimized UPSERT functionality
    • Implementation for PSQL provided for re-use by repos using this module
    • Slightly annoying workaround to sq semantics called out in comments
  • Provides a new optimization type that requests DB optimization of the upsert
    • Didn't feel ready to make this the default
    • Falls back to update optimization if not available on DB provider

Signed-off-by: Peter Broadhurst <peter.broadhurst@kaleido.io>
Signed-off-by: Peter Broadhurst <peter.broadhurst@kaleido.io>
Signed-off-by: Peter Broadhurst <peter.broadhurst@kaleido.io>
@peterbroadhurst peterbroadhurst marked this pull request as ready for review May 16, 2024 22:34
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.96%. Comparing base (b6bcbb4) to head (e1a20e0).
Report is 6 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #138   +/-   ##
=======================================
  Coverage   99.96%   99.96%           
=======================================
  Files          79       80    +1     
  Lines        6507     6567   +60     
=======================================
+ Hits         6505     6565   +60     
  Misses          1        1           
  Partials        1        1           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@peterbroadhurst peterbroadhurst marked this pull request as draft May 16, 2024 22:42
Signed-off-by: Peter Broadhurst <peter.broadhurst@kaleido.io>
Signed-off-by: Peter Broadhurst <peter.broadhurst@kaleido.io>
@peterbroadhurst peterbroadhurst marked this pull request as ready for review May 16, 2024 22:59
if err != nil || !ok {
return insert, i18n.NewError(ctx, i18n.MsgDBErrorBuildingStatement, err)
}
return insert.Suffix(fmt.Sprintf("ON CONFLICT (%s) DO UPDATE", idColumn)).SuffixExpr(sq.Expr(updateSQL, updateValues...)).Suffix("RETURNING " + returnCol), nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this alternate cheat would also work (using a single space as the table name, and then not performing any stripping):

insert = sq.Insert(table).Columns(insertCols...).Values(insertValues...)
update := sq.Update(" " /* cheat to avoid table name */)
for _, c := range updateCols {
	update = update.Set(c, values[c])
}
updateSQL, updateValues, err := update.ToSql()
if err != nil {
	return insert, i18n.NewError(ctx, i18n.MsgDBErrorBuildingStatement, err)
}
return insert.Suffix(fmt.Sprintf("ON CONFLICT (%s) DO", idColumn)).
	SuffixExpr(sq.Expr(updateSQL, updateValues...)).
	Suffix("RETURNING " + returnCol), nil

Happy for you to keep it as is or switch it (not sure either is better/worse - just noting it as a possibility).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to merge for now, thank you though 👍

@peterbroadhurst peterbroadhurst merged commit d4e905e into main May 24, 2024
2 checks passed
@peterbroadhurst peterbroadhurst deleted the db-conflict branch May 24, 2024 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants