Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make spanner changelog stream retention configurable #1457

Open
vroldanbet opened this issue Jul 26, 2023 · 2 comments
Open

make spanner changelog stream retention configurable #1457

vroldanbet opened this issue Jul 26, 2023 · 2 comments
Labels
area/datastore Affects the storage system kind/feature request Something is missing priority/3 low This would be nice to have state/needs discussion This can't be worked on yet

Comments

@vroldanbet
Copy link
Contributor

The SpiceDB Datastore implementation for Spanner now uses changelogs as of version 1.23.0 instead of its own bookkeeping changelog table. The migration used the default retention of 1 day. This is enough for most users, but some may want to change this. Spanner supports up to 7 days of data retention.

@vroldanbet vroldanbet added the area/datastore Affects the storage system label Jul 26, 2023
@josephschorr
Copy link
Member

So this raises an interesting question: The change stream is created via one of the migrations, but the retention period is configured in the same statement... so it would have to be a flag on the migrate call, and it would only be applicable once.

Any thoughts on how to make this devex better?

@josephschorr josephschorr added priority/3 low This would be nice to have state/needs discussion This can't be worked on yet kind/feature request Something is missing labels Aug 4, 2023
@vroldanbet
Copy link
Contributor Author

there is already a precedent of leaving certain DB configuration flags outside of migrate, like cockroach_cluster_gc_window_nanos. We can either have folks change that themselves or we could introduce support for migration revision-specific options. Not a fan of it though, would have to be pushed to the operator too, and require folks to edit CRs to add the specific options they want to be passed to the corresponding migration. Thoughts / other ideas @ecordell?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/datastore Affects the storage system kind/feature request Something is missing priority/3 low This would be nice to have state/needs discussion This can't be worked on yet
Projects
None yet
Development

No branches or pull requests

2 participants