Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support non-idempotent target indexes while indexing by index #1430

Closed
jjezra opened this issue Oct 13, 2021 · 0 comments · Fixed by #1904
Closed

Support non-idempotent target indexes while indexing by index #1430

jjezra opened this issue Oct 13, 2021 · 0 comments · Fixed by #1904

Comments

@jjezra
Copy link
Contributor

jjezra commented Oct 13, 2021

To support this, we'll probably need:

  1. Have a way to online index cumulative indexes in SNAPSHOT resolution (While indexing, scan cumulative (non-idempotent) indexes in SNAPSHOT resolution  #1429)
  2. Allow a range to be defined by the source index keys (instead of the record's)

With the combination of these two, a write only index can be defined with this special mode of maintaining the target index iff the generated source index is within an "already built" range.
@jjezra jjezra mentioned this issue Oct 13, 2021
Merged
alecgrieser added a commit to alecgrieser/fdb-record-layer that referenced this issue Nov 9, 2022
@alecgrieser
Resolves FoundationDB#1430: Support non-idempotent target indexes whi…
602d8eb 
…le indexing by index

This adds support for building a non-idempotent target index when indexing from a different source index. This works in a manner that is analogous to the way that this operation works for non-idempotent indexes built by a record scan, except that as the range set contains ranges of index entries from the source index, the maintainer needs to be updated to: (1) check the indexing type stamp and (2) modify the range set check to use the index key instead of the primary key. Then there are some updates to the indexers to adjust logic that assumed the target index type would always be idempotent.

This resolves FoundationDB#1430.
@alecgrieser alecgrieser mentioned this issue Nov 9, 2022
Merged
alecgrieser added a commit that referenced this issue Nov 17, 2022
@alecgrieser
Resolves #1430: Support non-idempotent target indexes while indexing …
aeb6ba5 
…by index (#1904)

* Resolves #1430: Support non-idempotent target indexes while indexing by index

This adds support for building a non-idempotent target index when indexing from a different source index. This works in a manner that is analogous to the way that this operation works for non-idempotent indexes built by a record scan, except that as the range set contains ranges of index entries from the source index, the maintainer needs to be updated to: (1) check the indexing type stamp and (2) modify the range set check to use the index key instead of the primary key. Then there are some updates to the indexers to adjust logic that assumed the target index type would always be idempotent.

This resolves #1430.
ammolitor pushed a commit that referenced this issue Nov 17, 2022
@alecgrieser @ammolitor
Resolves #1430: Support non-idempotent target indexes while indexing …
087e5b2 
…by index (#1904)

* Resolves #1430: Support non-idempotent target indexes while indexing by index

This adds support for building a non-idempotent target index when indexing from a different source index. This works in a manner that is analogous to the way that this operation works for non-idempotent indexes built by a record scan, except that as the range set contains ranges of index entries from the source index, the maintainer needs to be updated to: (1) check the indexing type stamp and (2) modify the range set check to use the index key instead of the primary key. Then there are some updates to the indexers to adjust logic that assumed the target index type would always be idempotent.

This resolves #1430.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@jjezra