Resolve #1398: Online Indexer: support multi target indexing #1399

jjezra · 2021-09-06T15:13:03Z

No description provided.

alecgrieser

Okay, went through these changes. The changes requested are non-trivial, but the basic theme is basically that I think that there's an opportunity here to collapse some special cases (and remove some code dupe) if we do some refactoring. One of the more important places where I'd like to see that is that I think there's a fair amount of overlap between the MultiTypeIndexingByRecords and the IndexingByRecords classes, and it would probably be good to clean that up, given that the code being duplicated is non-trivial. But that might be a bit of an involved change, so I'm not sure how we want to go about doing that

...yer-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingBase.java

fdb-record-layer-core/src/main/proto/index_build.proto

...yer-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingBase.java

...er-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/OnlineIndexer.java

...t/java/com/apple/foundationdb/record/provider/foundationdb/OnlineIndexerMultiTargetTest.java

jjezra · 2021-10-06T15:28:16Z

Okay, went through these changes. The changes requested are non-trivial, but the basic theme is basically that I think that there's an opportunity here to collapse some special cases (and remove some code dupe) if we do some refactoring. One of the more important places where I'd like to see that is that I think there's a fair amount of overlap between the MultiTypeIndexingByRecords and the IndexingByRecords classes, and it would probably be good to clean that up, given that the code being duplicated is non-trivial. But that might be a bit of an involved change, so I'm not sure how we want to go about doing that

I think that at some point, after sufficient testing, we should refactor the IndexingByRecords to be a simple case of MultiTypeIndexingByRecords with a single item in the targets list. Does it make sense to you?

alecgrieser · 2021-10-06T22:42:30Z

I think that at some point, after sufficient testing, we should refactor the IndexingByRecords to be a simple case of MultiTypeIndexingByRecords with a single item in the targets list. Does it make sense to you?

I think my inclination is that we should do that sooner rather than waiting for a later refactoring, given the complexity of the classes involved and the level of code duplication in the code currently. I suppose there's an argument against that if it gets multi-targeting in faster (and we really want the performance gains), but I think we'd rather do the refactoring now if possible.

...yer-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingBase.java

alecgrieser · 2021-10-07T00:42:49Z

...-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingByIndex.java

@@ -192,7 +187,7 @@ private Index getSourceIndex(RecordMetaData metaData) {

            return iterateRangeOnly(store, cursor,
                    this::getRecordIfTypeMatch,
-                    lastResult, hasMore, recordsScanned)
+                    lastResult, hasMore, recordsScanned, true)


How is using snapshot isolation essential to scanning by an index?

This is sort of a theme here (in multiple comments on this PR), but it seems like this change is introducing code in a few places (here included) where assumptions about what features are and are not supported are being baked in in sort of subtle ways, and I'm concerned that they will be hard to disentangle in the future, which is why I'm bringing them up.

...-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingByIndex.java

...r-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingCommon.java

...n/java/com/apple/foundationdb/record/provider/foundationdb/IndexingMultiTargetByRecords.java

...er-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/OnlineIndexer.java

...n/java/com/apple/foundationdb/record/provider/foundationdb/IndexingMultiTargetByRecords.java

alecgrieser · 2021-10-07T01:14:26Z

...n/java/com/apple/foundationdb/record/provider/foundationdb/IndexingMultiTargetByRecords.java

+    }
+
+    @Nonnull
+    private CompletableFuture<Boolean> buildRangeOnly(@Nonnull FDBRecordStore store, byte[] startBytes, byte[] endBytes, @Nonnull AtomicLong recordsScanned) {


A bit of reiterating but: I think I'm skeptical that we want to take on the debt of having two diverging implementations here for the benefit of not disturbing the existing implementation. If we're not confident in our testing, then it seems like we should think about adding additional tests.

...t/java/com/apple/foundationdb/record/provider/foundationdb/OnlineIndexerMultiTargetTest.java

foundationdb-ci · 2021-10-11T23:24:20Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: c362f7b3b85fded2d5853959218981eafd6fad94
Result: FAILED
Error: Unexpected EC2 error: RequestLimitExceeded
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-11T23:24:24Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto3
Commit ID: c362f7b3b85fded2d5853959218981eafd6fad94
Result: FAILED
Error: Unexpected EC2 error: RequestLimitExceeded
Build Logs (available for 30 days)

alecgrieser

I like that synthetic type support was added, and I think the record type selection now looks right to me (i.e., records only go to indexes of the right type now). Most of the things I've left comments on are more stylistic, though it also sounded like you might have another set of tweaks you wanted to make here

...yer-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingBase.java

...r-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingCommon.java

...t/java/com/apple/foundationdb/record/provider/foundationdb/OnlineIndexerMultiTargetTest.java

...yer-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingBase.java

alecgrieser · 2021-10-12T00:38:37Z

...yer-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingBase.java

+                            }
+                            // Index is built because there is no missing range.
+                            System.out.println(("marking readable:" + index.getName()));
+                            return store.markIndexReadable(index)


I guess this method has always been like this, but it's a bit sketch. The markIndexReadable method already checks the range set before it marks something as readable. Given that the old method also did this, this is probably fine, but it's worth being aware of.

...yer-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingBase.java

foundationdb-ci · 2021-10-12T12:05:16Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: e93c021c60d76fe312e25416aceb870fd357ec9a
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-12T12:05:38Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto3
Commit ID: e93c021c60d76fe312e25416aceb870fd357ec9a
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-12T12:41:55Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto3
Commit ID: 6f398e61a8066e50c9e023431d9d835d5a6e51a3
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-12T12:42:56Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: 6f398e61a8066e50c9e023431d9d835d5a6e51a3
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-12T16:19:17Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: 6159c8a9164c57be9b9f7aaf50c6435e0d5da534
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-12T16:19:24Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto3
Commit ID: 6159c8a9164c57be9b9f7aaf50c6435e0d5da534
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-12T16:20:35Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: ccb98c3dd19780174cde3ad446662c39e9f1a020
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-12T16:24:09Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto3
Commit ID: ccb98c3dd19780174cde3ad446662c39e9f1a020
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-12T16:42:38Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: 7fc3c9aac2b24d306513453c6b3e93623a82b10d
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

alecgrieser · 2021-10-12T16:45:17Z

Interesting. Not sure I've seen this test failure before:

SizeStatisticsCollectorTest > tryStaleRead() FAILED
    com.apple.foundationdb.record.provider.foundationdb.FDBExceptions$FDBStoreException: Version not valid
        at com.apple.foundationdb.record.provider.foundationdb.FDBExceptions.wrapException(FDBExceptions.java:189)
        at com.apple.foundationdb.record.provider.foundationdb.FDBDatabase.lambda$new$0(FDBDatabase.java:169)
        at com.apple.foundationdb.record.provider.foundationdb.FDBDatabase.mapAsyncToSyncException(FDBDatabase.java:1058)
        at com.apple.foundationdb.record.provider.foundationdb.SizeStatisticsCollectorTest$SizeStatisticsCollector.lambda$collectAsync$1(SizeStatisticsCollectorTest.java:358)
        at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930)
        at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907)
        at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
        at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
        at com.apple.foundationdb.async.AsyncUtil$LoopPartial.apply(AsyncUtil.java:346)
        at com.apple.foundationdb.async.AsyncUtil$LoopPartial.apply(AsyncUtil.java:332)
        at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930)
        at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:907)
        at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478)
        at com.apple.foundationdb.async.TaskNotifyingExecutor$Notifier.run(TaskNotifyingExecutor.java:77)
        at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

        Caused by:
        com.apple.foundationdb.FDBException: Version not valid
            at com.apple.foundationdb.NativeFuture.Future_getError(Native Method)
            at com.apple.foundationdb.FutureResults.getIfDone_internal(FutureResults.java:47)
            at com.apple.foundationdb.FutureResults.getIfDone_internal(FutureResults.java:28)
            at com.apple.foundationdb.NativeFuture.marshalWhenDone(NativeFuture.java:63)
            ... 7 more

It seems unlikely that this is caused by this change, though, and it's probably a flaky test somehow. Strange that we haven't seen it before

foundationdb-ci · 2021-10-12T16:49:02Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto3
Commit ID: 7fc3c9aac2b24d306513453c6b3e93623a82b10d
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

jjezra · 2021-10-12T17:15:11Z

tryStaleRead passes on my local environment. "Test this please" doesn't seem to work. I'll make a minor change and try to push..

foundationdb-ci · 2021-10-12T17:32:15Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto3
Commit ID: 9da4acfec4a4572c8c0fd1bfe4346519a4167a98
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-12T17:32:16Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: 9da4acfec4a4572c8c0fd1bfe4346519a4167a98
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-12T17:54:09Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: 9023b1406ff5e27b98faf2100a7400ee11a3284a
Result: FAILED
Error: Error in UPLOAD_ARTIFACTS phase: [code_coverage: [report files not found in build]]
Build Logs (available for 30 days)

jjezra · 2021-10-12T21:33:22Z

Rebased.

foundationdb-ci · 2021-10-12T21:47:42Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: 95abfc3
Result: SUCCEEDED
Error: N/A
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-12T21:48:27Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto3
Commit ID: 95abfc3
Result: SUCCEEDED
Error: N/A
Build Logs (available for 30 days)

alecgrieser

A few minor things remaining, but things are largely shaping up.

I think the "fundamental disagreement" is still there over whether this should go in like this (as a separate indexer) versus changing the existing "ByRecords" indexer. My preference, I think, would still be to just change the existing indexer and rely on test coverage to find any potential problems (or make our test coverage better if we think it's lacking somewhere). It's possible that we should take this any way, if you feel strongly, but I don't like the amount of duplicated code that this introduces, if I'm honest.

...yer-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingBase.java

alecgrieser · 2021-10-12T19:02:09Z

...-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingByIndex.java

@@ -192,7 +187,7 @@ private Index getSourceIndex(RecordMetaData metaData) {

            return iterateRangeOnly(store, cursor,
                    this::getRecordIfTypeMatch,
-                    lastResult, hasMore, recordsScanned)
+                    lastResult, hasMore, recordsScanned, true)


Okay, I see; the idempotency requirement is a bit more intertwined with how the indexer updates than I guess I thought. I still think that this check here should base its interpretation of whether the index update is idempotent based on the actual index rather than hard-coding true, though. The main reason is that it seems like it would be something that we'd easily forget to update if we came up with a way of building a non-idempotent index from another index. (For example, by checking if a record's corresponding index entry is in a built range.)

...r-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingCommon.java

alecgrieser · 2021-10-12T22:16:41Z

.../src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingScrubMissing.java

@@ -174,7 +174,7 @@ public IndexingScrubMissing(@Nonnull final IndexingCommon common,
            final long scanLimit = scrubbingPolicy.getEntriesScanLimit();

            return iterateRangeOnly(store, cursor, this::getRecordIfMissingIndex,
-                    lastResult, hasMore, recordsScanned)
+                    lastResult, hasMore, recordsScanned, true)


I think I'm still of the opinion (as expressed in a comment elsewhere) that hard-coding this to true could be easily missed if we ever decide we want to not restrict scrubbers to idempotent indexes

...er-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/OnlineIndexer.java

fdb-record-layer-core/src/main/proto/index_build.proto

...ava/com/apple/foundationdb/record/provider/foundationdb/OnlineIndexerIndexFromIndexTest.java

…split some functions. This seems to be a complicated way to reduce complexity. Most methods are, and should be, a "single task" leaf, but some are skeletons/trunks - and implementing a certain "storyline" agorithm. Spreading this algorithm among multiple functions makes it anything but more readable.

foundationdb-ci · 2021-10-13T00:12:09Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: 13d361d
Result: SUCCEEDED
Error: N/A
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-13T00:13:56Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto3
Commit ID: 13d361d
Result: SUCCEEDED
Error: N/A
Build Logs (available for 30 days)

jjezra · 2021-10-13T00:45:16Z

A few minor things remaining, but things are largely shaping up.

I think the "fundamental disagreement" is still there over whether this should go in like this (as a separate indexer) versus changing the existing "ByRecords" indexer. My preference, I think, would still be to just change the existing indexer and rely on test coverage to find any potential problems (or make our test coverage better if we think it's lacking somewhere). It's possible that we should take this any way, if you feel strongly, but I don't like the amount of duplicated code that this introduces, if I'm honest.

Yes. I strongly feel that it's important that for the time being, while multi indexing is new, users can use the good ol' ByRecord path as a fallback if anything goes wrong.
I'll be more than happy to merge them after becoming a little less paranoid about it - and, in the meantime, will consider your confidence in my code as a great compliment :)

foundationdb-ci · 2021-10-13T01:05:35Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: e31c476
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-13T01:09:21Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto3
Commit ID: e31c476
Result: FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-13T01:26:38Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: 739b3e5
Result: SUCCEEDED
Error: N/A
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-13T01:27:27Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto3
Commit ID: 739b3e5
Result: SUCCEEDED
Error: N/A
Build Logs (available for 30 days)

alecgrieser

Okay, a few minor cleanup things, mainly around making sure we have issues filed for any follow up work we want to do post-merge

alecgrieser · 2021-10-13T16:28:04Z

...-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingByIndex.java

@@ -257,9 +247,10 @@ private Index getSourceIndex(RecordMetaData metaData) {
        final AtomicReference<RecordCursorResult<FDBIndexedRecord<Message>>> lastResult = new AtomicReference<>(RecordCursorResult.exhausted());
        final AtomicBoolean hasMore = new AtomicBoolean(true);

+        final boolean isIdempotent = true ; // Note that currently indexing by index is online implemented for idempotent indexes


Is there an issue about supporting non-idempotent indexes being built from another index? If so, could you link it here so that there's something for a future implementor of that to search for. If not, I think one should probably be filed (and linked here). We probably want something similar for the scrubbers as well.

Also - #1433 (Support multi target indexes while indexing by a source index)

...er-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/OnlineIndexer.java

alecgrieser · 2021-10-13T16:35:40Z

...n/java/com/apple/foundationdb/record/provider/foundationdb/IndexingMultiTargetByRecords.java

+ * This indexer scans records to build multiple indexes.
+ */
+@API(API.Status.INTERNAL)
+public class IndexingMultiTargetByRecords extends IndexingBase {


There should probably be an issue filed about combining this with IndexingByRecords (or maybe more generally about combining multi-target indexing with single-target indexing), even if this goes in as a separate indexer just for tracking purposes.

sonarcloud · 2021-10-13T18:25:11Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
14 Code Smells

0.0% Coverage
1.2% Duplication

foundationdb-ci · 2021-10-13T18:25:53Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto2
Commit ID: 5183d44
Result: SUCCEEDED
Error: N/A
Build Logs (available for 30 days)

foundationdb-ci · 2021-10-13T18:26:04Z

AWS CodeBuild CI Report for Linux CentOS 7

CodeBuild project: fdb-record-layer-pull-request-build-proto3
Commit ID: 5183d44
Result: SUCCEEDED
Error: N/A
Build Logs (available for 30 days)

jjezra requested a review from alecgrieser September 6, 2021 15:13

jjezra requested a review from scgray October 5, 2021 19:07

alecgrieser requested changes Oct 6, 2021

View reviewed changes

jjezra force-pushed the oi_multi_target branch from b05d481 to 3c59460 Compare October 6, 2021 23:32

alecgrieser requested changes Oct 7, 2021

View reviewed changes

...t/java/com/apple/foundationdb/record/provider/foundationdb/OnlineIndexerMultiTargetTest.java Show resolved Hide resolved

...t/java/com/apple/foundationdb/record/provider/foundationdb/OnlineIndexerMultiTargetTest.java Show resolved Hide resolved

jjezra removed the request for review from scgray October 7, 2021 19:58

alecgrieser requested changes Oct 12, 2021

View reviewed changes

jjezra force-pushed the oi_multi_target branch from c362f7b to e93c021 Compare October 12, 2021 11:57

jjezra requested a review from alecgrieser October 12, 2021 12:33

jjezra force-pushed the oi_multi_target branch from 6159c8a to ccb98c3 Compare October 12, 2021 16:11

jjezra added 3 commits October 12, 2021 17:32

remove comment

15865fe

spotbugsMain adjustment

4a2fe46

test this please?

95abfc3

jjezra force-pushed the oi_multi_target branch from 4ea072e to 95abfc3 Compare October 12, 2021 21:33

alecgrieser requested changes Oct 12, 2021

View reviewed changes

alecgrieser linked an issue Oct 13, 2021 that may be closed by this pull request

Online Indexer: support multi target indexing #1398

Closed

implement more requested changes

e31c476

jjezra requested a review from alecgrieser October 13, 2021 00:59

leftover bug fix

739b3e5

alecgrieser requested changes Oct 13, 2021

View reviewed changes

This was referenced Oct 13, 2021

SizeStatisticsCollectorTest.tryStaleRead sometimes fails with "version not valid" #1431

Closed

Fixes #1371: Uniqueness violations during check version appear to be silently ignored #1373

Merged

Add clarification to addTargetIndex javadoc

5183d44

jjezra requested a review from alecgrieser October 13, 2021 18:16

alecgrieser approved these changes Oct 13, 2021

View reviewed changes

alecgrieser merged commit fbd60ce into FoundationDB:main Oct 13, 2021

Resolve #1398: Online Indexer: support multi target indexing #1399

Resolve #1398: Online Indexer: support multi target indexing #1399

Conversation

jjezra commented Sep 6, 2021

alecgrieser left a comment

Choose a reason for hiding this comment

jjezra commented Oct 6, 2021

alecgrieser commented Oct 6, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

foundationdb-ci commented Oct 11, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 11, 2021

AWS CodeBuild CI Report for Linux CentOS 7

alecgrieser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

alecgrieser commented Oct 12, 2021

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

jjezra commented Oct 12, 2021

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

jjezra commented Oct 12, 2021

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 12, 2021

AWS CodeBuild CI Report for Linux CentOS 7

alecgrieser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

foundationdb-ci commented Oct 13, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 13, 2021

AWS CodeBuild CI Report for Linux CentOS 7

jjezra commented Oct 13, 2021

foundationdb-ci commented Oct 13, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 13, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 13, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 13, 2021

AWS CodeBuild CI Report for Linux CentOS 7

alecgrieser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarcloud bot commented Oct 13, 2021

foundationdb-ci commented Oct 13, 2021

AWS CodeBuild CI Report for Linux CentOS 7

foundationdb-ci commented Oct 13, 2021

AWS CodeBuild CI Report for Linux CentOS 7