Resolves #1430: Support non-idempotent target indexes while indexing by index #1904

alecgrieser · 2022-11-09T16:24:33Z

This adds support for building a non-idempotent target index when indexing from a different source index. This works in a manner that is analogous to the way that this operation works for non-idempotent indexes built by a record scan, except that as the range set contains ranges of index entries from the source index, the maintainer needs to be updated to: (1) check the indexing type stamp and (2) modify the range set check to use the index key instead of the primary key. Then there are some updates to the indexers to adjust logic that assumed the target index type would always be idempotent.

I played around with the tests added in this PR, and I've validated that if I inject a few bugs into the code (like checking the wrong value in the range set) that they fail, so I'm fairly confident in their ability to cover the interesting scenarios. The exception there would be deleteRecordsWhere, which is not currently validated in any of the indexing tests, though which I believe to be correctly handled.

This resolves #1430.

…le indexing by index This adds support for building a non-idempotent target index when indexing from a different source index. This works in a manner that is analogous to the way that this operation works for non-idempotent indexes built by a record scan, except that as the range set contains ranges of index entries from the source index, the maintainer needs to be updated to: (1) check the indexing type stamp and (2) modify the range set check to use the index key instead of the primary key. Then there are some updates to the indexers to adjust logic that assumed the target index type would always be idempotent. This resolves FoundationDB#1430.

foundationdb-ci · 2022-11-09T16:43:27Z

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

Commit ID: 602d8eb
Duration 0:18:43
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-09T16:44:03Z

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

Commit ID: 602d8eb
Duration 0:19:16
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

…rrent inserts

foundationdb-ci · 2022-11-10T12:18:38Z

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

Commit ID: de7b817
Duration 0:15:45
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-10T12:19:30Z

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

Commit ID: de7b817
Duration 0:16:36
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

jjezra · 2022-11-10T14:53:06Z

...yer-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingBase.java

        IndexBuildProto.IndexBuildIndexingStamp indexingTypeStamp = getIndexingTypeStamp(store);

-        return forEachTargetIndex(index -> setIndexingTypeOrThrow(store, continuedBuild, transaction, index, indexingTypeStamp));


Would this code change all the indexes' typestamp with a single transaction? (to avoid changing few indexes but not all..)

Yes, the transactionality of the operation here hasn't changed. The only thing that changed was that the Transaction object is now no longer passed separately from the store, which I made because that way, it is clearer that the store and transaction are linked (i.e., that someone wasn't doing something sneaky like passing a store that was created with a different transaction)

foundationdb-ci · 2022-11-10T18:24:15Z

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

Commit ID: f3baaf8
Duration 0:16:15
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-10T18:24:50Z

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

Commit ID: f3baaf8
Duration 0:16:47
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

alecgrieser · 2022-11-10T18:32:32Z

One thing that hasn't been addressed by this PR (yet): how should this be rolled out? The main concern here with roll out is that if someone starts an index-from-index build on a non-idempotent index, but there are still instances accessing the same record store that are running an older version of the code, then the record update won't check the correct values in the range set, which can result in index corruption. Note that the data written by builds are unchanged, so any ongoing index build can continue

foundationdb-ci · 2022-11-10T18:43:47Z

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

Commit ID: 91feb09
Duration 0:15:47
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-10T18:44:50Z

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

Commit ID: 91feb09
Duration 0:16:48
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-11T18:02:12Z

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

Commit ID: 1c050bf
Duration 0:07:16
Result: ❌ FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-11T18:02:12Z

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

Commit ID: 1c050bf
Duration 0:07:17
Result: ❌ FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

…s that sets maintenance filter and format version

…se a new store format version to prevent data corruption

foundationdb-ci · 2022-11-11T18:31:18Z

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

Commit ID: df488e7
Duration 0:16:11
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-11T18:31:21Z

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

Commit ID: df488e7
Duration 0:16:12
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

jjezra

Nice!

jjezra · 2022-11-11T20:33:53Z

...r-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/FDBRecordStore.java

@@ -3499,6 +3492,32 @@ public CompletableFuture<IndexBuildState> getIndexBuildStateAsync(Index index) {
        return IndexBuildState.loadIndexBuildStateAsync(this, index);
    }

+    @API(API.Status.INTERNAL)
+    @Nonnull
+    public CompletableFuture<IndexBuildProto.IndexBuildIndexingStamp> loadIndexBuildStampAsync(Index index) {


Would loadIndexingTypeStamp be more compatible with the rest of the code?
(+ changing all indexBuildStamp to IndexingTypeStampbelow)

Sure. Changed.

jjezra · 2022-11-11T20:52:03Z

...er-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/OnlineIndexer.java

+             * Some sanity checks will be performed, but it is the caller's responsibility to verify that this
+             * source-index covers <em>all</em> the relevant records for the target-index. Also, note that
+             * if the {@linkplain OnlineIndexer.Builder#setIndex(Index) target index} is not idempotent,
+             * the index build will not be executed using the given source index unless the store's


Shouldn't it be: "the indexer will throw an exception and may fall back to by record scan unless..." ?

I've added a clarification as to what happens in that case (that there is a fallback). I believe the old documentation is still correct (the build won't be executed using the source index in the case specified), but I've added more detail about what does happen.

jjezra · 2022-11-11T21:01:09Z

...ava/com/apple/foundationdb/record/provider/foundationdb/indexes/StandardIndexMaintainer.java

+    }
+
+    private <M extends Message> CompletableFuture<Void> updateWriteOnlyByRecords(@Nullable final FDBIndexableRecord<M> oldRecord, @Nullable final FDBIndexableRecord<M> newRecord) {
+        // Check if the record has been built be checking its primary key in the range set. Update the index


"be" = "by"?

jjezra · 2022-11-11T21:16:25Z

...ava/com/apple/foundationdb/record/provider/foundationdb/indexes/StandardIndexMaintainer.java

+                return addedRangeWithKey(oldEntryKey)
+                        .thenCompose(oldInRange -> oldInRange ? update(oldRecord, null) : AsyncUtil.DONE)
+                        .thenCompose(ignore -> addedRangeWithKey(newEntryKey))
+                        .thenCompose(newInRange -> newInRange ? update(null, newRecord) : AsyncUtil.DONE);


Would allOf make sense here?

Sort of. Index maintainers are not thread safe when executing on the same subspace, so the index updates need to be in serial. However, the range set checks should be able to be done concurrently.

foundationdb-ci · 2022-11-14T10:49:56Z

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

Commit ID: 0cd43c9
Duration 0:00:54
Result: ❌ FAILED
Error: Error while executing command: yum -y install https://github.com/apple/foundationdb/releases/download/${FDB_VERSION}/foundationdb-clients-${FDB_VERSION}-1.el7.x86_64.rpm https://github.com/apple/foundationdb/releases/download/${FDB_VERSION}/foundationdb-server-${FDB_VERSION}-1.el7.x86_64.rpm. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-14T11:04:39Z

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

Commit ID: 0cd43c9
Duration 0:15:43
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

…operations

foundationdb-ci · 2022-11-14T16:40:27Z

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

Commit ID: d1e7cc7
Duration 0:15:56
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-14T16:40:54Z

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

Commit ID: d1e7cc7
Duration 0:16:21
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

jjezra · 2022-11-17T16:36:18Z

...-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexingByIndex.java

+        // respond appropriately
+        if (store.getFormatVersion() < FDBRecordStore.CHECK_INDEX_BUILD_TYPE_DURING_UPDATE_FORMAT_VERSION) {
+            validateOrThrowEx(maintainer.isIdempotent(), "target index is not idempotent");
+        }


Would

validateOrThrowEx(maintainer.isIdempotent() || (store.getFormatVersion() >= FDBRecordStore.CHECK_INDEX_BUILD_TYPE_DURING_UPDATE_FORMAT_VERSION), "target index is not idempotent")

be a little more elegant?
(not a big deal..)

jjezra · 2022-11-17T16:52:02Z

...ava/com/apple/foundationdb/record/provider/foundationdb/OnlineIndexerIndexFromIndexTest.java

+        try (OnlineIndexer indexBuilder = newIndexerBuilder()
+                .addTargetIndex(tgtIndex)
+                // Set a format version on the store that does not allow index-from-index builds
+                // Because the store already has this format version, though it should be allowed


I'm not sure that I understand the second line of the comment.
Did you mean to mention that the indexing fallback to "by records" scan?
(fallsback? fallbacks? falls back?)

ScottDugas · 2022-11-17T14:56:05Z

...r-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/FDBRecordStore.java


    // The current code can read and write up to the format version below
-    public static final int MAX_SUPPORTED_FORMAT_VERSION = READABLE_UNIQUE_PENDING_FORMAT_VERSION;
+    public static final int MAX_SUPPORTED_FORMAT_VERSION = CHECK_INDEX_BUILD_TYPE_DURING_UPDATE_FORMAT_VERSION;


Outside the scope of this PR, and it would break backwards compatibility, but the naming of these is confusing, and could probably benefit from being moved to a separate class, then it would be FDBRecordStoreFormatVersions.CHECK_INDEX_BUILD_TYPE_DURING_UPDATE

ScottDugas · 2022-11-17T16:20:22Z

...-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexMaintainer.java

+    /**
+     * Update the associated index for a changed record while the index is in
+     * {@link com.apple.foundationdb.record.IndexState#WRITE_ONLY} mode. For most indexes, this should do the
+     * same thing that a normal update does, but if the index is not {@linkplain #isIdempotent() idempotent},


I think, at some point, the existence of updateWhileWriteOnly could allow us to remove isIdempontent, although we may want to expand the documentation to more clearly describe the behavior that an implementor needs to handle for both update and updateWhileWriteOnly

ScottDugas · 2022-11-17T16:21:45Z

...-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/IndexMaintainer.java

+     * @return a future that is complete when the index update is done
+     */
+    @Nonnull
+    public abstract <M extends Message> CompletableFuture<Void> updateWhileWriteOnly(@Nullable FDBIndexableRecord<M> oldRecord,


I can't recall what the stance is, but this is a breaking change for implementors of IndexMaintainer. Should this have a default implementation that is what we previously used to do in the record store, and then, comment that the default implementation will be removed in the future?

I don't think we ever came down on a super firm stance here, but in this case, I think we want to encourage an adopter who is directly extending IndexMaintainer to take a look at their implementation to see if they need to do anything different for WRITE_ONLY indexes, now that the option is available, and it's better that that be the case than

ScottDugas · 2022-11-17T16:23:13Z

...r-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/FDBRecordStore.java

+    @API(API.Status.INTERNAL)
+    @Nonnull
+    public CompletableFuture<IndexBuildProto.IndexBuildIndexingStamp> loadIndexingTypeStampAsync(Index index) {
+        byte[] stampKey = IndexingBase.indexBuildTypeSubspace(this, index).pack();


It seems a bit weird that IndexingBase continues to manage what the subspace is, but not how to write to it.

ScottDugas · 2022-11-17T16:36:01Z

...ava/com/apple/foundationdb/record/provider/foundationdb/indexes/StandardIndexMaintainer.java

+            // index key expression always returns a single value. In this case, the record is excluded from the index
+            return null;
+        } else if (entries.size() != 1) {
+            throw new RecordCoreException("index produced incorrect number of entries for use as source index");


Does this restriction apply for idempotent indexes, or is this a new restriction for non-idempotent indexes?

This is not a new restriction: there was already a check in the indexer to make sure the index did not create duplicates. However, it is more important for non-idempotent target indexes.

Along those lines, there are actually two codepaths, a transactional build and a non-transactional build, and it turns out this check was missed in the transactional case. I've added the check in the the other case, which also added a few other missed checks.

ScottDugas · 2022-11-17T16:44:51Z

...apple/foundationdb/record/provider/foundationdb/OnlineIndexerBuildGroupedCountIndexTest.java

+                .setRecordsPerSecond(OnlineIndexer.DEFAULT_RECORDS_PER_SECOND * 100)
+                .build()) {
+            CompletableFuture<?> buildFuture = indexer.buildIndexAsync(true);
+            recordsAfter = updater.update(recordsBefore, otherRecordsBefore);


There's definitely a risk that these won't really end up testing concurrent problems, at least with the numbers in the test, but I don't have a good solution, other than make a very large test that has a ton of records, takes hours to build, and randomly does all sorts of mutations while doing so, and then asserts that the result is correct. But, the record layer doesn't really have the infrastructure to do that kind of testing.

ScottDugas · 2022-11-17T16:49:24Z

...c/test/java/com/apple/foundationdb/record/provider/foundationdb/TerribleIndexMaintainer.java

+    @Nonnull
+    @Override
+    public <M extends Message> CompletableFuture<Void> updateWhileWriteOnly(@Nullable final FDBIndexableRecord<M> oldRecord, @Nullable final FDBIndexableRecord<M> newRecord) {
+        return AsyncUtil.DONE;


Should this just call update? or return a failed exception

I had this return DONE rather than throw an error because this index maintainer always returns false to isIdempotent, so this is behavior preserving. I didn't try throwing an error, but IIRC, this maintainer is used in some somewhat delicate tests, so it could cause some tests to break, I think

ScottDugas · 2022-11-17T16:56:59Z

...r-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/FDBRecordStore.java

-                if (!MoreAsyncUtil.isCompletedNormally(future)) {
-                    futures.add(future);
-                }
+            if (isIndexWriteOnly(index)) {


There are other calls to update, do any of those need to be updated to do this check? They weren't doing the previous check, so they may have already been broken for non-idempotent use cases.

Yeah, that seems to be the case. The other calls are:

Within the TextIndexMaintainer, which will only get called on (idempotent) text indexes, so we're fine

Within the indexer, which wants to update the index maintainer unconditionally

Within FDBRecordStore::runSyntheticMaintainers, which I'm less certain about, but that already was in a bit of a suspect state

So nothing seems to be updated in response to this, but maybe we should revisit this after this is in

alecgrieser · 2022-11-17T19:53:54Z

There are a few cleanup/comment/refactoring things that I will try and get into a follow up, but I've focused on addressing things that might affect correctness in this PR right now, but I don't want to delay this too much given that without it, affected index builds could be delayed significantly

sonarcloud · 2022-11-17T20:06:11Z

SonarCloud Quality Gate failed.

0 Bugs
0 Vulnerabilities
0 Security Hotspots
25 Code Smells

0.0% Coverage
0.0% Duplication

foundationdb-ci · 2022-11-17T20:06:26Z

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

Commit ID: f075849
Duration 0:18:02
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-17T20:07:14Z

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

Commit ID: f075849
Duration 0:18:50
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

…by index (#1904) * Resolves #1430: Support non-idempotent target indexes while indexing by index This adds support for building a non-idempotent target index when indexing from a different source index. This works in a manner that is analogous to the way that this operation works for non-idempotent indexes built by a record scan, except that as the range set contains ranges of index entries from the source index, the maintainer needs to be updated to: (1) check the indexing type stamp and (2) modify the range set check to use the index key instead of the primary key. Then there are some updates to the indexers to adjust logic that assumed the target index type would always be idempotent. This resolves #1430.

alecgrieser added 2 commits November 10, 2022 11:17

add additional tests of building a sum index from an index with concu…

758e1ab

…rrent inserts

add tests for deletes and updates during sum index builds

de7b817

add tests for building with a sum index with an index filter

04b16d1

jjezra reviewed Nov 10, 2022

View reviewed changes

update documentation

f3baaf8

add test assert back

91feb09

alecgrieser marked this pull request as ready for review November 10, 2022 18:30

alecgrieser requested review from MMcM and ScottDugas November 10, 2022 18:30

alecgrieser added 2 commits November 11, 2022 18:14

refactor online indexing tests to use common method to create indexer…

5b54f9a

…s that sets maintenance filter and format version

add a requirement for non-idempotent indexes-from-indexes that they u…

df488e7

…se a new store format version to prevent data corruption

alecgrieser force-pushed the update-range-set-index-from-index branch from 1c050bf to df488e7 Compare November 11, 2022 18:14

jjezra requested changes Nov 11, 2022

View reviewed changes

update names and documentation, and execute range checks in parallel

0cd43c9

Add tests for building a grouped COUNT index with deleteRecordsWhere …

d1e7cc7

…operations

MMcM previously approved these changes Nov 17, 2022

View reviewed changes

jjezra self-requested a review November 17, 2022 16:25

jjezra reviewed Nov 17, 2022

View reviewed changes

ScottDugas reviewed Nov 17, 2022

View reviewed changes

jjezra previously approved these changes Nov 17, 2022

View reviewed changes

make sure indexes with duplicates are not chosen innappropriately

f075849

alecgrieser dismissed stale reviews from jjezra and MMcM via f075849 November 17, 2022 19:48

MMcM approved these changes Nov 17, 2022

View reviewed changes

alecgrieser merged commit aeb6ba5 into FoundationDB:main Nov 17, 2022

		IndexBuildProto.IndexBuildIndexingStamp indexingTypeStamp = getIndexingTypeStamp(store);

		return forEachTargetIndex(index -> setIndexingTypeOrThrow(store, continuedBuild, transaction, index, indexingTypeStamp));

Resolves #1430: Support non-idempotent target indexes while indexing by index #1904

Resolves #1430: Support non-idempotent target indexes while indexing by index #1904

Conversation

alecgrieser commented Nov 9, 2022 • edited

foundationdb-ci commented Nov 9, 2022

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

foundationdb-ci commented Nov 9, 2022

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

foundationdb-ci commented Nov 10, 2022

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

foundationdb-ci commented Nov 10, 2022

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

jjezra Nov 10, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

foundationdb-ci commented Nov 10, 2022

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

foundationdb-ci commented Nov 10, 2022

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

alecgrieser commented Nov 10, 2022

foundationdb-ci commented Nov 10, 2022

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

foundationdb-ci commented Nov 10, 2022

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

foundationdb-ci commented Nov 11, 2022

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

foundationdb-ci commented Nov 11, 2022

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

foundationdb-ci commented Nov 11, 2022

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

foundationdb-ci commented Nov 11, 2022

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

jjezra left a comment

Choose a reason for hiding this comment

jjezra Nov 11, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

foundationdb-ci commented Nov 14, 2022

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

foundationdb-ci commented Nov 14, 2022

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

foundationdb-ci commented Nov 14, 2022

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

foundationdb-ci commented Nov 14, 2022

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alecgrieser commented Nov 17, 2022

sonarcloud bot commented Nov 17, 2022

foundationdb-ci commented Nov 17, 2022

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

foundationdb-ci commented Nov 17, 2022

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

alecgrieser commented Nov 9, 2022 •

edited

jjezra Nov 10, 2022 •

edited

jjezra Nov 11, 2022 •

edited