Resolves #1923: Avoid copying continuation byte arrays by using ByteString internally #1924

alecgrieser · 2022-11-18T12:09:08Z

This updates the logic in the RecordCursorContinuation objects to avoid serializing a continuation all the way to a byte[] if it is not necessary. For example, when constructing protobuf objects that back continuations, child continuations can be constructed by calling .toByteString() on the continuation and passing that to the protobuf builder instead of serializing all the to a byte[] and then wrapping it back in a ByteString. This also makes sure that (as much as possible) the cursor implementations are more efficient when constructing a ByteString. For example, for the proto-based continuations, it should be able to return .toByteString() on the protobuf object instead of serializing the protobuf all the way to a byte[] and then wrapping it back up, which some implementations did previously.

Adopters who need a ByteString rather than a byte[] (for example, to pack the continuation into a protobuf) are also encouraged to switch to the newer method.

This resolves #1923. This also follows up from #1921, which introduced the zero copy ByteString to avoid copying immutable arrays.

foundationdb-ci · 2022-11-18T12:17:02Z

Result of fdb-record-layer-pr-proto2 on Linux CentOS 7

Commit ID: f3ea6ba
Duration 0:07:43
Result: ❌ FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-18T12:17:12Z

Result of fdb-record-layer-pr-proto3 on Linux CentOS 7

Commit ID: f3ea6ba
Duration 0:07:55
Result: ❌ FAILED
Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest sonarqube -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

MMcM

Changes look good, but some test failures look to be actual regressions.

… using ByteString internally This updates the logic in the `RecordCursorContinuation` objects to avoid serializing a continuation all the way to a `byte[]` if it is not necessary. For example, when constructing protobuf objects that back continuations, child continuations can be constructed by calling `.toByteString()` on the continuation and passing that to the protobuf builder instead of serializing all the to a `byte[]` and then wrapping it back in a `ByteString`. This also makes sure that (as much as possible) the cursor implementations are more efficient when constructing a `ByteString`. For example, for the proto-based continuations, it should be able to return `.toByteString()` on the protobuf object instead of serializing the _protobuf_ all the way to a `byte[]` and then wrapping it back up, which some implementations did previously. Adopters who need a `ByteString` rather than a `byte[]` (for example, to pack the continuation into a protobuf) are also encouraged to switch to the newer method. This resolves FoundationDB#1923.