Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate feasibility of zero-allocation protobuf marshalers #5727

Closed
jack-berg opened this issue Aug 17, 2023 · 2 comments
Closed

Investigate feasibility of zero-allocation protobuf marshalers #5727

jack-berg opened this issue Aug 17, 2023 · 2 comments
Labels
Feature Request Suggest an idea for this project

Comments

@jack-berg
Copy link
Member

The OTLP protobuf marshalers maintain a bunch of state as they transform SpanData, MetricData, LogRecordData to an intermediate representation before serializing. We should investigate whether we can eliminate this state to produce zero allocation marshalers.

Discussed in #5105 and in discussion #5724.

@jack-berg jack-berg added the Feature Request Suggest an idea for this project label Aug 17, 2023
@asafm
Copy link
Contributor

asafm commented Jan 30, 2024

Protocol Buffers forces you to know the size of an embedded message before you write its binary encoding.
This means you must:

  1. Encode the embedded message to a temporary buffer
  2. Grab the length of the buffer
  3. Write the size of the buffer
  4. Write the buffer (encoding of the message).

The intermediate classes' primary goal (i.e., Marshaler*) is to contain the size for each. Another goal is mapping the protobuf schema since the schema and the MetricData are not entirely 1:1 mapping.

I spent two days and had yet to do the serialization in a streaming fashion; hence, the only way out I saw was the reusable objects and object pooling. A very rough idea is at #6171.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature Request Suggest an idea for this project
Projects
None yet
Development

No branches or pull requests

2 participants