Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-use nested container encoding buffers #726

Open
zslayton opened this issue Mar 7, 2024 · 0 comments
Open

Re-use nested container encoding buffers #726

zslayton opened this issue Mar 7, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@zslayton
Copy link
Contributor

zslayton commented Mar 7, 2024

In the binary writers (v1.0 and v1.1), length-prefixed container writers bump-allocate a fresh encoding buffer where they can encode their child values to calculate their aggregated byte length. When encoding is done, the length is encoded into a parent buffer followed by the encoding buffer's contents. After this point, the encoding buffer becomes inaccessible, and the memory it occupies remains unavailable until the bump allocator is eventually reset during a flush().

Instead, we should maintain a stack of these encoding buffers that we can pop from and push to over the course of encoding, minimizing the amount of memory that is temporarily "leaked" between calls to flush(). This will require plumbing an (e.g.) EncodingResources reference through the ValueWriter implementations so buffers can be acquired and released at any level of encoding depth.

cc @popematt

@zslayton zslayton added the enhancement New feature or request label Mar 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant