Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better segment pools #311

Open
7 tasks
fzhinkin opened this issue May 6, 2024 · 0 comments
Open
7 tasks

Better segment pools #311

fzhinkin opened this issue May 6, 2024 · 0 comments

Comments

@fzhinkin
Copy link
Collaborator

fzhinkin commented May 6, 2024

Currently, segment pools exist only on JVM (on other platforms, implementations are effectively no-op) and behave more like caches than pools.

There are a few directions in which we may/should develop segment pools:

  • support pools configuration (both in terms of the pool capacity and the size of allocated segments)
  • make sure every segment is returned to a pool once there are no users remaining
  • support unlinking a segment from a pool (for scenarios when we have a byte-buffer backed segment and now sending it via Netty -> segment is no longer in use once we "consumed" it by wrapping into a ByteBuf, but it could not be released as a Netty owns it now)
  • support adding already allocated data into a buffer (related to Missing API: zero-copy wrapping of ByteArray with Source. #166)
  • support pools on other platforms
  • support pool-level isolation (for instance, if there are multiple threads make sure that each of them uses a separate pool, so that data used in one thread would never leak into another thread)
  • support leak tracing (Leak tracing mechanism #144)

This is an epic describing what could be done and tracking progress rather than an instruction to what should be implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant