Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GODRIVER-3132 Use dst buf pool for zstd encoding. #1577

Open
wants to merge 1 commit into
base: v1
Choose a base branch
from
Open

GODRIVER-3132 Use dst buf pool for zstd encoding. #1577

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
15 changes: 14 additions & 1 deletion x/mongo/driver/compression.go
View file
Open in desktop
Expand Up @@ -105,6 +105,13 @@ func (e *zlibEncoder) Encode(dst, src []byte) ([]byte, error) {
return dst, nil
}

var zstdBufPool = sync.Pool{
New: func() interface{} {
s := make([]byte, 0)
return &s
},
}

// CompressPayload takes a byte slice and compresses it according to the options passed
func CompressPayload(in []byte, opts CompressionOpts) ([]byte, error) {
switch opts.Compressor {
Expand All @@ -123,7 +130,13 @@ func CompressPayload(in []byte, opts CompressionOpts) ([]byte, error) {
if err != nil {
return nil, err
}
return encoder.EncodeAll(in, nil), nil
ptr := zstdBufPool.Get().(*[]byte)
b := encoder.EncodeAll(in, *ptr)
Comment on lines +133 to +134
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of adding an additional pool of buffers just for Zstd compression, can we leverage the existing op buffer pool to improve memory usage for all compression types?

For example, in compression.go, change CompressPayload to accept a destination byte slice:

func CompressPayload(src, dst []byte, opts CompressionOpts) ([]byte, error) {
	// ...
	return encoder.EncodeAll(src, dst)
	// ...

Then in connection.go, pass the existing destination byte slice into CompressPayload:

func (c *Connection) CompressWireMessage(src, dst []byte) ([]byte, error) {
	// ...
	compressed, err := driver.CompressPayload(rem, dst, opts)
	// ...

Then remove the following call to AppendCompressedCompressedMessage, which is just an alias for append.

dst := make([]byte, len(b))
copy(dst, b)
*ptr = b[:0]
zstdBufPool.Put(ptr)
return dst, nil
default:
return nil, fmt.Errorf("unknown compressor ID %v", opts.Compressor)
}
Expand Down