Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use exact size, if known, to allocate decompression buffer #3048

Merged
merged 5 commits into from Oct 4, 2019
Merged

Use exact size, if known, to allocate decompression buffer #3048

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4d7f286
Use exact size, if known, to allocate decompression buffer
bboreham Sep 27, 2019
7384879
Simplify DecompressedSize()
bboreham Sep 28, 2019
08ec715
Define CompressorSizer in comments rather than a named interface
bboreham Sep 28, 2019
57a5195
Treat DecompressedSize() as an estimate
bboreham Oct 4, 2019
5e9af82
Use make() instead of buf.Grow()
bboreham Oct 4, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
14 changes: 4 additions & 10 deletions rpc_util.go
View file
Open in desktop
Expand Up @@ -686,18 +686,12 @@ func decompress(compressor encoding.Compressor, d []byte, maxReceiveMessageSize
if size > maxReceiveMessageSize {
return nil, size, nil
}
// size is used as an estimate to size the buffer, but we
// will read more data if available.
var buf bytes.Buffer
buf.Grow(size + bytes.MinRead) // extra space guarantees no reallocation
dfawley marked this conversation as resolved.
Show resolved Hide resolved
// Limit to size+1 so we can detect if the compressed data
// is actually bigger than the reported size
bytesRead, err := buf.ReadFrom(io.LimitReader(dcReader, int64(size)+1))
if err != nil {
return nil, size, err
}
if bytesRead != int64(size) {
return nil, size, fmt.Errorf("read different size than expected (%d vs. %d)", bytesRead, size)
}
return buf.Bytes(), size, nil
bytesRead, err := buf.ReadFrom(io.LimitReader(dcReader, int64(maxReceiveMessageSize)+1))
return buf.Bytes(), int(bytesRead), err
}
}
// Read from LimitReader with limit max+1. So if the underlying
Expand Down