Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement NewBatchWithSize method on DB interface to pre-allocate buffer #1

Closed
wants to merge 2 commits into from

Conversation

p0mvn
Copy link
Member

@p0mvn p0mvn commented Feb 24, 2022

Background

Ethereum recently discovered that growing the buffer on goleveldb is slow and implemented a fix:
ethereum/go-ethereum#24392

This PR introduced the same method with pre-allocated buffer to tm-db

  • Did not implement the fix for rocksdb, badger etc for now, planning to test and benchmark first

@p0mvn p0mvn removed the request for review from tac0turtle February 24, 2022 03:43
@p0mvn p0mvn marked this pull request as draft February 24, 2022 03:43
@@ -63,6 +63,9 @@ type DB interface {
// NewBatch creates a batch for atomic updates. The caller must call Batch.Close.
NewBatch() Batch

// NewBatch creates a batch for atomic updates with pre-allocated buffer. The caller must call Batch.Close.
NewBatchWithSize(size int) Batch

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

part of me thinks it would make sense to not break that api and merely use it goleveldb. Unless we see improvements on rocks and others

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.

I'm making changes to IAVL now to benchmark how much of a difference this change makes. I will come back here after and look into other databases. Otherwise, rethink this PR

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the Stale label Apr 13, 2022
@github-actions github-actions bot closed this Apr 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
2 participants