Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: add lru cache to speed up scanning logs #302

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

BusyJay
Copy link
Member

@BusyJay BusyJay commented Mar 30, 2023

# Master:
test engine::tests::bench_engine_fetch_entries              ... bench:      23,738 ns/iter (+/- 1,027)

# This PR:
test engine::tests::bench_engine_fetch_entries              ... bench:      18,127 ns/iter (+/- 517)

It's not a fair comparison because thread local cache can only cache a block, but global lru cache can cache 256MiB. But it's just a way to show potential improvement on scanning logs.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
Global cache is wrong if there are multiple raft engine or raft
engine is reopen multiple times.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@codecov
Copy link

codecov bot commented Mar 30, 2023

Codecov Report

Patch coverage: 94.21% and project coverage change: -0.07 ⚠️

Comparison is base (39f4db4) 97.89% compared to head (02f4c85) 97.82%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #302      +/-   ##
==========================================
- Coverage   97.89%   97.82%   -0.07%     
==========================================
  Files          31       32       +1     
  Lines       12228    12435     +207     
==========================================
+ Hits        11970    12164     +194     
- Misses        258      271      +13     
Impacted Files Coverage Δ
src/lib.rs 100.00% <ø> (ø)
src/cache.rs 93.51% <93.51%> (ø)
src/engine.rs 98.54% <95.08%> (-0.10%) ⬇️
src/config.rs 98.09% <100.00%> (+<0.01%) ⬆️
src/filter.rs 82.39% <100.00%> (-0.07%) ⬇️
src/log_batch.rs 98.07% <100.00%> (+<0.01%) ⬆️
src/pipe_log.rs 95.45% <100.00%> (ø)
src/purge.rs 97.46% <100.00%> (+0.01%) ⬆️

... and 2 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
@tonyxuqqi tonyxuqqi requested review from tabokie and hicqu April 25, 2023 06:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant