-
Notifications
You must be signed in to change notification settings - Fork 523
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(streaming): adapt backfill rate limit according to barrier latency #16678
base: main
Are you sure you want to change the base?
Conversation
a407270
to
07e0d11
Compare
The default adaptive rate limit strategy doesn't work well when I test it with the high amplification case.
Then we end up having too high a rate limit due to 1 and 2. At that point, when adaptive rate limit kicks in, it's too slow as well, since its barrier latency is high (as rate limit is too high). |
acc0499
to
f4b560f
Compare
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
wip.
We have a few test scenarios:
Amplification is medium x100
Backpressure can handle this case well. Adaptive Rate Limit does not have any discernible effect.
Amplification is extreme, e.g. x500
In this case, adaptive rate limit will not be able to keep barrier latency low.
Since barrier latency is a lagging metric, it means that rate limit will only be adjusted much later, and will take a while for it to take effect. In such a scenario, barrier latency will remain high.
Checklist
./risedev check
(or alias,./risedev c
)Documentation
Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.