Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot produce message to those topics without setting topic policies, when enable preciseTopicPublishRateLimiterEnable #10603

Closed
gaozhangmin opened this issue May 17, 2021 · 4 comments · Fixed by #10384 or #11442
Assignees
Labels
area/broker type/bug The PR fixed a bug or issue reported a bug
Milestone

Comments

@gaozhangmin
Copy link
Contributor

gaozhangmin commented May 17, 2021

Describe the bug
Topics without setting topic level publishRate and namespace level publishRate(without setting policies or rate <= 0), cannot produce message to it. when setting:
preciseTopicPublishRateLimiterEnable=true. I found those topics are throttled by default.

to reproduce
enable
preciseTopicPublishRateLimiterEnable=true
systemTopicEnabled=true
topicLevelPoliciesEnabled=true

create a topic without setting topic level policies and namespace level policies:
bin/pulsar-perf produce -o 10000 -p 100000 -s 1024 -r 1000 -bm 1 1p

throttling should be disabled by default when topics level policies are not set.

@lhotari
Copy link
Member

lhotari commented May 17, 2021

several problems are fixed in draft PR #10384 .

@gaozhangmin
Copy link
Contributor Author

gaozhangmin commented May 17, 2021

this issue is caused by:

when topic and namespace publish rate are not set or its value <=0, updatePublishDispatcher would return PublishRateLimiter.DISABLED_RATE_LIMITER. which caused
@Override public boolean isTopicPublishRateExceeded(int numberMessages, int bytes) { return preciseTopicPublishRateLimitingEnable && !this.topicPublishRateLimiter.tryAcquire(numberMessages, bytes); }
always true.

@lhotari
Copy link
Member

lhotari commented May 17, 2021

this issue is caused by:

when topic and namespace publish rate are not set or its value <=0, updatePublishDispatcher would return PublishRateLimiter.DISABLED_RATE_LIMITER. which caused
@Override public boolean isTopicPublishRateExceeded(int numberMessages, int bytes) { return preciseTopicPublishRateLimitingEnable && !this.topicPublishRateLimiter.tryAcquire(numberMessages, bytes); }
always true.

@gaozhangmin Thanks for pointing that out. Yes, it's a clear issue. I'll include the fix to #10384 since it fixes various other preciseTopicPublishRate limiter issues and those fixes are needed for using the feature.

lhotari added a commit to lhotari/pulsar that referenced this issue May 17, 2021
@lhotari
Copy link
Member

lhotari commented May 17, 2021

I added a fix to #10384

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/broker type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
3 participants