Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix flaky test testUpdateDynamicLocalConfiguration #11115

Conversation

hangc0276
Copy link
Contributor

Motivation

When running test, it fails sporadically.

example failure

Error:  Tests run: 73, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: 922.345 s <<< FAILURE! - in org.apache.pulsar.broker.admin.AdminApiTest
Error:  testUpdateDynamicLocalConfiguration(org.apache.pulsar.broker.admin.AdminApiTest)  Time elapsed: 10.05 s  <<< FAILURE!
org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in org.apache.pulsar.broker.admin.AdminApiTest was not fulfilled within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:165)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:895)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:864)
	at org.apache.pulsar.broker.admin.AdminApiTest.testUpdateDynamicLocalConfiguration(AdminApiTest.java:626)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

#11059 fix this test failed with Awaitility.await().until to wait for zk sync, however the default wait timeout is 10s, if zk sync reaches 10s, it will also run fail.

Modification

  1. simplify check logic and increase wait timeout to 30s.

// verify value is updated
assertEquals(pulsar.getConfiguration().getBrokerShutdownTimeoutMs(), shutdownTime);
Awaitility.waitAtMost(30, TimeUnit.SECONDS).untilAsserted(() -> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does the delay of zk watch exceed 10 seconds?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have logs?

30 seconds is very much.

And also we are using some MockZookeeper here probably

// verify value is updated
assertEquals(pulsar.getConfiguration().getBrokerShutdownTimeoutMs(), shutdownTime);
Awaitility.waitAtMost(30, TimeUnit.SECONDS).untilAsserted(() -> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have logs?

30 seconds is very much.

And also we are using some MockZookeeper here probably

@codelipenghui codelipenghui added this to the 2.9.0 milestone Jun 29, 2021
@sijie sijie merged commit 9a66cbf into apache:master Jul 22, 2021
Technoboy- pushed a commit to Technoboy-/pulsar that referenced this pull request Jul 22, 2021
### Motivation
When running test, it fails sporadically.

[example failure](https://github.com/apache/pulsar/pull/11114/checks?check_run_id=2921358630)
```
Error:  Tests run: 73, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: 922.345 s <<< FAILURE! - in org.apache.pulsar.broker.admin.AdminApiTest
Error:  testUpdateDynamicLocalConfiguration(org.apache.pulsar.broker.admin.AdminApiTest)  Time elapsed: 10.05 s  <<< FAILURE!
org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in org.apache.pulsar.broker.admin.AdminApiTest was not fulfilled within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:165)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:895)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:864)
	at org.apache.pulsar.broker.admin.AdminApiTest.testUpdateDynamicLocalConfiguration(AdminApiTest.java:626)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
```

apache#11059  fix this test failed with `Awaitility.await().until` to wait for zk sync, however the default wait timeout is 10s, if zk sync reaches 10s, it will also run fail.

### Modification
1. simplify check logic and increase wait timeout to 30s.
codelipenghui pushed a commit that referenced this pull request Jul 23, 2021
When running test, it fails sporadically.

[example failure](https://github.com/apache/pulsar/pull/11114/checks?check_run_id=2921358630)
```
Error:  Tests run: 73, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: 922.345 s <<< FAILURE! - in org.apache.pulsar.broker.admin.AdminApiTest
Error:  testUpdateDynamicLocalConfiguration(org.apache.pulsar.broker.admin.AdminApiTest)  Time elapsed: 10.05 s  <<< FAILURE!
org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in org.apache.pulsar.broker.admin.AdminApiTest was not fulfilled within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:165)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:895)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:864)
	at org.apache.pulsar.broker.admin.AdminApiTest.testUpdateDynamicLocalConfiguration(AdminApiTest.java:626)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
```

1. simplify check logic and increase wait timeout to 30s.

(cherry picked from commit 9a66cbf)
@codelipenghui codelipenghui added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label Jul 23, 2021
bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this pull request Mar 18, 2022
### Motivation
When running test, it fails sporadically.

[example failure](https://github.com/apache/pulsar/pull/11114/checks?check_run_id=2921358630)
```
Error:  Tests run: 73, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: 922.345 s <<< FAILURE! - in org.apache.pulsar.broker.admin.AdminApiTest
Error:  testUpdateDynamicLocalConfiguration(org.apache.pulsar.broker.admin.AdminApiTest)  Time elapsed: 10.05 s  <<< FAILURE!
org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in org.apache.pulsar.broker.admin.AdminApiTest was not fulfilled within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:165)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:895)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:864)
	at org.apache.pulsar.broker.admin.AdminApiTest.testUpdateDynamicLocalConfiguration(AdminApiTest.java:626)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
```

apache#11059  fix this test failed with `Awaitility.await().until` to wait for zk sync, however the default wait timeout is 10s, if zk sync reaches 10s, it will also run fail.

### Modification
1. simplify check logic and increase wait timeout to 30s.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants