Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix flaky test testReacquireLocksAfterSessionLost #11815

Conversation

hangc0276
Copy link
Contributor

Motivation

Then run ZKSessionTest#testReacquireLocksAfterSessionLost, it will fail occasionally.

Error:  testReacquireLocksAfterSessionLost(org.apache.pulsar.metadata.ZKSessionTest)  Time elapsed: 12.974 s  <<< FAILURE!
org.awaitility.core.ConditionTimeoutException: Assertion condition defined as a lambda expression in org.apache.pulsar.metadata.ZKSessionTest that uses org.apache.pulsar.metadata.api.coordination.ResourceLock expected [false] but found [true] within 10 seconds.
  at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:165)
  at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
  at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)
  at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:895)
  at org.awaitility.core.ConditionFactory.untilAsserted(ConditionFactory.java:679)
  at org.apache.pulsar.metadata.ZKSessionTest.testReacquireLocksAfterSessionLost(ZKSessionTest.java:130)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:566)
  at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
  at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
  at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
  at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.AssertionError: expected [false] but found [true]
  at org.testng.Assert.fail(Assert.java:99)
  at org.testng.Assert.failNotEquals(Assert.java:1037)
  at org.testng.Assert.assertFalse(Assert.java:67)
  at org.testng.Assert.assertFalse(Assert.java:77)
  at org.apache.pulsar.metadata.ZKSessionTest.lambda$testReacquireLocksAfterSessionLost$0(ZKSessionTest.java:131)
  at org.awaitility.core.AssertionCondition.lambda$new$0(AssertionCondition.java:53)
  at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:222)
  at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:209)
  ... 4 more

Modification

  1. Change the waiting time from default 10s to 30s.

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@hangc0276 hangc0276 merged commit adf1007 into apache:master Aug 27, 2021
@hangc0276 hangc0276 added this to the 2.9.0 milestone Aug 27, 2021
@lhotari
Copy link
Member

lhotari commented Aug 27, 2021

why does this fix the issue?

The reason I'm asking is that it should not be possible that assertFalse(lock.getLockExpiredFuture().isDone()) becomes false after it has become true. lock.getLockExpiredFuture() is a CompletableFuture and the reference doesn't get switched at any point in the test. If the CompletableFuture is done, it will never become undone.

hangc0276 added a commit that referenced this pull request Aug 27, 2021
### Motivation
Then run ZKSessionTest#testReacquireLocksAfterSessionLost, it will fail occasionally.

Error:  testReacquireLocksAfterSessionLost(org.apache.pulsar.metadata.ZKSessionTest)  Time elapsed: 12.974 s  <<< FAILURE!
org.awaitility.core.ConditionTimeoutException: Assertion condition defined as a lambda expression in org.apache.pulsar.metadata.ZKSessionTest that uses org.apache.pulsar.metadata.api.coordination.ResourceLock expected [false] but found [true] within 10 seconds.
  at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:165)
  at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
  at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)
  at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:895)
  at org.awaitility.core.ConditionFactory.untilAsserted(ConditionFactory.java:679)
  at org.apache.pulsar.metadata.ZKSessionTest.testReacquireLocksAfterSessionLost(ZKSessionTest.java:130)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:566)
  at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
  at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
  at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
  at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.AssertionError: expected [false] but found [true]
  at org.testng.Assert.fail(Assert.java:99)
  at org.testng.Assert.failNotEquals(Assert.java:1037)
  at org.testng.Assert.assertFalse(Assert.java:67)
  at org.testng.Assert.assertFalse(Assert.java:77)
  at org.apache.pulsar.metadata.ZKSessionTest.lambda$testReacquireLocksAfterSessionLost$0(ZKSessionTest.java:131)
  at org.awaitility.core.AssertionCondition.lambda$new$0(AssertionCondition.java:53)
  at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:222)
  at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:209)
  ... 4 more

###  Modification
Change the waiting time from default 10s to 30s.

(cherry picked from commit adf1007)
@hangc0276 hangc0276 added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label Aug 27, 2021
@merlimat
Copy link
Contributor

merlimat commented Sep 1, 2021

Yes, the error is still happening and as @lhotari mentioned, we shouldn't even need to use Awaitily here: we're testing that the future is not completed, so if the assertion fails, it will never be able to get fixed later.

bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this pull request Mar 18, 2022
### Motivation
Then run ZKSessionTest#testReacquireLocksAfterSessionLost, it will fail occasionally.

Error:  testReacquireLocksAfterSessionLost(org.apache.pulsar.metadata.ZKSessionTest)  Time elapsed: 12.974 s  <<< FAILURE!
org.awaitility.core.ConditionTimeoutException: Assertion condition defined as a lambda expression in org.apache.pulsar.metadata.ZKSessionTest that uses org.apache.pulsar.metadata.api.coordination.ResourceLock expected [false] but found [true] within 10 seconds.
  at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:165)
  at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
  at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)
  at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:895)
  at org.awaitility.core.ConditionFactory.untilAsserted(ConditionFactory.java:679)
  at org.apache.pulsar.metadata.ZKSessionTest.testReacquireLocksAfterSessionLost(ZKSessionTest.java:130)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:566)
  at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
  at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
  at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
  at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.AssertionError: expected [false] but found [true]
  at org.testng.Assert.fail(Assert.java:99)
  at org.testng.Assert.failNotEquals(Assert.java:1037)
  at org.testng.Assert.assertFalse(Assert.java:67)
  at org.testng.Assert.assertFalse(Assert.java:77)
  at org.apache.pulsar.metadata.ZKSessionTest.lambda$testReacquireLocksAfterSessionLost$0(ZKSessionTest.java:131)
  at org.awaitility.core.AssertionCondition.lambda$new$0(AssertionCondition.java:53)
  at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:222)
  at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:209)
  ... 4 more

###  Modification
Change the waiting time from default 10s to 30s.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants