VerificationCollector doesn't work for invocations with non-matching args

[kriegfrj]

I found the incubating VerificationCollector (#287), which is similar to AssertJ SoftAssertions in that it collects multiple verifications and then asserts on the whole, rather than throwing an exception on the first one and skipping the rest of the verifications. Very useful for complicated testing.

However, I found that the verification collector doesn't work for verifications on method invocations with parameters. I had a quick check of the source code of the test, and I noticed that all of the tests in VerificationCollectorImplTest use no-arg invocations.

Hopefully this would be an easy fix, because this looks like an extremely useful feature for my application but I need to verify invocations with arguments. At the moment, my workaround is to wrap each verification call in an AssertJ soft assertion, but this makes writing the tests a bit more tedious and less readable.

[kriegfrj]

Update: I found that the lazy verification works properly when you use VerificationCollector directly. However, if you try and use it as a JUnit rule that's when this bug manifests.

I have updated VerificationCollectorImplTest to properly test for this situation and fail as appropriate. I will have a quick look and see if it is an easy fix; otherwise I may submit a PR with the test only and leave a more experienced hand to fix it.

[kriegfrj]

Another update: I have gotten to the root of this problem.

I found that the lazy verification works properly when you use VerificationCollector directly. However, if you try and use it as a JUnit rule that's when this bug manifests.

This wasn't quite correct. The distinguishing feature wasn't the JUnit rule, but if you do a verification for an invocation that tries to match on an argument while running under JUnit.

There is code in org.mockito.internal.junit.ExceptionFactory that checks to see if JUnit is on the classpath - if so, then it will throw org.mockito.exceptions.verification.junit.ArgumentsAreDifferent rather than org.mockito.exceptions.verification.ArgumentsAreDifferent. The JUnit version inherits from junit.framework.ComparisonFailure whereas the basic version inherits from org.mockito.exceptions.base.MockitoAssertionError.

It seems that the purpose of this is to help JUnit-aware GUIs to produce more helpful diagnostic output. This makes sense (although it would make more sense to derive from a more-up-to-date version of ComparisonFailure if it is available, like org.junit.ComparisonFailure - perhaps this could be the subject of a different issue/PR). However, because it is not a subclass of MockitoAssertionError, it skips out of the try/catch in VerificationCollectorImpl (see

mockito/src/main/java/org/mockito/internal/junit/VerificationCollectorImpl.java

Line 92 in b745b5d

} catch (MockitoAssertionError error) {

).

There are two possible solutions:

1. Add another catch() to catch ComparisonFailure as well.
2. Broaden the catch for MockitoAssertionError to a common superclass (like AssertionError).

The second probably has more generic appeal and would future-proof the solution if other types of argument comparison were used (eg, a custom argument matcher). I will submit a PR for this shortly.