Fixes #2648 : Add support for customising strictness via @Mock annotation and MockSettings #2650

sivaprasadreddy · 2022-05-23T04:18:41Z

This PR fixes the issue #2648.

This is to add support for customising strictness level for mock using @mock annotation or MockSettings.

Example 1:

class MyTest {
     @Mock(strictness = Strictness.LENIENT)
     Foo foo;
     ....
}

Example 2:

class MyTest {
     Foo mock = Mockito.mock(Foo.class, withSettings().strictness(Strictness.WARN));
     ....
}

…tion and MockSettings

…epresentation

codecov-commenter · 2022-05-26T09:05:25Z

Codecov Report

Merging #2650 (ff8a4ee) into main (d7a8ae0) will decrease coverage by 0.03%.
The diff coverage is 75.00%.

❗ Current head ff8a4ee differs from pull request most recent head d090780. Consider uploading reports for the commit d090780 to get more accurate results

@@             Coverage Diff              @@
##               main    #2650      +/-   ##
============================================
- Coverage     86.24%   86.21%   -0.04%     
- Complexity     2761     2762       +1     
============================================
  Files           314      314              
  Lines          8282     8290       +8     
  Branches       1031     1031              
============================================
+ Hits           7143     7147       +4     
- Misses          872      874       +2     
- Partials        267      269       +2

Impacted Files	Coverage Δ
src/main/java/org/mockito/Mockito.java	`96.10% <ø> (ø)`
...java/org/mockito/internal/exceptions/Reporter.java	`90.47% <0.00%> (-0.29%)`	⬇️
...rg/mockito/internal/creation/MockSettingsImpl.java	`86.13% <60.00%> (-1.50%)`	⬇️
...nternal/configuration/MockAnnotationProcessor.java	`97.05% <100.00%> (+0.08%)`	⬆️
...o/internal/creation/settings/CreationSettings.java	`100.00% <100.00%> (ø)`
...ito/internal/stubbing/InvocationContainerImpl.java	`94.20% <100.00%> (ø)`
.../mockito/internal/stubbing/StrictnessSelector.java	`80.00% <0.00%> (-20.00%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d7a8ae0...d090780. Read the comment docs.

TimvdLippe

Nice work! It's already looking good, mostly some nits. After that, this is ready to merge 🎉

TimvdLippe · 2022-05-26T13:08:21Z

src/main/java/org/mockito/Mock.java

+     * Mock will have custom strictness, see {@link MockSettings#strictness(Strictness)}.
+     * For examples how to use 'Mock' annotation and parameters see {@link Mock}.
+     *
+     * @since 4.5.2


Nit: @since 4.6.0 since we will include this in the next minor release

TimvdLippe · 2022-05-26T13:09:50Z

src/main/java/org/mockito/MockSettings.java

+
+    /**
+     * Specifies strictness level for the mock.
+     * The default strictness level is STRICT_STUBS


Nit: the default is the default of the runner. If you use the lenient runner, it will be lenient. I suggest to formulate it the following way:

The default strictness level is determined by the rule/runner used. If you are using no rule/runner, the default strictness level is LENIENT.

src/main/java/org/mockito/internal/creation/settings/CreationSettings.java

TimvdLippe · 2022-05-26T13:11:33Z

src/main/java/org/mockito/mock/MockCreationSettings.java

+     * Sets strictness level for the mock, e.g. having {@link Strictness#STRICT_STUBS} characteristic.
+     * For more information about using mocks with custom strictness, see {@link MockSettings#strictness(Strictness)}.
+     *
+     * @since 4.5.2


src/main/java/org/mockito/internal/creation/MockSettingsImpl.java

src/main/java/org/mockito/Mock.java

TimvdLippe · 2022-05-26T13:15:37Z

src/main/java/org/mockito/Mockito.java

+ * <h3 id="52">52. <a class="meaningful_link" href="#mockito_strictness" name="mockito_strictness">
+ *  New strictness attribute for @Mock annotation and <code>MockSettings.strictness()</code> methods (Since 4.5.2)</a></h3>
+ *
+ * By default Mocks use Strict stubbing and now you can customize the strictness level


Nit: mocks by default are lenient. I would omit that part altogether and reword it to:

You can now customize the strictness level for a single mock, either using `@Mock` annotation strictness attribute or using `MockSettings.strictness()`. This can be useful if you want all of your mocks to be strict, but one of the mocks to be lenient.

src/test/java/org/mockitousage/strictness/StrictnessMockAnnotationTest.java

TimvdLippe · 2022-05-26T13:17:18Z

src/test/java/org/mockitousage/strictness/StrictnessMockAnnotationTest.java

+
+        // but regular mock throws:
+        Assertions.assertThatThrownBy(
+                        new ThrowableAssert.ThrowingCallable() {


Nit: I think we can use a lambda here, can we not?

() -> ProductionCode.simpleMethod(regularMock, "4")

TimvdLippe · 2022-05-26T13:17:30Z

src/test/java/org/mockitousage/strictness/StrictnessWithSettingsTest.java

+        when(regularMock.simpleMethod("3")).thenReturn("3");
+        when(strictMock.simpleMethod("2")).thenReturn("2");
+
+        // then lenient mock does not throw:


Same here, let's have 3 separate 3 test cases here.

Separated this into multiple tests.

TimvdLippe

Thanks for the PR and great implementation 🎉

chadlwilson · 2022-05-30T06:06:00Z

src/test/java/org/mockitousage/strictness/StrictnessWithSettingsTest.java

+    }
+
+    @Test
+    public void mock_is_strict_with_default_settings() {


Quick Q - was it intentional for this to change mocks defined by mock(Blah.class) i.e with no rule/Junit MockitoSettings to strict by default?

I believe the previous behaviour was lenient by default?

We have not changed the behavior here. This test is using a Strict rule explicitly (https://github.com/mockito/mockito/pull/2650/files/d0907808e9fce24f6dcddc94d1ea2c3ebfa15676#diff-3040268f6b18ef562dd495ac7fc13465723311889d5f326bfb615cc3aed37415R21) to verify that we handle that case correctly.

In case you are hitting any regressions, please file a new issue. You are most likely running into a different problem.

OK, thanks. We definitely have regressions (previously lenient mocks now behaving as strict) but I may have misidentified the possible cause. Will need to dig further if I get time.

Another person has beat me to raising it at #2656 but this seems to be the same issue we are having.

@mock

## Changes - **mockito: 4.4.0 -> 4.6.1** (https://github.com/mockito/mockito/releases) Most important updates: - Fixes mockito/mockito#2648 : Add support for customising strictness via @mock annotation and MockSettings mockito/mockito#2650 ## Why is this change needed? According to the [Mockito documentation](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html#when(T)) : > Although it is possible to verify a stubbed invocation, usually it's just redundant. Let's say you've stubbed foo.bar(). If your code cares what foo.bar() returns then something else breaks(often before even verify() gets executed). If your code doesn't care what get(0) returns then it should not be stubbed. While working on the [Replace EasyMock and PowerMock with Mockito for StreamsMetricsImplTest ](https://issues.apache.org/jira/browse/KAFKA-12947) I noticed that described behavior wasn't applied when you create a new `mock` like this. ```java final Metrics metrics = mock(Metrics.class); when(metrics.metric(metricName)).thenReturn(null); ... invoke SUT verify(metrics).metric(metricName); // this should be redundant (according to docs) ``` After further investigation I figured out that described behaviour wasn't implemented until`v4.6.1`. With this change we are now able to mock objects like this: ```java Foo explicitStrictMock = mock(Foo.class, withSettings().strictness(Strictness.STRICT_STUBS)); ``` - link to docs: [MockSettings.html#strictness](https://javadoc.io/static/org.mockito/mockito-core/4.6.1/org/mockito/quality/Strictness.html#STRICT_STUBS) It looks like I can accomplish the same thing by using the `@RunWith(MockitoJUnitRunner.StrictStubs.class) ` instead of the `@RunWith(MockitoJUnitRunner.class)` so mockito dependency version update is not mandatory, but it would be nice to stay up-to-date and use the latest version (it's up to MR reviewer to decide if we are going to merge this now, or just close the MR and update mockito version later). Reviewers: Ismael Juma <ismael@juma.me.uk>

@mock

## Changes - **mockito: 4.4.0 -> 4.6.1** (https://github.com/mockito/mockito/releases) Most important updates: - Fixes mockito/mockito#2648 : Add support for customising strictness via @mock annotation and MockSettings mockito/mockito#2650 ## Why is this change needed? According to the [Mockito documentation](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html#when(T)) : > Although it is possible to verify a stubbed invocation, usually it's just redundant. Let's say you've stubbed foo.bar(). If your code cares what foo.bar() returns then something else breaks(often before even verify() gets executed). If your code doesn't care what get(0) returns then it should not be stubbed. While working on the [Replace EasyMock and PowerMock with Mockito for StreamsMetricsImplTest ](https://issues.apache.org/jira/browse/KAFKA-12947) I noticed that described behavior wasn't applied when you create a new `mock` like this. ```java final Metrics metrics = mock(Metrics.class); when(metrics.metric(metricName)).thenReturn(null); ... invoke SUT verify(metrics).metric(metricName); // this should be redundant (according to docs) ``` After further investigation I figured out that described behaviour wasn't implemented until`v4.6.1`. With this change we are now able to mock objects like this: ```java Foo explicitStrictMock = mock(Foo.class, withSettings().strictness(Strictness.STRICT_STUBS)); ``` - link to docs: [MockSettings.html#strictness](https://javadoc.io/static/org.mockito/mockito-core/4.6.1/org/mockito/quality/Strictness.html#STRICT_STUBS) It looks like I can accomplish the same thing by using the `@RunWith(MockitoJUnitRunner.StrictStubs.class) ` instead of the `@RunWith(MockitoJUnitRunner.class)` so mockito dependency version update is not mandatory, but it would be nice to stay up-to-date and use the latest version (it's up to MR reviewer to decide if we are going to merge this now, or just close the MR and update mockito version later). Reviewers: Ismael Juma <ismael@juma.me.uk>

Fixes #2648 : Add support for customizing strictness via @mock annota…

2e0390a

…tion and MockSettings

sivaprasadreddy mentioned this pull request May 24, 2022

Any way to enable Strict Stubbing when using Mockito.mock() without using @Mock? #2648

Closed

sivaprasadreddy added 2 commits May 26, 2022 05:53

Fixes #2648 : Update Javadoc comment to escape @ symbol

5835229

Fixes #2648 : Update Javadoc comment to escape @ symbol using octal r…

ff8a4ee

…epresentation

TimvdLippe requested changes May 26, 2022

View reviewed changes

sivaprasadreddy added 2 commits May 26, 2022 21:37

Fixes #2648 : Updated addressing review comments.

1ffc0f0

Fixes #2648 : Applying google java formatting.

d090780

TimvdLippe approved these changes May 26, 2022

View reviewed changes

TimvdLippe merged commit 70cf2d2 into mockito:main May 26, 2022

TimvdLippe mentioned this pull request May 26, 2022

Added support for updating releases shipkit/shipkit-changelog#124

Merged

chadlwilson reviewed May 30, 2022

View reviewed changes

chadlwilson mentioned this pull request May 30, 2022

Bump mockito-core from 4.5.1 to 4.6.0 gocd/gocd#10479

Closed

picimako mentioned this pull request May 30, 2022

Support strictness for mock creation settings picimako/mockitools#24

Closed

This was referenced May 31, 2022

Regression? Strictness set in @MockitoSettings ignored after upgrade from 4.5.1 to 4.6.0 #2656

Closed

Fix for Issue2656 #2659

Merged

dplavcic mentioned this pull request Jul 29, 2022

MINOR: Upgrade mockito test dependencies apache/kafka#12460

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes #2648 : Add support for customising strictness via @Mock annotation and MockSettings #2650

Fixes #2648 : Add support for customising strictness via @Mock annotation and MockSettings #2650

sivaprasadreddy commented May 23, 2022

codecov-commenter commented May 26, 2022 •

edited

TimvdLippe left a comment

TimvdLippe May 26, 2022

sivaprasadreddy May 26, 2022

TimvdLippe May 26, 2022

sivaprasadreddy May 26, 2022

TimvdLippe May 26, 2022

sivaprasadreddy May 26, 2022

TimvdLippe May 26, 2022

sivaprasadreddy May 26, 2022

TimvdLippe May 26, 2022

sivaprasadreddy May 26, 2022

TimvdLippe May 26, 2022

sivaprasadreddy May 26, 2022

TimvdLippe left a comment

chadlwilson May 30, 2022 •

edited

TimvdLippe May 30, 2022

chadlwilson May 30, 2022

chadlwilson May 31, 2022

Fixes #2648 : Add support for customising strictness via @Mock annotation and MockSettings #2650

Fixes #2648 : Add support for customising strictness via @Mock annotation and MockSettings #2650

Conversation

sivaprasadreddy commented May 23, 2022

codecov-commenter commented May 26, 2022 • edited

Codecov Report

TimvdLippe left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TimvdLippe left a comment

Choose a reason for hiding this comment

chadlwilson May 30, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented May 26, 2022 •

edited

chadlwilson May 30, 2022 •

edited