Strictness in Mockito #769
Comments
Needed for work on Mockito strictness, see #769. Decoupled JUnit API from the work Mockito does before and after every test. No public API exposed, listeners and friends need to be refactored and documented. Don't merge. This refactoring on its own does not add value.
|
I think from a code maintenance point of view, "behavior: warning on the console" is not really useful - in such a case a CI system would not inform anyone about unused stubs. An exception would be a lot better. |
Needed for work on Mockito strictness, see #769. Decoupled JUnit API from the work Mockito does before and after every test. No public API exposed, listeners and friends need to be refactored and documented. Don't merge. This refactoring on its own does not add value.
Needed for work on Mockito strictness, see #769. Decoupled JUnit API from the work Mockito does before and after every test. No public API exposed, listeners and friends need to be refactored and documented. Don't merge. This refactoring on its own does not add value.
|
@szczepiq could you add list items ( |
I will do that once this epic turns into an execution plan. Currently it is intended to scope out the details and I don't know if all of them will be executed (and in what form). I annotated one item as merged by using strikethrough. I hope it is clearer now. |
|
Since your Javadoc asks for feedback, here's my case: It tests auto-completion of an input field.
Update: |
|
@micheljung, my recommendation for this scenario is exactly using verify + never :) I will assume that your use case is satisfied. Looks like strict stubbing nudged you to improve this test! Thank you very much for reporting. This is exactly kind of feedback we're looking for when evaluating strictness feature. |
|
I would like an option to
I'm simulating an external API (Selenium) to ensure I wrap it properly. It's very easy to accidentally not throw the exception when we're expecting no element; the correct thing selenium consumers generally want to do is verify no elements. Essentially, I can't be DRY with Strict without breaking it's strict "clean and maintainable" expectations. BTW, I agree this is not "clean" but it is certainly very much more maintainable (and correct) than what STRICT mode allows. |
|
So in terms of 8.5.47's implementation, I'd like to have Probably what I want is DRY without strictness as mockito wants to define it. IMO, |
|
Also, having tried for half an hour trying to grok If there's some sort of workaround where I can make this auto-verification automatic, please respond. Should there be another ticket to add auto-verification of stubbed invocations (which is an awesome feature BTW) that's separate from this ticket (as it should be)? |
|
Thank you for reporting and debugging Mockito strictness! Let me give you some feedback about the use case first:
It seems that you are suggesting a different Strictness level, something between "lenient" and "strict stubs":
Have you tried this: verifyNoMoreInteractions(ignoreStubs(mock1, mock2));Before we jump to the impl details of TestListener (which is an internal type, not intended for regular use, but absolutely open to experimentation and creative usage), let's nail down the use case first. Thank you for reporting! This is a very interesting use case. |
|
I like strict stubbing very much. |
|
Can this be exposed by The following "works" but uses a whole lot more internal stuff than I like: import org.mockito.MockSettings;
import org.mockito.Mockito;
import org.mockito.internal.creation.settings.CreationSettings;
import org.mockito.internal.listeners.StubbingLookupListener;
import org.mockito.mock.MockCreationSettings;
import org.mockito.quality.Strictness;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import static org.mockito.Mockito.withSettings;
public final class StrictMockito
{
private StrictMockito()
{
}
public static <T> T strictMock(Class<T> classToMock) {
final MockSettings mockSettings = withSettings();
createDefaultStubbingLookupListener(Strictness.STRICT_STUBS);
final T mock = Mockito.mock(classToMock, mockSettings);
final MockCreationSettings<?> mockCreationSettings = Mockito.mockingDetails(mock).getMockCreationSettings();
((CreationSettings<?>)mockCreationSettings).getStubbingLookupListeners().add( createDefaultStubbingLookupListener(Strictness.STRICT_STUBS));
return mock;
}
private static StubbingLookupListener createDefaultStubbingLookupListener(Strictness strictness)
{
try
{
final Class<?> clazz = Class.forName("org.mockito.internal.junit.DefaultStubbingLookupListener");
final Constructor<?> constructor = clazz.getDeclaredConstructor(Strictness.class);
constructor.setAccessible(true);
return (StubbingLookupListener) constructor.newInstance(strictness);
}
catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | InstantiationException | InvocationTargetException e)
{
throw new RuntimeException(e);
}
}
} |
|
If possible, the failure message around passing null and collections could use improvement. As is, we can get errors like the following: It is not clear from the failure message that only the middle argument is the source of the problem. One is likely to expect the final argument is the problem because it is the entry that is different. Code to reproduce the above: public interface Sut
{
Object getValue(String foo, Object bar, Set<String> baz);
}
@Mock
Sut sut;
@Test
public void strictExample() {
Mockito.when(sut.getValue(ArgumentMatchers.anyString(), ArgumentMatchers.any(Object.class),
ArgumentMatchers.eq(Collections.emptySet()))).thenReturn(null);
sut.getValue("", null, Collections.emptySet());
} |
|
Hey @adamvoss! Thanks for checking in!
Mocks don't have to be class members to leverage strictness. You can use MockitoSession object to start mocking before you create mocks. Would that solve your problem? |
|
@mockitoguy Sorry, I missed the notification on this one. Here is what that looks like: public final class MockTest
{
@Test
public void doTest() {
final MockitoSession session = Mockito.mockitoSession().strictness(Strictness.STRICT_STUBS).startMocking();
final IWhatever mock = Mockito.mock(IWhatever.class);
when(mock.getResult("bar")).thenReturn("yes");
Assert.assertNotNull(mock.getResult("foo"));
// Should be in tear down (after) method
session.finishMocking();
}
}
interface IWhatever {
public String getResult(String foo);
}I would say that is more flexible, but not exactly what I was looking for. One of the motivations being able to specify it on a per-mock setting is to be able to introduce them gradually. That way I can use strict when I want to (including existing tests) without changing any of the existing mocks. With you wanting to make this eventually the default mode, then I can see the argument for not allowing granular configuration. (Though once on v3, people may want to gradually use some lenient mocks). That aside, it is an annoyance to need to setup and tear down a session just to create a mock, but maybe that is unavoidable. Was there any thought to making an even stricter mode? When I was working on a reproduction to demonstrate using public final class MockTest
{
@Test
public void doTest() {
final MockitoSession session = Mockito.mockitoSession().strictness(Strictness.STRICT_STUBS).startMocking();
final IWhatever mock = Mockito.mock(IWhatever.class);
Assert.assertNull(mock.getResult("foo"));
}
}
interface IWhatever {
public String getResult(String foo);
}and public final class MockTest
{
@Test
public void doTest() {
final MockitoSession session = Mockito.mockitoSession().strictness(Strictness.STRICT_STUBS).startMocking();
final IWhatever mock = Mockito.mock(IWhatever.class);
Assert.assertNull(mock.getResult());
}
}
interface IWhatever {
public String getResult();
}In both cases the test will pass when I would want the strict mocks to fail because I didn't configure the methods invoked. |
|
While writing an integration test I came across a specific example where this Strictness may actually be useful. In this example, I want to explicitly mock a call to a service and ensure it is definitely called. I have, for example, the following method: The two things I need to test here are:
I don't extrinsically care whether To test this, I need something like the following: The problem here is that there is no guarantee that
Assert some flag is setObvious problem with this: it requires more code. It's also not as succinct for describing what is happening. Rely on Strict modeThis is the current solution. You called Possible issues:
VerifyThe most natural of the options, I want to explicitly verify that something happened. But for that, it need the following line: Problem: duplicated code. Pretty obvious that that's not ideal. An alternative solutionWhat about instead of changing the API (which is what Strict-mode does; it means you have to change all your tests in some way), the API is expanded with an extra command:
This would solve all the above issues. Non-mocked method calls could still throw an exception for missing stubbings (which does indicate a problem). |
Introduction
This issue explores the addition of "strict" behavior and APIs to Mockito. Traditionally, Mockito is a lenient mocking framework. For background and motivation, check out Szczepan's short article on LinkedIn or in-depth article on Mockito blog.
Why strictness in Mockito:
Concrete API proposals:
Opt-in stubbing strictness implemented in JUnit rules #770Example exception message indicating subbing problem:
Details
Strictness in Mockito can have 2 forms:
Strictness behavior can be implemented as:
Strictness API can be implemented in:
Future direction:
Implementation as of Mockito 2.2.x:
Mockito 2.x proposal:
Opt-in stubbing strictness implemented in JUnit rules #770Mockito 3 proposal:
The text was updated successfully, but these errors were encountered: