Mocked function not called?

[bartkummel]

Prerequisites

• I am running the latest version
• I checked the documentation and found no answer
• I checked to make sure that this issue has not already been filed

Expected Behavior

Test succeeds

Current Behavior

Test fails

Failure Information (for bugs)

My test fails because the mocked function is never called:

my.package.kafka.TestResourceEventsKafkaListener > Resiliency scenarios > my.package.kafka.TestResourceEventsKafkaListener.A failing second message should retry only the failed message FAILED
    java.lang.AssertionError: Eventually block failed after 10s; attempted 342 time(s); FixedInterval(duration=25ms) delay between attempts
    The first error was caused by: Verification failed: call 1 of 1: ResourceService(#932).updateResource(matcher<ResourceId>(), any(), any())) was not called

Steps to Reproduce

I have the following test:

            "A failing second message should retry only the failed message" {
                every {
                    resourceService.updateResource(ResourceId(someID, "AAAA_1_SUCCEEDS"), any(), any())
                } returns UUID.randomUUID()
                publishKafkaMessage("AAAA_1_SUCCEEDS")

                every {
                    resourceService.updateResource(ResourceId(someID, "AAAA_1_FAILS_AND_THEN_SUCCEEDS"), any(), any())
                } throws RuntimeException(":boom:") andThen UUID.randomUUID()
                publishKafkaMessage("AAAA_1_FAILS_AND_THEN_SUCCEEDS")

                eventually(Duration.seconds(10)) {
                    coVerify(exactly = 1) { // this messages won't be retried when only the next one fails
                        resourceService.updateResource(withArg { it.localId shouldBe "AAAA_1_SUCCEEDS" }, any(), any())
                    }
                    coVerify(exactly = 2) { // this message will be retried once
                        resourceService.updateResource(withArg { it.localId shouldBe "AAAA_1_FAILS_AND_THEN_SUCCEEDS" }, any(), any())
                    }
                }
            }

It succeeds when running with Mockk 1.10.6 up and including 1.12.2. But it fails when running with Mockk 1.12.3. I can't figure out what change might have caused this.

Context

I suppose it's not needed to know the rest of my implementation. resourceService is a service that does some database updates and is mocked in this test. publishKafkaMessage is a helper function that publishes a message to Kafka. This is an integration test and we use testContainers to spin up a "real" Kafka cluster. The code under test here is listening for events from Kafka, and will be triggered by the messages that are published by this helper function.

• MockK version: 1.12.3. (I've verified that this occurs in 1.12.3 and NOT in 1.10.6, 1.11.0, 1.12.0, 1.12.1 and 1.12.2.)
• OS: Fails both locally on macOS and on the CI, running Linux.
• Kotlin version: 1.5.31
• JDK version: openjdk 14.0.2 2020-07-14
• JUnit Kotest version: 4.6.4
• Type of test: unit test

Failure Logs

my.package.kafka.TestResourceEventsKafkaListener > Resiliency scenarios > my.package.kafka.TestResourceEventsKafkaListener.A failing second message should retry only the failed message FAILED
    java.lang.AssertionError: Eventually block failed after 10s; attempted 342 time(s); FixedInterval(duration=25ms) delay between attempts
    The first error was caused by: Verification failed: call 1 of 1: ResourceService(#932).updateResource(matcher<ResourceId>(), any(), any())) was not called

Stack trace

N/A, the test fails as if the mocked function was not called.

Minimal reproducible code (the gist of this issue)

// -----------------------[ GRADLE DEFINITIONS ] -----------------------
dependencies {
    implementation("org.apache.kafka:kafka-clients:2.7.0")
    implementation("io.confluent:kafka-json-serializer:6.2.3")
    testImplementation("io.grpc:grpc-testing:$grpcVersion")
    testImplementation("io.kotest:kotest-assertions-core-jvm:4.6.4")
    testImplementation("io.kotest:kotest-assertions-json:4.6.4")
    testImplementation("io.kotest:kotest-runner-junit5-jvm:4.6.4")
    testImplementation("io.kotest:kotest-framework-datatest:4.6.4")
    testImplementation("io.mockk:mockk:1.12.3")
    testImplementation("org.testcontainers:kafka:1.16.2")
}
// -----------------------[ YOUR CODE STARTS HERE ] -----------------------
@Message("resource_created", "3")
data class ResourceCreatedV3 constructor(
    override val resourceId: String,
    val resourceUuid: UUID,
    val market: MarketId,
    val owner: AccountId?,
) : ResourceMessage {

    @JsonCreator private constructor(
        @JsonProperty("resource_id") resourceId: String,
        @JsonProperty("resource_uuid") resourceUuid: UUID,
        @JsonProperty("market") market: MarketId,
        @JsonProperty("owner") owner: AccountId?
    ) : this(
        resourceId, resourceUuid, market, owner
    )
}

        "Resiliency scenarios" - {
            fun publishKafkaMessage(resourceId: String) {
                client.send(
                    ProducerRecord(
                        "topic",
                        Envelope.forMessage(
                            ResourceCreatedV3(
                                resourceId = resourceId,
                                resourceUuid = UUID(0, 1),
                                market = blah,
                                owner = null,
                            )
                        )
                    ),

                ).get()
            }

            "A failing second message should retry only the failed message" {
                every {
                    resourceService.updateResource(ResourceId(someID, "AAAA_1_SUCCEEDS"), any(), any())
                } returns UUID.randomUUID()
                publishKafkaMessage("AAAA_1_SUCCEEDS")

                every {
                    resourceService.updateResource(ResourceId(someID, "AAAA_1_FAILS_AND_THEN_SUCCEEDS"), any(), any())
                } throws RuntimeException(":boom:") andThen UUID.randomUUID()
                publishKafkaMessage("AAAA_1_FAILS_AND_THEN_SUCCEEDS")

                eventually(Duration.seconds(10)) {
                    coVerify(exactly = 1) { // this messages won't be retried when only the next one fails
                        resourceService.updateResource(withArg { it.localId shouldBe "AAAA_1_SUCCEEDS" }, any(), any())
                    }
                    coVerify(exactly = 2) { // this message will be retried once
                        resourceService.updateResource(withArg { it.localId shouldBe "AAAA_1_FAILS_AND_THEN_SUCCEEDS" }, any(), any())
                    }
                }
            }
// -----------------------[ YOUR CODE ENDS HERE ] -----------------------

[Raibaz]

Since you are using coVerify, i'm assuming your test involves coroutines.

Just wondering, does it work if you use the timeout parameter for coVerify rather then eventually?

[bartkummel]

Thanks for your suggestion, @Raibaz !

I've tried your approach, but it doesn't work either. In the process of trying, I discovered that the issue is actually only in version 1.12.3, and not in versions 1.11.0 to 1.12.2, as I claimed before. (I don't know how that happened, I probably didn't do me initial reproduction efforts totally right.) I'll correct the initial issue report.

But, to summarize: your approach does work in versions where the eventually approach also does work. And where the eventually approach fails (version 1.12.3), the approach using timeout also fails.

[Raibaz]

Right, I just noticed that you are using multiple withArg calls in the same coVerify block, so this behavior was probably introduced by #776.

I just merged #792 that reverts it, so the regression should likely be removed in the next release.

[bartkummel]

Excellent. I'll update when the release is out.

[Raibaz]

Just released v1.12.4, closing this.