Instrumentation tests do not run with service registered junit5 extensions

[yschimke]

See this PR

When running existing working JVM tests on Android also. I need to add a compile time dependency on the library providing and registering the extension

https://github.com/square/okhttp/pull/6420/files#diff-4e370c937fb7a9a796291fe5a460c222a5fa57f5e400aa48c5eef8a7d22d7ec7

Then explicitly refer to it

https://github.com/square/okhttp/pull/6420/files#diff-83a70a0ad7b461e83feaa5a3e9412f76257a3fdde5a6a2f603ab142a13da806a

[yschimke]

I might just need to set junit.jupiter.extensions.autodetection.enabled, I'll reopen if something more is required.

[yschimke]

Looking at https://github.com/mannodermaus/android-junit5/blob/master/plugin/android-junit5/src/main/kotlin/de/mannodermaus/gradle/plugins/junit5/Plugin.kt#L80

I tried without luck

android {
...
    testOptions {
      junitPlatform {
        configurationParameter 'junit.jupiter.extensions.autodetection.enabled', 'true'
      }
    }
  }

[mannodermaus]

Thanks for bringing this up. The bridge API for running JUnit 5 tests using a JUnit 4-compatible Runner (which is how the instrumentation test support library works currently) may not be able to pick up extensions automatically. The corresponding section in their User Guide doesn't say anything about extensions being unsupported, but that might have been an oversight.

[yschimke]

Feel free to close if you don't intend to support. It's frustrating but there is a verbose workaround.

[mannodermaus]

I'll leave it open for now and will check in with the JUnit Team to sync on the intended support for extensions using the old Runner. Will update here as new info comes in!

[mannodermaus]

Looking into this again. Since the JUnit 5 RunnerBuilder creates its own discovery request from scratch, it will definitely be possible to send custom configuration parameters to instrumentation tests, too. One annoying part about this is that, until this point, I have not figured out a way to set testInstrumentationRunnerArguments from a Gradle plugin yet - any changes to this map I tried to make won't be reflected in the final run. This means that we may have to introduce a way to explicitly set the configuration parameters to the runner, duplicating the DSL statements used for plain unit tests.

Either way, I'm targeting this for the next release.

[yschimke]

Is there a change I should make to the discovery request I'm building here? https://github.com/square/okhttp/blob/master/native-image-tests/src/main/kotlin/okhttp3/RunTests.kt#L109-L116

Before or after your next release?

Thanks for your help on this.

[mannodermaus]

The change would only affect the on-device tests running with the JUnit4/5 hybrid. Not sure about the environment that your RunTests file executes in, but I don't think anything would be needed there. Let me rephrase the work that'll happen here:

Starting from the next version, you'll be able to apply configuration parameters (including the one for auto-detection of extensions) to the instrumentation test runtime through the existing DSL in the build script. This will allow androidTest code to pick up configuration parameters automatically.

android {
  defaultConfig {
    // This will become possible in 1.7.0.0.
    // The value is a comma-separated list, if you need to set more than one
    testInstrumentationRunnerArgument("configurationParameters", "junit.jupiter.extensions.autodetection.enabled=true")
  }
}

The video below exercises a simple instrumentation test with an Int parameter, resolved by this extension:

class ExampleExtension : ParameterResolver {
  override fun supportsParameter(parameterContext: ParameterContext, extensionContext: ExtensionContext): Boolean =
      parameterContext.parameter.type == Int::class.java

  override fun resolveParameter(parameterContext: ParameterContext, extensionContext: ExtensionContext): Any =
      1000
}

class ExampleExtensionTest {
  @Test
  fun test(value: Int) {
    assertEquals(1000, value)
  }
}

The test passes once the extension is auto-detected at runtime.

---

I will provide a 1.7.0.0-SNAPSHOT including this feature within today, so you can try it out for your use case in MockWebServer. I wanted to test it myself but it seems like the mockwebserver3-junit5 artifact isn't available on JCenter yet.

[mannodermaus]

This is now released in 1.7.0.0. Please let me know if you have any trouble getting this to work with the new API