Provide JUnit 5 alternative to XtextRunner

[trancexpress]

For our product, we have tests that use XtextRunner, which in turn uses JUnit 4:

public class XtextRunner extends BlockJUnit4ClassRunner {

It would be nice to provide a replacement that uses JUnit 5, so migration from JUnit 4 to JUnit 5 is not hindered by tests for xtext functionality. XtextRunner itself is used to inject classes:

	@Override
	protected Object createTest() throws Exception {
		Object object = super.createTest();
		IInjectorProvider injectorProvider = getOrCreateInjectorProvider();
		if (injectorProvider != null) {
			Injector injector = injectorProvider.getInjector();
			if (injector != null)
				injector.injectMembers(object);
		}
		return object;
	}

	@Override
	protected Statement methodBlock(FrameworkMethod method) {
		IInjectorProvider injectorProvider = getOrCreateInjectorProvider();
		if (injectorProvider instanceof IRegistryConfigurator) {
			IRegistryConfigurator registryConfigurator = (IRegistryConfigurator) injectorProvider;
			registryConfigurator.setupRegistry();
			Statement methodBlock = superMethodBlock(method);
			return new Statement() {
				@Override
				public void evaluate() throws Throwable {
					try {
						methodBlock.evaluate();
					} finally {
						registryConfigurator.restoreRegistry();
					}
				}
			};
		} else {
			return superMethodBlock(method);
		}
	}

[szarnekow]

Did you see https://github.com/eclipse/xtext/blob/main/org.eclipse.xtext.testing/src/org/eclipse/xtext/testing/extensions/InjectionExtension.java and the examples/unit tests here: https://github.com/eclipse/xtext/tree/main/org.eclipse.xtext.testing.junit5.tests/src/org/eclipse/xtext/testing/tests/junit5 ?

[trancexpress]

Did you see https://github.com/eclipse/xtext/blob/main/org.eclipse.xtext.testing/src/org/eclipse/xtext/testing/extensions/InjectionExtension.java and the examples/unit tests here: https://github.com/eclipse/xtext/tree/main/org.eclipse.xtext.testing.junit5.tests/src/org/eclipse/xtext/testing/tests/junit5 ?

No :) Thank you, I'll try using them! I guess this can be closed.

[trancexpress]

@szarnekow , is there also a replacement for org.eclipse.xtext.testing.smoketest.XtextSmokeTestRunner? If not, what are the main things that I should be aware of when writing a JUnit 5 replacement?

[cdietrich]

not to my knowledge.
the 2nd part @szarnekow might have an answer for

[szarnekow]

We didn’t think smoke testing was wildly used, so there is no JUnit5 based smoke test factory available yet. The tricky part was especially the isolation of the model recording injector and the actual language implementation. The extensible architecture of JUnit5 should certainly allow this but I don’t have a blue-print available.

[trancexpress]

@szarnekow , what does the smoke test runner actually do? We have some 650 tests, of these only 4 calls to ScenarioProcessor.processFile(String) are made. There is the class org.eclipse.xtext.testing.smoketest.Scenario, but due to setting processCompleteInput = true the values in Scenario seem to not be used (see constructor of org.eclipse.xtext.testing.smoketest.internal.AllScenariosRunner).

Am I seeing this right, that in our case we run a test suite with some extra validation when an exception of type org.eclipse.xtext.testing.smoketest.internal.TestDataCarrier is thrown? I.e. when some specific loading is done. That would simplify moving to JUnit 5, if nothing else is going on (in our case).

[szarnekow]

The SmokeTestRunner alters the injector used to run the regular test class. As soon as a model is loaded, it's recorded by throwing a TestDataCarrier. Subsequently, additional validation can happen with that model, e.g. usually, the scenarios would create permutations of the tested model to run other services with these permutations. In your case, it's likely skipping the permutations that are available by default in favor of some custom logic.

[trancexpress]

Using InjectionExtension.java works fine. Still in the process of replacing XtextSmokeTestRunner.