Enable default TestExecutionListeners in JUnit 4 and TestNG base test classes

[oopschen]

As spring test framework support testExecutionListener automatic discovery, when i build project using spring security with testng, it can not load META-INF/spring.factories file in org.springframework.security:spring-security-test library.(Same as Junit4, but works for junit5).

May affected spring version

since 4.1

After searching the source code of spring test framework, below is how it happened:

spring test framework testExecutionListener automatic discovery

Please check file org.springframework.test.context.support.AbstractTestContextBootstrapper at method getTestExecutionListeners.
The method prefer to load @TestExecutionListeners over META-INF/spring.factories , only if no @TestExecutionListeners annotation found, it loads default execution listeners(META-INF/spring.factories ).

IMPORTANT CODE FROM SPRING FRAMEWORK

                                // If there are no listeners to inherit, we might need to merge the
				// locally declared listeners with the defaults.
				if ((!inheritListeners || parentDescriptor == null) &&
						testExecutionListeners.mergeMode() == MergeMode.MERGE_WITH_DEFAULTS) {
					if (logger.isDebugEnabled()) {
						logger.debug(String.format("Merging default listeners with listeners configured via " +
								"@TestExecutionListeners for class [%s].", descriptor.getRootDeclaringClass().getName()));
					}
					usingDefaults = true;
					classesList.addAll(getDefaultTestExecutionListenerClasses());
				}

As the code show, it must have testExecutionListeners.mergeMode() set to MergeMode.MERGE_WITH_DEFAULTS and either it has no super class or set testExecutionListeners.inheritListeners to false.

TestNG support

TestNG support spring test framework througth AbstractTestNGSpringContextTests and AbstractTransactionalTestNGSpringContextTests classes, both class annotated by @TestExecutionListeners annotation.
With the default value: inheritListeners set to true, mergeMode set to MergeMode.REPLACE_DEFAULTS

The Cause

Any class inherit AbstractTestNGSpringContextTests and AbstractTransactionalTestNGSpringContextTests will not load the default execution listeners(META-INF/spring.factories ). It will fail to take advantage of testExecutionListener automatic discovery mechanism, the same to Junit4.

Temporary solution

Annotate class inheriting AbstractTestNGSpringContextTests :

@TestExecutionListeners(value = {ServletTestExecutionListener.class,
    DirtiesContextBeforeModesTestExecutionListener.class,
    ApplicationEventsTestExecutionListener.class, DependencyInjectionTestExecutionListener.class,
    DirtiesContextTestExecutionListener.class, EventPublishingTestExecutionListener.class
}, mergeMode = MergeMode.MERGE_WITH_DEFAULTS, inheritListeners = false)

Solution Advice

1. Better to remove @TestExecutionListeners annotation in AbstractTestNGSpringContextTests , AbstractTransactionalTestNGSpringContextTests, AbstractTransactionalJUnit4SpringContextTests and AbstractTransactionalJUnit4SpringContextTests, let the project choose to use TestExecutionListeners through testExecutionListener automatic discovery mechanism.
2. Add new Composed Annotation to let the project use in any test class.
3. change TestExecutionListeners default values in AbstractXXXXSpringContextTests classes: inheritListeners set to false, mergeMode set to MergeMode.MERGE_WITH_DEFAULTS

[sbrannen]

Temporary solution

Annotate class inheriting AbstractTestNGSpringContextTests :

@TestExecutionListeners(value = {ServletTestExecutionListener.class,
    DirtiesContextBeforeModesTestExecutionListener.class,
    ApplicationEventsTestExecutionListener.class, DependencyInjectionTestExecutionListener.class,
    DirtiesContextTestExecutionListener.class, EventPublishingTestExecutionListener.class
}, mergeMode = MergeMode.MERGE_WITH_DEFAULTS, inheritListeners = false)

What happens if you use the following instead?

@TestExecutionListeners(listeners = {}, inheritListeners = false, mergeMode = MergeMode.MERGE_WITH_DEFAULTS)

That should allow you to switch to using all default listeners.

[oopschen]

Temporary solution

Annotate class inheriting AbstractTestNGSpringContextTests :

@TestExecutionListeners(value = {ServletTestExecutionListener.class,
    DirtiesContextBeforeModesTestExecutionListener.class,
    ApplicationEventsTestExecutionListener.class, DependencyInjectionTestExecutionListener.class,
    DirtiesContextTestExecutionListener.class, EventPublishingTestExecutionListener.class
}, mergeMode = MergeMode.MERGE_WITH_DEFAULTS, inheritListeners = false)

What happens if you use the following instead?

@TestExecutionListeners(listeners = {}, inheritListeners = false, mergeMode = MergeMode.MERGE_WITH_DEFAULTS)

That should allow you to switch to using all default listeners.

Thanks for the reply.
It works, better to make the AbstractTestNGSpringContextTests utilize the testExecutionListener automatic discovery mechanism(official document mention). It requires some work to find the truth while the official document haven't mention it.
Beginners fails to start a project by integrating 3rd party library using testExecutionListener automatic discovery mechanism with TestNG or Junit4(That's how i find it).

[sbrannen]

Hi @oopschen,

Thanks for the feedback.

For 6.0 we will remove the @TestExecutionListeners on those 4 base classes to allow users to benefit from default TestExecutionListener registration.

Since that may potentially be a breaking change for some users, we will only update the documentation for 5.3.x. I've created #29281 to address that.