Error on executing tests with spring-kafka-test 3.0.0

[wilsonrf]

In what version(s) of Spring for Apache Kafka are you seeing this issue?

3.0.0

Describe the bug

There's a problem executing tests when using the spring-kafka-test 3.0.0

To Reproduce

1. Create a new codebase
2. Run tests - ./gradlew clean build (will work fine)
3. Add the spring-kafka-test dependency
4. Run tests again - ./gradlew clean build
5. Tests will fail

org.gradle.api.internal.tasks.testing.TestSuiteExecutionException: Could not complete execution for Gradle Test Executor 2.
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:63)
	at java.base@17.0.3/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base@17.0.3/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base@17.0.3/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base@17.0.3/java.lang.reflect.Method.invoke(Method.java:568)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: java.lang.NoClassDefFoundError: org/junit/platform/launcher/TestExecutionListener
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1012)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
	at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:862)
	at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:467)
	at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(ServiceLoader.java:1217)
	at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1228)
	at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
	at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309)
	at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132)
	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at org.junit.platform.launcher.core.LauncherFactory.registerTestExecutionListeners(LauncherFactory.java:179)
	at org.junit.platform.launcher.core.LauncherFactory.createDefaultLauncher(LauncherFactory.java:137)
	at org.junit.platform.launcher.core.LauncherFactory.create(LauncherFactory.java:125)
	at org.junit.platform.launcher.core.LauncherFactory.create(LauncherFactory.java:109)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:97)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
	... 18 more
Caused by: java.lang.ClassNotFoundException: org.junit.platform.launcher.TestExecutionListener
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
	... 50 more

Expected behavior

The tests should be executed

Sample
with spring-boot 3.0.0 https://github.com/wilsonrf/spring-kafka-test-sample
without spring-boot 3.0.0 https://github.com/wilsonrf/spring-kafka-test-sample/tree/without-springboot

[garyrussell]

The new global embedded broker feature added a hard dependency for the platform launcher.

As a work around, please use

dependencies {
	testImplementation("org.junit.jupiter:junit-jupiter:5.9.1")
	testImplementation("org.junit.platform:junit-platform-launcher:1.9.1")
	testImplementation("org.springframework.kafka:spring-kafka-test:3.0.0")
}

With the non-boot version. With boot, you don’t need the version.

[garyrussell]

@sbrannen Looking at the stack trace, it is clear that the launcher is on the gradle class path, but, obviously, not on the test runtime class path, while trying to load test execution listeners via the service loader.

Do you think that this is something that JUnit and/or Gradle should handle, or are we forced to make the launcher a hard transitive dependency of spring-kafka-test? The latter doesn't seem right because (perhaps rarely) not all users will use JUnit.

FWIW, I don't see the same problem with maven, and I don't see the launcher on the CP there either.

cc/ @wilkinsona

[wilkinsona]

AIUI, Gradle offers better classpath separation than Maven does here. This means that the launcher isn't leaked into the classpath of the tests as it isn't typically needed. I have a feeling that TestExecutionListener may not be the right tool for the job.

[artembilan]

So, there is something wrong with class loader manipulations in JUnit by itself:

	at org.junit.platform.launcher.core.LauncherFactory.registerTestExecutionListeners(LauncherFactory.java:179)
	at org.junit.platform.launcher.core.LauncherFactory.createDefaultLauncher(LauncherFactory.java:137)
	at org.junit.platform.launcher.core.LauncherFactory.create(LauncherFactory.java:125)
	at org.junit.platform.launcher.core.LauncherFactory.create(LauncherFactory.java:109)

The LauncherFactory is there and tries to load listener via service loader factory and then it fails finding its own org.junit.platform.launcher.TestExecutionListener.
Even if we can look into something else for our own solution or exclude this listener from discovery, anyone else with his/her own implementation for the TestExecutionListener would encounter the same classpath issue.
I do not see any specific note for Gradle when we develop custom TestExecutionListener in JUnit 5 docs.

I'm not sure though how then that out-of-the-box UniqueIdTrackingListener works well: it is indeed is present in the classpath same way as our GlobalEmbeddedKafkaTestExecutionListener and has its entry for service loader.

/CC @sbrannen

[garyrussell]

@sbrannen - we need some guidance here, please; thanks!

[sbrannen]

AIUI, Gradle offers better classpath separation than Maven does here. This means that the launcher isn't leaked into the classpath of the tests as it isn't typically needed.

That's correct. The JUnit Platform Launcher API is not on the test's classpath by default with Gradle, and that's by design. Users normally don't need direct access to the launcher in their test suites.

However, if that's necessary, then it's expected that the build include the junit-platform-launcher artifact in the test runtime classpath -- if the junit-platform-launcher is not automatically pulled in as a transitive dependency.

For example, adding the following to the supplied sample project allows it to build.

main branch:

testRuntimeOnly("org.junit.platform:junit-platform-launcher")

without-springboot branch:

testRuntimeOnly("org.junit.platform:junit-platform-launcher:1.9.0")

I have a feeling that TestExecutionListener may not be the right tool for the job.

That's a bit tricky to assess.

A TestExecutionListener benefits from automatic registration; however, for Gradle the "test runtime classpath" may have to be augmented (see above).

Whereas, a global Jupiter Extension might be more suitable (in that it's limited to the Jupiter TestEngine and can interact directly with the ExtensionContext.Store), but Jupiter extensions cannot be registered automatically without setting the junit.jupiter.extensions.autodetection.enabled configuration parameter to true.

So there are pros and cons for each approach.