How do I get transitive dependencies? #11239

pro-wh · 2022-04-13T00:26:48Z

pro-wh
Apr 13, 2022

We're using Play with Scala in our project. I don't know much sbt, but does it have a way to get transitive dependencies automatically?

For example, from my build.sbt:

// (among others)
libraryDependencies += jdbc

Clearly there's some level of knowledge it has about transitive dependencies, such as in the output of sbt dependencyTree:

[info]   +-com.typesafe.play:play-jdbc_2.12:2.8.8 [S]
[info]   | +-com.googlecode.usc:jdbcdslog:1.0.6.2
[info]   | | +-org.slf4j:slf4j-api:1.5.10 (evicted by: 1.7.33)
[info]   | | +-org.slf4j:slf4j-api:1.7.33
...
[info]   +-com.typesafe.play:play-logback_2.12:2.8.8 [S]
[info]   | +-ch.qos.logback:logback-classic:1.2.3
[info]   | | +-ch.qos.logback:logback-core:1.2.3
[info]   | | +-org.slf4j:slf4j-api:1.7.25 (evicted by: 1.7.33)
[info]   | | +-org.slf4j:slf4j-api:1.7.33

(slf4j-api appears in many other places in addition to the snippet above)

(I just realized dependencyTree is mentioned nowhere in sbt reference manual. what even am I doing?)

But when I use sbt run to try out my app, it fails with this:

[error] java.lang.NoClassDefFoundError: org/slf4j/spi/LoggerFactoryBinder
[error] 	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
[error] 	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
[error] 	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
[error] 	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555)
[error] 	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
[error] 	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
[error] 	at java.base/java.security.AccessController.doPrivileged(Native Method)
[error] 	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
[error] 	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
[error] 	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
[error] 	at play.api.libs.logback.LogbackLoggerConfigurator.loggerFactory(LogbackLoggerConfigurator.scala:21)
[error] 	at play.api.libs.logback.LogbackLoggerConfigurator.configure(LogbackLoggerConfigurator.scala:85)
[error] 	at play.api.libs.logback.LogbackLoggerConfigurator.init(LogbackLoggerConfigurator.scala:31)
[error] 	at play.core.server.DevServerStart$.$anonfun$mainDev$1(DevServerStart.scala:106)
[error] 	at play.utils.Threads$.withContextClassLoader(Threads.scala:22)
[error] 	at play.core.server.DevServerStart$.mainDev(DevServerStart.scala:76)
[error] 	at play.core.server.DevServerStart$.mainDevHttpMode(DevServerStart.scala:50)
[error] 	at play.core.server.DevServerStart.mainDevHttpMode(DevServerStart.scala)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[error] 	at play.runsupport.Reloader$.startDevMode(Reloader.scala:306)
[error] 	at play.sbt.run.PlayRun$.devModeServer$lzycompute$1(PlayRun.scala:100)
[error] 	at play.sbt.run.PlayRun$.devModeServer$1(PlayRun.scala:83)
[error] 	at play.sbt.run.PlayRun$.$anonfun$playRunTask$3(PlayRun.scala:107)
[error] 	at play.sbt.run.PlayRun$.$anonfun$playRunTask$3$adapted(PlayRun.scala:67)
[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] Caused by: java.lang.ClassNotFoundException: org.slf4j.spi.LoggerFactoryBinder
[error] 	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
[error] 	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
[error] 	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
[error] 	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
[error] 	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
[error] 	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
[error] 	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555)
[error] 	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
[error] 	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
[error] 	at java.base/java.security.AccessController.doPrivileged(Native Method)
[error] 	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
[error] 	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
[error] 	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
[error] 	at play.api.libs.logback.LogbackLoggerConfigurator.loggerFactory(LogbackLoggerConfigurator.scala:21)
[error] 	at play.api.libs.logback.LogbackLoggerConfigurator.configure(LogbackLoggerConfigurator.scala:85)
[error] 	at play.api.libs.logback.LogbackLoggerConfigurator.init(LogbackLoggerConfigurator.scala:31)
[error] 	at play.core.server.DevServerStart$.$anonfun$mainDev$1(DevServerStart.scala:106)
[error] 	at play.utils.Threads$.withContextClassLoader(Threads.scala:22)
[error] 	at play.core.server.DevServerStart$.mainDev(DevServerStart.scala:76)
[error] 	at play.core.server.DevServerStart$.mainDevHttpMode(DevServerStart.scala:50)
[error] 	at play.core.server.DevServerStart.mainDevHttpMode(DevServerStart.scala)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[error] 	at play.runsupport.Reloader$.startDevMode(Reloader.scala:306)
[error] 	at play.sbt.run.PlayRun$.devModeServer$lzycompute$1(PlayRun.scala:100)
[error] 	at play.sbt.run.PlayRun$.devModeServer$1(PlayRun.scala:83)
[error] 	at play.sbt.run.PlayRun$.$anonfun$playRunTask$3(PlayRun.scala:107)
[error] 	at play.sbt.run.PlayRun$.$anonfun$playRunTask$3$adapted(PlayRun.scala:67)
[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] stack trace is suppressed; run last Compile / run for the full output
[error] (Compile / run) java.lang.reflect.InvocationTargetException
[error] Total time: 1 s, completed Apr 12, 2022, 5:14:46 PM

I also tried looking at the output of sbt "show Runtime / fullClasspath", and slf4j-api is indeed not listed.

It would be best if I didn't have to list all the transitive dependencies in build.sbt. But will I have to after all?

Answered by mkurz

Apr 13, 2022

Transitive dependencies are fetched automatically, you don't have to do anything.
Which version of sbt are you using?
Also please upgrade to latest Play 2.8.15.

View full answer

mkurz · 2022-04-13T19:21:29Z

mkurz
Apr 13, 2022
Maintainer

Transitive dependencies are fetched automatically, you don't have to do anything.
Which version of sbt are you using?
Also please upgrade to latest Play 2.8.15.

1 reply

pro-wh Apr 15, 2022
Author

this turned out to be the answer 🙃, that it should get transitive dependencies automatically. in this case it turned out that there's something suspicious going on with my local maven repo

jducoeur · 2022-04-13T21:45:08Z

jducoeur
Apr 13, 2022

As @mkurz says, sbt should deal with transitive dependencies automatically. So the sort of error you're seeing here is usually caused by conflicting dependencies, where Play and your libraries depend on different versions of the transitive dependency -- only one can actually be loaded, so you can get runtime crashes like this because a library is trying to call a function or invoke a class that doesn't exist in the version of the dependency that actually wound up linked in. (This is what those eviction warnings are about, in the output you included. Note for example that jdbcdslog depends on slf4j 1.5.10, but what actually got loaded is version 1.7.33.)

In practice, the best way to avoid this is usually to make sure you have the most up-to-date version of all of your direct dependencies -- if they are all being actively maintained, that tends to avoid this sort of incompatible-version collision.

0 replies

pro-wh · 2022-04-13T21:51:51Z

pro-wh
Apr 13, 2022
Author

That's interesting, thanks for the info. I saw that there was some documentation on the default conflict resolution being to use the highest version, and I take it that the "1.7.33" and earlier versions being "evicted" to mean that this has taken place.

But other than the actual interoperability issues between versions of slf4j-api (actually should be fine, to my understanding), I should at least see the 1.7.33 version of slf4j-api in the classpath, right?

I'm on the latest sbt, but I'm going to try updating to the latest play as well.

I realize there's not enough here to reproduce the issue as is. Thanks for answering without feeling that you need to solve the issue for me. I am seeking (i) general strategies for troubleshooting this sort of thing (ii) feedback on whether I'm reading these printouts right, e.g. what does that [S] part mean in the dependency tree, and (iii) whatever technical information people have about steps that go between what shows up in the dependency tree and the runtime classpath.

0 replies

pro-wh · 2022-04-14T00:25:39Z

pro-wh
Apr 14, 2022
Author

I updated several dependencies in places where we have access. The result is that the app gets as far as starting up, and there are no more complaints about slf4j. But this turned out not to be the only classpath issue. Here's more of the same genre of issue, coming during the first request:

2022-04-13 17:13:08 ERROR p.api.http.DefaultHttpErrorHandler  

! @7nanpe401 - Internal server error, for (GET) [/] ->
 
play.api.UnexpectedException: Unexpected exception[NoClassDefFoundError: com/google/common/util/concurrent/internal/InternalFutureFailureAccess]
	at play.core.server.DevServerStart$$anon$1.reload(DevServerStart.scala:258)
	at play.core.server.DevServerStart$$anon$1.get(DevServerStart.scala:148)
	at play.core.server.AkkaHttpServer.handleRequest(AkkaHttpServer.scala:302)
	at play.core.server.AkkaHttpServer.$anonfun$createServerBinding$1(AkkaHttpServer.scala:224)
	at akka.stream.impl.fusing.MapAsync$$anon$30.onPush(Ops.scala:1307)
	at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:542)
	at akka.stream.impl.fusing.GraphInterpreter.processEvent(GraphInterpreter.scala:496)
	at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:390)
	at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:650)
	at akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:521)
	at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:625)
	at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:800)
	at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:818)
	at akka.actor.Actor.aroundReceive(Actor.scala:537)
	at akka.actor.Actor.aroundReceive$(Actor.scala:535)
	at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:716)
	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580)
	at akka.actor.ActorCell.invoke(ActorCell.scala:548)
	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
	at akka.dispatch.Mailbox.run(Mailbox.scala:231)
	at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.lang.NoClassDefFoundError: com/google/common/util/concurrent/internal/InternalFutureFailureAccess
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3472)
	at com.google.common.cache.LocalCache$LoadingValueReference.<init>(LocalCache.java:3476)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2134)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
	at com.google.common.cache.LocalCache.get(LocalCache.java:3962)
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3985)
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4946)
	at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4952)
	at com.google.inject.internal.Annotations$AnnotationChecker.hasAnnotations(Annotations.java:271)
	at com.google.inject.internal.Annotations.isBindingAnnotation(Annotations.java:337)
	at com.google.inject.Key.ensureIsBindingAnnotation(Key.java:357)
	at com.google.inject.Key.strategyFor(Key.java:326)
	at com.google.inject.Key.get(Key.java:232)
	at play.api.inject.guice.GuiceKey$.apply(GuiceInjectorBuilder.scala:416)
	at play.api.inject.guice.GuiceableModuleConversions$$anon$4.$anonfun$configure$2(GuiceInjectorBuilder.scala:376)
	at play.api.inject.guice.GuiceableModuleConversions$$anon$4.$anonfun$configure$2$adapted(GuiceInjectorBuilder.scala:374)
	at scala.collection.immutable.List.foreach(List.scala:431)
	at play.api.inject.guice.GuiceableModuleConversions$$anon$4.configure(GuiceInjectorBuilder.scala:374)
	at com.google.inject.AbstractModule.configure(AbstractModule.java:61)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:347)
	at com.google.inject.spi.Elements.getElements(Elements.java:104)
	at com.google.inject.util.Modules$OverrideModule.configure(Modules.java:214)
	at com.google.inject.AbstractModule.configure(AbstractModule.java:61)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:347)
	at com.google.inject.spi.Elements.getElements(Elements.java:104)
	at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:137)
	at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:105)
	at com.google.inject.Guice.createInjector(Guice.java:87)
	at com.google.inject.Guice.createInjector(Guice.java:78)
	at play.api.inject.guice.GuiceBuilder.injector(GuiceInjectorBuilder.scala:200)
	at play.api.inject.guice.GuiceApplicationBuilder.build(GuiceApplicationBuilder.scala:155)
	at play.api.inject.guice.GuiceApplicationLoader.load(GuiceApplicationLoader.scala:21)
	at play.core.server.DevServerStart$$anon$1.$anonfun$reload$2(DevServerStart.scala:193)
	at play.utils.Threads$.withContextClassLoader(Threads.scala:22)
	at play.core.server.DevServerStart$$anon$1.reload(DevServerStart.scala:185)
	... 25 common frames omitted
Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 90 common frames omitted

Which again, in various places in the dependency tree:

[info]   | +-com.google.inject:guice:4.2.3
[info]   | | +-aopalliance:aopalliance:1.0
[info]   | | +-com.google.guava:guava:27.1-jre (evicted by: 30.1.1-jre)
[info]   | | +-com.google.guava:guava:30.1.1-jre
[info]   | | | +-com.google.code.findbugs:jsr305:3.0.2
[info]   | | | +-com.google.errorprone:error_prone_annotations:2.5.1
[info]   | | | +-com.google.guava:failureaccess:1.0.1
[info]   | | | +-com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-wit..
[info]   | | | +-com.google.j2objc:j2objc-annotations:1.3
[info]   | | | +-org.checkerframework:checker-qual:3.10.0
[info]   | | | +-org.checkerframework:checker-qual:3.8.0 (evicted by: 3.10.0)

(com.google.guava:failureaccess:1.0.1 being the one that has com/google/common/util/concurrent/internal/InternalFutureFailureAccess)

As before, that jar is absent from the show Runtime / fullClasspath output too.

This time there isn't even any conflicts or "evicted" going on for this one.

15 replies

mkurz Apr 14, 2022
Maintainer

Oh yes and with Scala 2.13.8 if that even matters.

pro-wh Apr 14, 2022
Author

Okay I'm on Scala 2.12.15, but I was able to reproduce that working classpath on a brand new project. You also have the older versions of Guava evicted right?

mkurz Apr 14, 2022
Maintainer

Yes same story: +-com.google.guava:guava:27.1-jre (evicted by: 30.1.1-jre)
So you are on Ubuntu and on a brand new project and getting that error? Can you push me that project to a new GitHub repo?

mkurz Apr 14, 2022
Maintainer

Sorry I misunderstood, so you are saying on a brand new project it is working.

pro-wh Apr 15, 2022
Author

I've gradually narrowed down my app to try to pinpoint what causes it to diverge from a brand new project. It turned out to be from this line:

resolvers += Resolver.mavenLocal

which we were using to include an unpublished library from elsewhere in our codebase. So now I gotta investigate why that's breaking things.

Thanks for your help and suggestions!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Play Framework

How do I get transitive dependencies? #11239

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 4 comments 16 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Play Framework

How do I get transitive dependencies? #11239

pro-wh Apr 13, 2022

Replies: 4 comments · 16 replies

mkurz Apr 13, 2022 Maintainer

pro-wh Apr 15, 2022 Author

jducoeur Apr 13, 2022

pro-wh Apr 13, 2022 Author

pro-wh Apr 14, 2022 Author

mkurz Apr 14, 2022 Maintainer

pro-wh Apr 14, 2022 Author

mkurz Apr 14, 2022 Maintainer

mkurz Apr 14, 2022 Maintainer

pro-wh Apr 15, 2022 Author

pro-wh
Apr 13, 2022

Replies: 4 comments 16 replies

mkurz
Apr 13, 2022
Maintainer

pro-wh Apr 15, 2022
Author

jducoeur
Apr 13, 2022

pro-wh
Apr 13, 2022
Author

pro-wh
Apr 14, 2022
Author

mkurz Apr 14, 2022
Maintainer

pro-wh Apr 14, 2022
Author

mkurz Apr 14, 2022
Maintainer

mkurz Apr 14, 2022
Maintainer

pro-wh Apr 15, 2022
Author