Look for other Maven artifacts that contain Guava classes, and list them in our metadata

[cpovirk]

Just as the current metadata can detect conflicts with google-collections, it could detect conflicts with those other artifacts.

For example, I keep forgetting that we ourselves released guava-base, etc. for exactly one release. And others have done this (example?), too.

Here's a person who recently got bit by this.

[ben-manes]

guava-osgi is another one (stackoverflow)

[pan3793]

guava-jdk5

[pan3793]

hive-exec (at least in 2.3.0~2.3.7)

[cpovirk]

Oh, thanks, I should definitely have thought of guava-jdk5.

Which versions of hive-exec have this problem? I pulled hive-exec-0.9.0.jar, hive-exec-1.2.2-core.jar, hive-exec-2.3.9-core.jar, hive-exec-3.1.3-core.jar, hive-exec-4.0.0-beta-1.jar, and hive-exec-4.0.0-beta-1-fallbackauthorizer.jar, and only 4.0.0-beta-1.jar appeared to contain Guava classes, which thankfully were repackaged and thus not a problem (though I also see other com.google classes in that jar that have not been repackaged, all under com.google.protobuf or com.google.re2j).

Not that we can necessarily do much about hive-exec: Since it contains additional classes beyond just Guava's, we can't say to replace its contents with Guava. At most, we might be able to say to replace Guava's contents with it, and I'm not sure that's likely to be a net win :(

[pan3793]

@cpovirk for instance, hive-exec-2.3.7.jar

$ wget https://repo1.maven.org/maven2/org/apache/hive/hive-exec/2.3.7/hive-exec-2.3.7.jar
$ jar -tf hive-exec-2.3.7.jar | grep 'com/google/common'
com/google/common/
com/google/common/annotations/
com/google/common/annotations/Beta.class
com/google/common/annotations/GwtCompatible.class
com/google/common/annotations/GwtIncompatible.class
com/google/common/annotations/VisibleForTesting.class
com/google/common/base/
com/google/common/base/Absent.class
com/google/common/base/AbstractIterator$1.class
com/google/common/base/AbstractIterator$State.class
...

[cpovirk]

Ah, thanks, I didn't pay close enough attention to that "core" suffix that I was looking at.

That's unfortunate. As noted in #6666 (comment), I'm not sure how much we can do when the jar contains more than just Guava (including, it appears, other libraries, like org.json). Maybe apache/hive#4542 will be progress toward having a normal dependency on Guava instead of including it in their jar?

[edit: And it does appear that hive-exec is used as a "normal dependency,", not just as some kind of standalone build tool or something.]

[pan3793]

@cpovirk I don't want to extend the topic too much, I list hive-exec.jar here just because it contains Guava classes. Actually, upgrading Hive(even for patched version) may have more risks and difficulties than upgrading Guava in Hadoop ecosystem projects. hive-exec-2.3.7.jar is actually a standalone jar, and hive-exec-2.3.7-core.jar is actually the original normal jar. Unfortunately, hive-exec-2.3.7.jar seems to be abused as a normal dependency widely.

[cpovirk]

@jensdietrich pointed me to https://github.com/github/advisory-database/pull/2444/files, which adds:

• de.mhus.ports:vaadin-shared-deps
• org.apache.servicemix.bundles:org.apache.servicemix.bundles.guava
• org.hudsonci.lib.guava:guava
• org.sonatype.sisu:sisu-guava

[edit: I was also just reminded of net.tribe7.seeds]

[cpovirk]

Oh, but I have to check how many of those use shading, rather than the original package names.