Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump error_prone version to 2.5.1 #13016

Closed

Conversation

davido
Copy link
Contributor

@davido davido commented Feb 13, 2021

Also update transitive dependencies.

Change-Id: I7b35948d1c6c7ba7f1856405a7d4f2d0be76b9af

Also update transitive dependencies.

Change-Id: I7b35948d1c6c7ba7f1856405a7d4f2d0be76b9af
@google-cla google-cla bot added the cla: yes label Feb 13, 2021
@davido
Copy link
Contributor Author

davido commented Feb 13, 2021

//CC @comius, @cushon

The procedure to bump java_tools in Bazel seems to be changed? To verify new build toolchain with EP 2.5.1, I built new java_tools version, conducted release v12.1 in my java_tools fork:

https://github.com/davido/java_tools/releases/download/v12.1/java_tools-v12.1.zip

and created this diff in bazel tree:

diff --git a/distdir_deps.bzl b/distdir_deps.bzl
index 1739a25c2a..1f29d5cc11 100644
--- a/distdir_deps.bzl
+++ b/distdir_deps.bzl
@@ -264,11 +264,10 @@ DIST_DEPS = {
     },
     "remote_java_tools": {
         "aliases": ["remote_java_tools_test", "remote_java_tools_for_testing"],
-        "archive": "java_tools-v11.1.zip",
-        "sha256": "12cffbb7c87622a6bd6e9231e81ecb9efdb118afbdd6e047ef06eeb3d72a7dc3",
+        "archive": "java_tools-v12.1.zip",
+        "sha256": "0504b2ea80179652121bf50f38c5c4c4bfdf9a4b357f0338bcbc60f1d43ad17f",
         "urls": [
-            "https://mirror.bazel.build/bazel_java_tools/releases/java/v11.1/java_tools-v11.1.zip",
-            "https://github.com/bazelbuild/java_tools/releases/download/java_v11.1/java_tools-v11.1.zip",
+            "https://github.com/davido/java_tools/releases/download/v12.1/java_tools-v12.1.zip",
         ],
         "used_in": [
             "additional_distfiles",

After activating all new EP checks from release: 2.5.0 [1], I could confirm that new bug patterns are flagged by EP, e.g.:

  $ bazel build :release
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
WARNING: Option 'java_toolchain' is deprecated
INFO: Invocation ID: e48a6477-47e9-434f-ad01-09339f56ae68
DEBUG: /home/davido/.cache/bazel/_bazel_davido/5c01f4f713b675540b8b424c5c647f63/external/bazel_toolchains/rules/rbe_repo/version_check.bzl:59:14: 
Current running Bazel is not a release version and one was not defined explicitly in rbe_autoconfig target. Falling back to '4.0.0'
INFO: Analyzed target //:release (1324 packages loaded, 46697 targets configured).
INFO: Found 1 target...
ERROR: /home/davido/projects/gerrit2/java/com/google/gerrit/extensions/BUILD:29:13: Building java/com/google/gerrit/extensions/libapi.jar (346 source files) and running annotation processors (AutoAnnotationProcessor, AutoValueProcessor, AutoOneOfProcessor) failed: (Exit 1): java failed: error executing command external/remotejdk11_linux/bin/java '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' ... (remaining 14 argument(s) skipped)
java/com/google/gerrit/extensions/common/FileInfo.java:37: error: [ObjectEqualsForPrimitives] Avoid unnecessary boxing by using plain == for primitive types.
          && Objects.equals(sizeDelta, fileInfo.sizeDelta)
                           ^
    (see https://errorprone.info/bugpattern/ObjectEqualsForPrimitives)
  Did you mean '&& (sizeDelta == fileInfo.sizeDelta)'?
java/com/google/gerrit/extensions/common/FileInfo.java:38: error: [ObjectEqualsForPrimitives] Avoid unnecessary boxing by using plain == for primitive types.
          && Objects.equals(size, fileInfo.size);
                           ^
    (see https://errorprone.info/bugpattern/ObjectEqualsForPrimitives)
  Did you mean '&& (size == fileInfo.size);'?
Target //:release failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 61.787s, Critical Path: 24.53s
INFO: 29 processes: 6 internal, 14 linux-sandbox, 9 worker.
FAILED: Build did NOT complete successfully

I even fixed some of them already: [2].

[1] https://gerrit-review.googlesource.com/c/gerrit/+/296780
[2] https://gerrit-review.googlesource.com/c/gerrit/+/296779

@jin jin added the team-Rules-Java Issues for Java rules label Mar 1, 2021
@comius comius self-requested a review March 1, 2021 12:31
@davido
Copy link
Contributor Author

davido commented Mar 26, 2021

Was done in 1ec7500 by @cushon.

@davido davido closed this Mar 26, 2021
@davido
Copy link
Contributor Author

davido commented Mar 26, 2021

@comius

Could you conduct new java_tools release with 1ec7500 included?

@davido
Copy link
Contributor Author

davido commented Mar 29, 2021

Could you conduct new java_tools release with 1ec7500 included?

To test Java 16 toolchain, I conducted new version of java_tools.

It turns out, that the new EP 2.5.1 is more stricter. Particularly,
Gerrit Code Review project cannot be built any more: [1]

  $ bazel build :release
[...]
ERROR: /home/davido/projects/gerrit/java/com/google/gerrit/server/cache/serialize/BUILD:3:13: Building java/com/google/gerrit/server/cache/serialize/libserialize.jar (9 source files) failed: (Exit 1): java failed: error executing command external/remotejdk16_linux/bin/java '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' ... (remaining 14 argument(s) skipped)
java/com/google/gerrit/server/cache/serialize/JavaCacheSerializer.java:51: error: [BanSerializableRead] Deserializing user input via the `Serializable` API is extremely dangerous
      object = oin.readObject();
                             ^
    (see https://errorprone.info/bugpattern/BanSerializableRead)
Target //:release failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 31.868s, Critical Path: 20.09s
INFO: 154 processes: 2 remote cache hit, 62 internal, 41 linux-sandbox, 49 worker.
FAILED: Build did NOT complete successfully

In the past there were concerns that unconditional upgrade of EP would mean disruptive change, so that I could imagine that other Bazel users would also complain.

[1] https://bugs.chromium.org/p/gerrit/issues/detail?id=14302

@cushon
Copy link
Contributor

cushon commented Mar 29, 2021

I don't think that should have been enabled as an error by default, I'll look at turning it off. It's of course possible to disable with -Xep: BanSerializableRead:OFF. @comius is there a good place to add Error Prone flags like that in the default toolchain? Otherwise I'll get this fixed in the next EP release, and we should hold off on a java_tools release with Error Prone 2.5.1.

@davido
Copy link
Contributor Author

davido commented Mar 29, 2021

I don't think that should have been enabled as an error by default, I'll look at turning it off.

+1.

Otherwise I'll get this fixed in the next EP release, and we should hold off on a java_tools release with Error Prone 2.5.1.

That's unfortunate, as my Java 16 Toolchain support PR is waiting to be reviewed/approved and released in java_tools.

@comius
Copy link
Contributor

comius commented Mar 30, 2021

I don't think that should have been enabled as an error by default, I'll look at turning it off. It's of course possible to disable with -Xep: BanSerializableRead:OFF. @comius is there a good place to add Error Prone flags like that in the default toolchain? Otherwise I'll get this fixed in the next EP release, and we should hold off on a java_tools release with Error Prone 2.5.1.

I believe you can add them to default_java_toolchain.bzl. It is used everywhere.

@cushon
Copy link
Contributor

cushon commented Mar 30, 2021

I believe you can add them to default_java_toolchain.bzl. It is used everywhere.

Thanks. I couldn't remember if that would be a problem for VanillaJavaBuilder (the -Xep flags are non-standard flags that don't work with a regular javac), but VanillaJavaBuilder filters them out:

JavacOptions.removeBazelSpecificFlags(optionsParser.getJavacOpts()),
. I sent you a CL.

@comius
Copy link
Contributor

comius commented Apr 13, 2021

@cushon, there are more downstream failures due to errorprone upgrade. I made java_tools rc1; didn't release it yet.

Here are downstream tests: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/1994#8ab8848c-ea09-4d5a-a7ca-344b0cc30860

 [ReturnValueIgnored] Return value of this method must be used
 [ProtoBuilderReturnValueIgnored] Unnecessary call to proto's #build() method.

@davido
Copy link
Contributor Author

davido commented Apr 13, 2021

@cushon @comius

Looking into Kythe breakage it seems that the EP upgrade is actually broken with NCFE: org/checkerframework/shaded/dataflow/analysis/ForwardTransferFunction:

xecution platform: @local_config_platform//:host
kythe/java/com/google/devtools/kythe/util/Span.java:66: error: An unhandled exception was thrown by the Error Prone static analysis plugin.
  public boolean equals(Object o) {
                 ^
     Please report this at https://github.com/google/error-prone/issues/new and include the following:
 
     error-prone version: 2.6.0
     BugPattern: EqualsBrokenForNull
     Stack Trace:
     java.lang.NoClassDefFoundError: org/checkerframework/shaded/dataflow/analysis/ForwardTransferFunction
  	at com.google.errorprone.bugpatterns.nullness.EqualsBrokenForNull.matchMethod(EqualsBrokenForNull.java:76)
  	at com.google.errorprone.scanner.ErrorProneScanner.processMatchers(ErrorProneScanner.java:450)
  	at com.google.errorprone.scanner.ErrorProneScanner.visitMethod(ErrorProneScanner.java:740)
  	at com.google.errorprone.scanner.ErrorProneScanner.visitMethod(ErrorProneScanner.java:151)
  	at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:898)
  	at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
  	at com.google.errorprone.scanner.Scanner.scan(Scanner.java:74)
  	at com.google.errorprone.scanner.Scanner.scan(Scanner.java:48)
  	at jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:90)
  	at jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
  	at jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
  	at jdk.compiler/com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:187)
  	at com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:549)
  	at com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:151)
  	at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:808)
  	at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
  	at com.google.errorprone.scanner.Scanner.scan(Scanner.java:74)
  	at com.google.errorprone.scanner.Scanner.scan(Scanner.java:48)
  	at jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
  	at jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
  	at jdk.compiler/com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:144)
  	at com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:561)
  	at com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:151)
  	at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:591)
  	at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:56)
  	at com.google.errorprone.scanner.Scanner.scan(Scanner.java:58)
  	at com.google.errorprone.scanner.ErrorProneScannerTransformer.apply(ErrorProneScannerTransformer.java:43)
  	at com.google.errorprone.ErrorProneAnalyzer.finished(ErrorProneAnalyzer.java:152)
  	at com.google.devtools.build.buildjar.javac.plugins.errorprone.ErrorPronePlugin.postFlow(ErrorPronePlugin.java:161)
  	at com.google.devtools.build.buildjar.javac.BlazeJavaCompiler.flow(BlazeJavaCompiler.java:115)
  	at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1365)
  	at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:960)
  	at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:104)
  	at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:147)
  	at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100)
  	at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94)
  	at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:123)
  	at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.fallback(ReducedClasspathJavaLibraryBuilder.java:103)
  	at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:65)
  	at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110)
  	at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118)
  	at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:99)
  	at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:79)
  	at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:151)
  	at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$createResponseThread$0(WorkRequestHandler.java:134)
  	at java.base/java.lang.Thread.run(Thread.java:834)
  Caused by: java.lang.ClassNotFoundException: org.checkerframework.shaded.dataflow.analysis.ForwardTransferFunction
  	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
  	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
  	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
  	... 46 more
(15:40:15) WARNING: Writing to Remote Cache:

@davido
Copy link
Contributor Author

davido commented Apr 13, 2021

Yea, while 2.6.0 includes this CL: google/error-prone@d1b93d4, where checker framework version was bumped to 3.11.0, the upgrade CL: 22c8fc3 missed to upgrade the checker framework to 3.11.0. It's still dataflow-shaded-3.1.2.jar.

Hm... just noticed, that this PR, that was actually abandoned, did update checker framework to 3.7.1.

cushon added a commit to cushon/bazel that referenced this pull request Apr 13, 2021
@cushon cushon mentioned this pull request Apr 13, 2021
@cushon
Copy link
Contributor

cushon commented Apr 13, 2021

22c8fc3 missed to upgrade the checker framework to 3.11.0

Thanks, mailed: #13348

@davido
Copy link
Contributor Author

davido commented Apr 13, 2021

Thanks, mailed: #13348

Thanks. Meantime, I filed this issue: #13346, may be update the commit message in your PR?

Moreover, comparing this PR to the merged PRs, other transitive dependencies were missed to be upgraded as well. I compared pom.xml in EP with third-party directory in Bazel:

auto/auto-service-1.0-rc7.jar
auto/auto-value-1.7.4.jar

Not sure it's required, though.

@cushon
Copy link
Contributor

cushon commented Apr 13, 2021

may be update the commit message in your PR?

Done

auto/auto-service-1.0-rc7.jar
auto/auto-value-1.7.4.jar

Upgrading these too is probably a good idea, but I suspect there weren't any breaking changes that matter for Error Prone, especially at runtime, since both of those are annotation processors.

bazel-io pushed a commit that referenced this pull request Apr 13, 2021
#13016 (comment)

Closes #13348.

Signed-off-by: Philipp Wollermann <philwo@google.com>
@cushon
Copy link
Contributor

cushon commented Apr 13, 2021

Here are downstream tests: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/1994#8ab8848c-ea09-4d5a-a7ca-344b0cc30860

 [ReturnValueIgnored] Return value of this method must be used
 [ProtoBuilderReturnValueIgnored] Unnecessary call to proto's #build() method.

@comius I think there's just the one breakage in bazel-buildfarm, I mailed bazelbuild/bazel-buildfarm#744

We can also disable that check globally, LMK if you have a preference.

@comius
Copy link
Contributor

comius commented Apr 14, 2021

We can also disable that check globally, LMK if you have a preference.

This is perfect! Thank @cushon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes team-Rules-Java Issues for Java rules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants