ResourceLeakChecker infinite loop

[delanym]

On some of my larger Maven modules, when I use ResourceLeakChecker, the compiler stop outputting logs (verbose logs) and hogs CPU. After 10min on one project I cancelled it.

I can't share these modules. What else can I do to troubleshoot this?
My configuration is like #5007 (comment)

Would be nice if checker-framework had a presence on Gitter, eg https://gitter.im/typetools/checker-framework

[mernst]

I'm sorry you are having trouble. Thanks for reporting it.

It would be useful if you could provide a stack trace.
This will help to determine which of the three components of the Resource Leak Checker contains the infinite loop.
Here are some ways to get a traceback (stack trace) from each Java thread (but keep running):

• jstack _pid_ (jcmd gives a list of pids)
• jcmd _pid_ Thread.print (just jcmd gives a list of pids)
• kill -QUIT _pid_
• ctrl-\ if Java is running in a shell.

It would also be useful to know which file triggers the infinite loop.
You can use one of the command-line options at
https://checkerframework.org/manual/#creating-debugging-options-progress
to yield insight into this.
Then, you can determine whether the infinite loop occurs when you invoke javac (with the Resource Leak Checker) on just that file. You may be able to simplify the code until it doesn't reveal any of your trade secrets. This will be very helpful in enabling us to reproduce the infinite loop locally, in order to ensure that we have fixed your problem.

Thanks for your help with this debugging!

[delanym]

@mernst here's the stack trace

"mvn-builder-artifact" #17 prio=5 os_prio=0 cpu=1051298.76ms elapsed=1064.16s tid=0x00007f86b0267800 nid=0x8bf09 runnable  [0x00007f8684ff4000]                                                                                                               
   java.lang.Thread.State: RUNNABLE                                                                                            
        at java.util.AbstractCollection.addAll(java.base@11.0.14.1/AbstractCollection.java:351)
        at java.util.TreeSet.addAll(java.base@11.0.14.1/TreeSet.java:312)                                                      
        at org.checkerframework.javacutil.AnnotationUtils.compareAnnotationMirrors(AnnotationUtils.java:356)                                                                                                                                                  
        at org.checkerframework.javacutil.AnnotationUtils$$Lambda$358/0x0000000800631440.compare(Unknown Source)               
        at java.util.TreeMap.compare(java.base@11.0.14.1/TreeMap.java:1292)
        at java.util.TreeMap.put(java.base@11.0.14.1/TreeMap.java:536)                                                       
        at java.util.TreeSet.add(java.base@11.0.14.1/TreeSet.java:255)                             
        at org.checkerframework.framework.flow.CFAbstractValue$AnnotationSetCombiner.combineSets(CFAbstractValue.java:665)                                                                                                                                    
        at org.checkerframework.framework.flow.CFAbstractValue.upperBound(CFAbstractValue.java:422)                          
        at org.checkerframework.framework.flow.CFAbstractValue.leastUpperBound(CFAbstractValue.java:380)    
        at org.checkerframework.framework.flow.CFAbstractStore.upperBoundOfValues(CFAbstractStore.java:1168)  
        at org.checkerframework.framework.flow.CFAbstractStore.upperBound(CFAbstractStore.java:1096)          
        at org.checkerframework.framework.flow.CFAbstractStore.leastUpperBound(CFAbstractStore.java:1078)
        at org.checkerframework.framework.flow.CFAbstractStore.leastUpperBound(CFAbstractStore.java:63)                        
        at org.checkerframework.dataflow.analysis.ConditionalTransferResult.getRegularStore(ConditionalTransferResult.java:121)                                                                                                                               
        at org.checkerframework.dataflow.analysis.ForwardAnalysisImpl.runAnalysisFor(ForwardAnalysisImpl.java:310)             
        at org.checkerframework.dataflow.analysis.AnalysisResult.runAnalysisFor(AnalysisResult.java:434)                     
        at org.checkerframework.dataflow.analysis.AnalysisResult.runAnalysisFor(AnalysisResult.java:402)                     
        at org.checkerframework.dataflow.analysis.AnalysisResult.getStoreAfter(AnalysisResult.java:378)
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.getStoreAfter(GenericAnnotatedTypeFactory.java:1203)                                                                                                                               
        at org.checkerframework.checker.resourceleak.MustCallConsistencyAnalyzer.propagateObligationsToSuccessorBlocks(MustCallConsistencyAnalyzer.java:1708)                                                                                                 
        at org.checkerframework.checker.resourceleak.MustCallConsistencyAnalyzer.analyze(MustCallConsistencyAnalyzer.java:503) 
        at org.checkerframework.checker.resourceleak.ResourceLeakAnnotatedTypeFactory.postAnalyze(ResourceLeakAnnotatedTypeFactory.java:119)                                                                                                                  
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.analyze(GenericAnnotatedTypeFactory.java:1559)      
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.performFlowAnalysis(GenericAnnotatedTypeFactory.java:1394)                                                                                                                         
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.checkAndPerformFlowAnalysis(GenericAnnotatedTypeFactory.java:1829)                                                                                                                 
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.preProcessClassTree(GenericAnnotatedTypeFactory.java:414)                                                                                                                          
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitClass(BaseTypeVisitor.java:475)                           
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitClass(BaseTypeVisitor.java:180)                                                                                                                                                          
        at com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(jdk.compiler@11.0.14.1/JCTree.java:808)                          
        at com.sun.source.util.TreePathScanner.scan(jdk.compiler@11.0.14.1/TreePathScanner.java:56)                            
        at org.checkerframework.framework.source.SourceVisitor.visit(SourceVisitor.java:82)         
        at org.checkerframework.framework.source.SourceChecker.typeProcess(SourceChecker.java:987)                             
        at org.checkerframework.common.basetype.BaseTypeChecker.typeProcess(BaseTypeChecker.java:542)
        at org.checkerframework.javacutil.AbstractTypeProcessor$AttributionTaskListener.finished(AbstractTypeProcessor.java:188)

[delanym]

I tried adding -Afilenames, but it doesn't display any filenames. Your documentation notes that "This does not work if you are using Maven, because with forked compilation, the maven-compiler-plugin queues up all the output and then prints it at the end.", but Im not even using forked compilation.

I enabled verbose output on the compiler and the last thing it shows is
[loading /home/sol/.m2/repository/com/google/code/findbugs/jsr305/3.0.1/jsr305-3.0.1.jar(/javax/annotation/CheckForNull.class)]

[msridhar]

FYI #5107 should yield a significant speedup in the Resource Leak Checker. The stack you posted is one that should be optimized by that PR. It won't solve an infinite loop, but it's possible this is a case where the RLC is just slow.

[msridhar]

I tried adding -Afilenames, but it doesn't display any filenames. Your documentation notes that "This does not work if you are using Maven, because with forked compilation, the maven-compiler-plugin queues up all the output and then prints it at the end.", but Im not even using forked compilation.

I don't think -Afilenames will work in Maven, independent of forked compilation. See codehaus-plexus/plexus-compiler#149. @delanym if you know anything about Maven internals and could contribute a fix for that issue it would be a huge help for a lot of Maven users I think. In the meantime, if you can figure out the exact javac command that Maven uses to build the relevant target, and then run that directly from the command line, -Afilenames will work. Output from running mvn -X should be helpful there.

[delanym]

here's another

"mvn-builder-artifact" #17 prio=5 os_prio=0 cpu=1285291.09ms elapsed=1300.32s tid=0x00007f38e4f66000 nid=0x8cbf3 runnable  [0x00007f38b88eb000]
   java.lang.Thread.State: RUNNABLE                                                                                            
        at java.util.IdentityHashMap.putAll(java.base@11.0.14.1/IdentityHashMap.java:508)
        at org.checkerframework.dataflow.analysis.AbstractAnalysis.setNodeValues(AbstractAnalysis.java:220)                                                                                                                                                   
        at org.checkerframework.dataflow.analysis.ForwardAnalysisImpl.runAnalysisFor(ForwardAnalysisImpl.java:253)
        at org.checkerframework.dataflow.analysis.AnalysisResult.runAnalysisFor(AnalysisResult.java:434)
        at org.checkerframework.dataflow.analysis.AnalysisResult.runAnalysisFor(AnalysisResult.java:402)                     
        at org.checkerframework.dataflow.analysis.AnalysisResult.getStoreAfter(AnalysisResult.java:378)
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.getStoreAfter(GenericAnnotatedTypeFactory.java:1203)                 
        at org.checkerframework.checker.resourceleak.MustCallConsistencyAnalyzer.propagateObligationsToSuccessorBlocks(MustCallConsistencyAnalyzer.java:1708)
        at org.checkerframework.checker.resourceleak.MustCallConsistencyAnalyzer.analyze(MustCallConsistencyAnalyzer.java:503)
        at org.checkerframework.checker.resourceleak.ResourceLeakAnnotatedTypeFactory.postAnalyze(ResourceLeakAnnotatedTypeFactory.java:119)
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.analyze(GenericAnnotatedTypeFactory.java:1559)
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.performFlowAnalysis(GenericAnnotatedTypeFactory.java:1394)
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.checkAndPerformFlowAnalysis(GenericAnnotatedTypeFactory.java:1829)
        at org.checkerframework.framework.type.GenericAnnotatedTypeFactory.preProcessClassTree(GenericAnnotatedTypeFactory.java:414)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitClass(BaseTypeVisitor.java:475)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitClass(BaseTypeVisitor.java:180)
        at com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(jdk.compiler@11.0.14.1/JCTree.java:808)

@msridhar thanks Ill try that

[msridhar]

here's another

This case should also (hopefully) be handled by #5107

[msridhar]

@delanym #5107 has landed on the main branch. Is there any chance you could try running the Resource Leak Checker with a snapshot build of Checker Framework that includes the fix? To do so, clone the repository and then run ./gradlew publishToMavenLocal. Assuming that works, you need to change your Maven config to depend on Checker Framework version 3.21.5-SNAPSHOT. If you're able to test this and report back, we would appreciate it.

[delanym]

@msridhar thanks it finished after 14m

[msridhar]

Thanks for reporting back!