Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[KSP] dagger-android hanging indefinitely on a large multi module project #4233

Open
wbonnefond opened this issue Jan 31, 2024 · 2 comments
Open

[KSP] dagger-android hanging indefinitely on a large multi module project #4233

wbonnefond opened this issue Jan 31, 2024 · 2 comments

Comments

@wbonnefond
Copy link

I've attempted to switch our project using dagger-android from KAPT to KSP, but have run into issues. After manually updating all internal provides/binds/contributes methods to be public to address this issue: #4193, the project will hang indefinitely on the kspDebugKotlin task in the app/root module. After updating to Kotlin 1.9.22 and KSP 1.9.22-1.0.17 which addresses the internal method issue, the same behavior occurs. It appears that the dagger KSP processor may be getting into an infinite loop somewhere. For reference, this app consists of ~300 modules.

I built dagger from source to try to add some additional logging, but it looks like logs don't reach gradle output until the task ends (I manually kill it) so it makes debugging very difficult. I'm looking for some insight into where the best place to add additional debug logs would be, or if there are any hunches on where the problem might lie.

This is reproducible with both Dagger 2.49 and 2.50
@eboudrant
Copy link

We can reproduce the issue, I was able to get the potential deadlock stacks (from YourKit), see here.

We use Dagger 2.49 (and made the switch to ksp recently).
The issue don't happen to everyone which is strange.
We also have ~300 modules.
kspDebugKotlin is getting stuck in random modules using dagger.

RMI TCP Connection(558)-127.0.0.1  Runnable CPU usage on sample: 1s 2ms
  java.io.BufferedInputStream.read1(BufferedInputStream.java:282)
  java.io.BufferedInputStream.read(BufferedInputStream.java:343)
  java.io.DataInputStream.readFully(DataInputStream.java:201)
  java.io.DataInputStream.readFully(DataInputStream.java:172)
  org.jetbrains.kotlin.com.intellij.util.io.IOUtil.readString(IOUtil.java:41)
  com.google.devtools.ksp.FileKeyDescriptor.read(PersistentMap.kt:39)
  com.google.devtools.ksp.FileKeyDescriptor.read(PersistentMap.kt:37)
  org.jetbrains.kotlin.com.intellij.util.io.keyStorage.AppendableStorageBackedByResizableMappedFile.read(AppendableStorageBackedByResizableMappedFile.java:94)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.findValueFor(PersistentEnumeratorBase.java:438)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.lambda$valueOf$2(PersistentEnumeratorBase.java:430)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase$$Lambda$617.0x000000080192b290.compute()
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.catchCorruption(PersistentEnumeratorBase.java:608)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.valueOf(PersistentEnumeratorBase.java:429)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentBTreeEnumerator.valueOf(PersistentBTreeEnumerator.java:667)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.isKeyAtIndex(PersistentEnumeratorBase.java:373)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentBTreeEnumerator.enumerateImpl(PersistentBTreeEnumerator.java:528)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.lambda$doEnumerate$0(PersistentEnumeratorBase.java:267)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase$$Lambda$572.0x00000008018a3008.compute()
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.catchCorruption(PersistentEnumeratorBase.java:608)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.doEnumerate(PersistentEnumeratorBase.java:266)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.enumerate(PersistentEnumeratorBase.java:279)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.enumerate(PersistentMapImpl.java:425)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.doPut(PersistentMapImpl.java:398)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.put(PersistentMapImpl.java:360)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentHashMap.put(PersistentHashMap.java:102)
  com.google.devtools.ksp.PersistentMap.set(PersistentMap.kt:26)
  com.google.devtools.ksp.IncrementalContextBase.updateCaches$update(IncrementalContextBase.kt:371)
  com.google.devtools.ksp.IncrementalContextBase.updateCaches(IncrementalContextBase.kt:390)
  com.google.devtools.ksp.IncrementalContextBase.updateCachesAndOutputs(IncrementalContextBase.kt:473)
  com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:358)
  org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:112)
  org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:77)
  org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:256)
  org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:247)
  org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:115)
  org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:247)
  org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.repeatAnalysisIfNeeded(KotlinToJVMBytecodeCompiler.kt:181)  <2 recursive calls>
  org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:87)
  org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli$default(KotlinToJVMBytecodeCompiler.kt:43)
  org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:165)
  org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:50)
  org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:104)
  org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:48)
  org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:101)
  org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1523)
  jdk.internal.reflect.GeneratedMethodAccessor96.invoke()
  jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  java.lang.reflect.Method.invoke(Method.java:568)
  sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:360)
  sun.rmi.transport.Transport$1.run(Transport.java:200)
  sun.rmi.transport.Transport$1.run(Transport.java:197)
  java.security.AccessController.executePrivileged(AccessController.java:807)
  java.security.AccessController.doPrivileged(AccessController.java:712)
  sun.rmi.transport.Transport.serviceCall(Transport.java:196)
  sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587)
  sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
  sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705)
  sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$90.0x0000000801085528.run()
  java.security.AccessController.executePrivileged(AccessController.java:776)
  java.security.AccessController.doPrivileged(AccessController.java:399)
  sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704)
  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
  java.lang.Thread.run(Thread.java:833)

RMI TCP Connection(565)-127.0.0.1  Parked CPU usage on sample: 0ms
  jdk.internal.misc.Unsafe.park(Native Method)
  java.util.concurrent.locks.LockSupport.park(LockSupport.java:211)
  java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:715)
  java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:938)
  java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:959)
  org.jetbrains.kotlin.com.intellij.util.io.StorageLockContext.lockWrite(StorageLockContext.java:70)
  org.jetbrains.kotlin.com.intellij.util.io.PagedFileStorage.lockWrite(PagedFileStorage.java:94)
  org.jetbrains.kotlin.com.intellij.util.io.ResizeableMappedFile.lockWrite(ResizeableMappedFile.java:279)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.lockStorageWrite(PersistentEnumeratorBase.java:229)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.<init>(PersistentEnumeratorBase.java:137)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentBTreeEnumerator.<init>(PersistentBTreeEnumerator.java:93)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumerator.createDefaultEnumerator(PersistentEnumerator.java:66)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.<init>(PersistentMapImpl.java:136)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentMapBuilder.buildImplementation(PersistentMapBuilder.java:62)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentMapBuilder.build(PersistentMapBuilder.java:44)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentHashMap.<init>(PersistentHashMap.java:39)
  org.jetbrains.kotlin.com.intellij.util.io.PersistentHashMap.<init>(PersistentHashMap.java:54)
  org.jetbrains.kotlin.incremental.storage.CachingLazyStorage.createMap(CachingLazyStorage.kt:128)
  org.jetbrains.kotlin.incremental.storage.CachingLazyStorage.getStorageIfExists(CachingLazyStorage.kt:47)
  org.jetbrains.kotlin.incremental.storage.CachingLazyStorage.getKeys(CachingLazyStorage.kt:65)
  org.jetbrains.kotlin.incremental.storage.LookupMap.getKeys(LookupMap.kt:56)
  org.jetbrains.kotlin.incremental.TrackedLookupMap.getKeys(LookupStorage.kt:306)
  org.jetbrains.kotlin.incremental.LookupStorage.getLookupSymbols(LookupStorage.kt:89)
  org.jetbrains.kotlin.incremental.classpathDiff.ClasspathSnapshotShrinker$shrinkClasspath$lookupSymbols$1.invoke(ClasspathSnapshotShrinker.kt:37)
  org.jetbrains.kotlin.incremental.classpathDiff.ClasspathSnapshotShrinker$shrinkClasspath$lookupSymbols$1.invoke(ClasspathSnapshotShrinker.kt:36)
  org.jetbrains.kotlin.incremental.classpathDiff.ClasspathSnapshotShrinker$MetricsReporter.getLookupSymbols(ClasspathSnapshotShrinker.kt:330)
  org.jetbrains.kotlin.incremental.classpathDiff.ClasspathSnapshotShrinker.shrinkClasspath(ClasspathSnapshotShrinker.kt:36)
  org.jetbrains.kotlin.incremental.classpathDiff.ClasspathChangesComputer.computeClasspathChanges(ClasspathChangesComputer.kt:50)
  org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.calculateSourcesToCompileImpl(IncrementalJvmCompilerRunner.kt:197)
  org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.calculateSourcesToCompile(IncrementalJvmCompilerRunner.kt:135)
  org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.calculateSourcesToCompile(IncrementalJvmCompilerRunner.kt:62)
  org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.tryCompileIncrementally$lambda$6$compile(IncrementalCompilerRunner.kt:207)
  org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.tryCompileIncrementally(IncrementalCompilerRunner.kt:247)
  org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compile(IncrementalCompilerRunner.kt:116)
  org.jetbrains.kotlin.daemon.CompileServiceImplBase.execIncrementalCompiler(CompileServiceImpl.kt:657)
  org.jetbrains.kotlin.daemon.CompileServiceImplBase.access$execIncrementalCompiler(CompileServiceImpl.kt:105)
  org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1620)
  jdk.internal.reflect.GeneratedMethodAccessor96.invoke()
  jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  java.lang.reflect.Method.invoke(Method.java:568)
  sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:360)
  sun.rmi.transport.Transport$1.run(Transport.java:200)
  sun.rmi.transport.Transport$1.run(Transport.java:197)
  java.security.AccessController.executePrivileged(AccessController.java:807)
  java.security.AccessController.doPrivileged(AccessController.java:712)
  sun.rmi.transport.Transport.serviceCall(Transport.java:196)
  sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587)
  sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
  sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705)
  sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$90.0x0000000801085528.run()
  java.security.AccessController.executePrivileged(AccessController.java:776)
  java.security.AccessController.doPrivileged(AccessController.java:399)
  sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704)
  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
  java.lang.Thread.run(Thread.java:833)

@bcorso
Copy link

bcorso commented Feb 7, 2024

@eboudrant thanks for getting the stack traces. It looks like this is stuck on the KSP side. Do you mind filing a bug with KSP team and giving them these stack traces?

@eboudrant eboudrant mentioned this issue Feb 7, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@eboudrant @wbonnefond @bcorso