Can you dump the bytecode of the instrumented JaCoCoOfflineTester and paste it on a gist? (javap -p -c /path/to/JaCoCoOfflineTester.class)

@hoisie yep, the JaCoCo instrumented classes will locate at integration_tests/jacoco-offline/build/classes/java/classes-instrumented after executing ./gradlew :integration_tests:jacoco-offline:test.

Compiled from "JaCoCoOfflineTester.java"
public class org.robolectric.integrationtests.jacoco.JaCoCoOfflineTester {
  public static final int VALUE;

  private static transient boolean[] $jacocoData;

  public org.robolectric.integrationtests.jacoco.JaCoCoOfflineTester();
    Code:
       0: invokestatic  #25                 // Method $jacocoInit:()[Z
       3: astore_1
       4: aload_0
       5: invokespecial #1                  // Method java/lang/Object."<init>":()V
       8: aload_1
       9: iconst_0
      10: iconst_1
      11: bastore
      12: return

  public int getValue();
    Code:
       0: invokestatic  #25                 // Method $jacocoInit:()[Z
       3: astore_1
       4: iconst_1
       5: aload_1
       6: iconst_1
       7: iconst_1
       8: bastore
       9: ireturn

  private static boolean[] $jacocoInit();
    Code:
       0: getstatic     #29                 // Field $jacocoData:[Z
       3: dup
       4: ifnonnull     21
       7: pop
       8: ldc2_w        #30                 // long 859009245881762949l
      11: ldc           #32                 // String org/robolectric/integrationtests/jacoco/JaCoCoOfflineTester
      13: iconst_2
      14: invokestatic  #38                 // Method org/jacoco/agent/rt/internal_b6258fc/Offline.getProbes:(JLjava/lang/String;I)[Z
      17: dup
      18: putstatic     #29                 // Field $jacocoData:[Z
      21: areturn
}

And when I disabled the JVM bytecode verification by the following:

    testOptions.unitTests.all {
        jvmArgs '-noverify'
    }

I got a NPE:

java.lang.NullPointerException
	at org.robolectric.integrationtests.jacoco.JaCoCoOfflineTester.__constructor__(JaCoCoOfflineTester.java)
	at org.robolectric.integrationtests.jacoco.JaCoCoOfflineTester.<init>(JaCoCoOfflineTester.java:3)
	at org.robolectric.integrationtests.jacoco.ShadowJaCoCoOfflineTesterTest.testGetValue(ShadowJaCoCoOfflineTesterTest.java:18)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.robolectric.RobolectricTestRunner$HelperTestRunner$1.evaluate(RobolectricTestRunner.java:583)
	at org.robolectric.internal.SandboxTestRunner$2.lambda$evaluate$2(SandboxTestRunner.java:287)
	at org.robolectric.internal.bytecode.Sandbox.lambda$runOnMainThread$0(Sandbox.java:99)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

And when I added shadow constructor for custom shadow class ShadowJaCoCoOfflineTester, the test passed.

  public void __constructor__() {
    System.out.println("__constructor__");
  }

From the above, I guess it's because this commit(b35b01c) introduced a call to $robo$init method, but the custom shadow has JaCoCo instrumented constructor and might have no custom constructor __constructor__ which causes a NPE on noverify or bytecode VerifyError without noverify.

@hoisie Hope this can help~

Currently we added logic to skip Robolectric instrumentation on constructors that are already instrumented by Jacoco. I'll try to repro this issue and see if I can track it down.

OK I can repro this issue by checking out the jacoco_offline_integration_tests branch and running /gradlew :integration_tests:jacoco-offline:testDebugUnitTest. Thanks! I anticipate this should be an easy fix.

@ZSmallX would you like to work together on a proper fix for Jacoco-instrumented constructors?

@hoisie Sure, that will be cool! If so, I think I can try to give a PR on it this week based on the reproducer.

@hoisie I added a PR to fix an edge case from JaCoCo offline instrument integration tests. #8015

And added a PR to add JaCoCo on-the-fly instrument integration tests. #8017
It could be a stand-alone PR to help to verify JaCoCo integration and code coverage 0 problem.

CC @utzcoz

@hoisie I came up with a possible generic solution, like:

Checking jacoco instrument by if there are two members in class with being mared as synthetic:

private static field with naming $jacocoData
private static method with naming $jacocoInit
could be a better solution.

According to the JaCoCo docs:

To collect execution data JaCoCo instruments the classes under test which adds two members to the classes: A private static field $jacocoData and a private static method $jacocoInit(). Both members are marked as synthetic.

See https://www.eclemma.org/jacoco/trunk/doc/faq.html

Additional info:

Both JaCoCo agent and offline instrument are sharing the same options, and they instrument code in class granularity. It means that once a class is to be instrumented, all methods of this class are instrumented.

https://www.jacoco.org/jacoco/trunk/doc/agent.html

I raised a question to JaCoCo to find a better solution: https://groups.google.com/g/jacoco/c/dW5kOO3ROEk

Copied from: #8015 (comment)

Thanks for all of your work on this @ZSmallX!

The logic to check if a class was Jacoco-instrumented was designed to be short-term. It was just a short-term solution to skip constructor instrumentation if the constructor was already instrumented by Jacoco. This means that constructors that are instrumented by Jacoco cannot be shadowed at the moment. Not a huge deal, but some teams may have custom shadows that have constructor shadows, and they cannot be run under code coverage.

At this point I think we should fix the Robolectric constructor instrumentation that happens if Jacoco is present.

Looks like when a method is instrumented by Jacoco, the first instructions are something like:

       0: invokestatic  #25                 // Method $jacocoInit:()[Z
       3: astore_1

So this means that when we split up the constructor during Robolectric instrumentation (details here), we may be able to add these two instructions to the original constructor method (named method in the code). I can provide a little more details tomorrow with some bytecode dumping.

Just for some extra context, this is what the public Matrix(Matrix m) constructor is split up to during Robolectric instrumentation:

   // the original Matrix(matrix) constructor. This is the method that has the broken Jacoco load instructions. This
   // is where we need to add the ` invokestatic  jacocoInit` and astore_(whatever)
   private void $$robo$$android_graphics_Matrix$__constructor__(Matrix src) {
      this.native_instance = nCreate(src != null ? src.native_instance : 0L);
      Matrix.NoImagePreloadHolder.sRegistry.registerNativeAllocation(this, this.native_instance);
   }
   
   
   // The shadowable part of the constructor
   private void __constructor__(Matrix var1) {
      this.$$robo$$android_graphics_Matrix$__constructor__(var1);
   }
 
 // The invokedynamic delegate that also contains the extracted call to super. This currently contains a valid 
// ` invokestatic  jacocoInit` and astore_(whatever).
   public Matrix(Matrix var1) {
      this.$$robo$init();
      this.__constructor__<invokedynamic>(this, var1);
   }

So after this line, we can detect if the method is Jacoco-instrumented and add a call to $jacocoInit as the first instruction followed by the astore_<n>, where n is the local used to store the special Jacoco array.

@hoisie What about requesting a new API from JaCoCo team that allows us to inject/instrument with JaCoCo correctly? @ZSmallX has shared a great idea to let JaCoCo help us to add special field that can help us to detect whether one class is instrumented by JaCoCo. I think @ZSmallX can share more details here.

@utzcoz in my opinion that may be a bit overkill. I do not expect Jacoco instrumentation to change much. And it is quite simple to detect (e.g. check for a static method call and a load instruction).

Yeah, Detecting by static members or instructions could be enough at this time and it's simple.

It's a trade-off here.
But as we know this detecting are based on JaCoCo internal implementation, it could bring some possible side effects.
And from the context, detecting JaCoCo is only just one small part of this issue that was designed for skip in some edge cases.

I would say it's "recommend" more rather than "request" that JaCoCo can provide a formal or unified way(e.g. flag or API) to support the detecting of all bytecode manipulation frameworks. It would make things even more simple and reliable.

IMO, maybe we should focus more on the instructions compatibility with JaCoCo on constructing Robolectric instrumentations?

I raised a PR to give the completed integration tests on both JaCoCo offline and on-the-fly instrument. #8021

And it seems I commited my WIP ClassInstrumentor patch(still skip JaCoCo instrumented constructor) by mistake and was merged in #8017

So the JaCoCo offline tests are passed in #8021 and can be used as regression tests.

And it seems I commited my WIP ClassInstrumentor patch(still skip JaCoCo instrumented constructor) by mistake and was merged in #8017

Never mind. We can continue improve it if need with new PRs. I think there are many days left for us to improve it before 4.10 released.

So after this line, we can detect if the method is Jacoco-instrumented and add a call to $jacocoInit as the first instruction followed by the astore_<n>, where n is the local used to store the special Jacoco array.

Thanks for the details and example, which make it vivid! @hoisie
After all the context and some related commits like:

• skip desugared and proguarded
• skip such as Kotlin-compiled classes, desugard classes, or classes
  instrumented by tools like Jacoco or Proguard

I tried to strip JaCoCo constructor probes data initialization instructions from callSuper instructions in this case and insert these instructions into the beginning of the original constructor.
And it works as in PR: #8057

But as a fresh new, I have some questions like:
How can we make sure the instructions compatibility when the original constructor's instructions are splited into 2 methods based on the "line"(call super() or this())?
In some cases, the n in astore_<n> could matter if the astore and aload are separared into 2 methods. Like JaCoCo.

From my perspective, I have no ideas whether this is a common case for some compilers or tools.

@ZSmallX Does 4.10 fix this issue? I think your fix and @hoisie 's fix were merged into 4.10.

@ZSmallX I think it has been fixed. Feel free to reopen it if some tests didn't work with 4.10.2.