How to scan my classes loaded by external tomcat servers within the premain method of a Java agent

[601724080]

tomcat server： 8.5.93
platform: win11
project info：spring-boot 2.1.13.RELEASE

When I used version 1, I scanned more than 10000 classes, but when I upgraded to version 4.8.162, I only scanned 5000 classes。
the correct thing is to scan out over 60000 classes，Currently, only JDK, tomcat, and my Java agent's classes are scanned out。

Is it possible to scan my classes loaded by external tomcat servers within the premain method of a Java agent？

[lukehutch]

tomcat server： 8.5.93 platform: win11 project info：spring-boot 2.1.13.RELEASE

When I used version 1, I scanned more than 10000 classes, but when I upgraded to version 4.8.162, I only scanned 5000 classes。 the correct thing is to scan out over 60000 classes，Currently, only JDK, tomcat, and my Java agent's classes are scanned out。

Is it possible to scan my classes loaded by external tomcat servers within the premain method of a Java agent？

You'll need to go through the logs in both versions and figure out which classpath elements are not found in the newer version. Then we can look into why those aren't found in the newer version.

As far as why there are 50k classes are missing even in the older version, it's possible that the Catalina classpath handler is incomplete. If you can figure out by introspection or using a debugger just how those classes are found in your runtime environment, we can extend the Catalina classpath handler to find those too.

[lukehutch]

This is the code that handles Catalina:

classgraph/src/main/java/nonapi/io/github/classgraph/classloaderhandler/TomcatWebappClassLoaderBaseHandler.java

Line 40 in 5f94ed6

/** Extract classpath entries from the Tomcat/Catalina WebappClassLoaderBase. */

[601724080]

Thank you for your suggestion. But I can not debugger in scan method，I don't know what I did wrong.
I wrote the code to reproduce the question；

https://github.com/601724080/scan-external-tomcat-project

[lukehutch]

OK, I'm traveling right now, I'll try to take a look soon.

However you don't need to debug the scan. You just need to figure out which classloader in your runtime environment contains references to the missing classes, by stilopping a "hello world" program at the start of the run, and looking up the classloader(s), then looking at their fields. Then you can look at the code I linked to see why ClassGraph doesn't find those classes.

[lukehutch]

@601724080 I got the Spring Boot app running, but I couldn't figure out how to get the external Tomcat app running. I installed Tomcat and tried installing the Tomcat tools plugin from Eclipse marketplace, but the Tomcat launch profile type did not appear.

[601724080]

@lukehutch en, the new version eclispe should download something to support it,i add the operation method in my reproduction project，you can give it a try.
https://github.com/601724080/scan-external-tomcat-project

[lukehutch]

Hi @601724080 , I followed your steps, but I can't get "Tomcat Server" to appear as a launch configuration type. It's not in the list.

[601724080]

@lukehutch What version of Eclipse are you using? I will use the same version to take a look.

[lukehutch]

2023-06, but I doubt the version of Eclipse matters much...

Can you please try getting the verbose logs in both run types, and attach them here? Call .verbose() before .scan().

[601724080]

the log is too long， so i commit it in my project， the branch_main.log print all my class and the branch_external-tomcat.log print nothing

[lukehutch]

@601724080 I'm sorry it took me so long to respond!

Your branch_main.log shows that this branch is scanning a much larger classpath than branch_external-tomcat.log. You can see the classpath entries that were found, near the top of the logfiles.

The main branch in particular finds D:/workspace/scan-external-tomcat-project/app/target/classes, which it looks like is where your gtx classes are.

In the external branch, can your code find the gtx classes directly, even if ClassGraph can't find them? If so, can you please get a reference to one of these classes, and call .getClassLoader(), then compare that reference with the AppClassLoader that ClassGraph is able to find? It is possible that ClassGraph is not able to find one of the classloaders.

One other thing that is puzzling me is how the main branch is able to find the gtx classfiles, given that the log does not indicate that they were found where they should be:

2023-09-20T10:28:22.655+0800	ClassGraph	---- Scanning Path: D:/workspace/scan-external-tomcat-project/app/target/classes (took 0.016182 sec)
2023-09-20T10:28:22.658+0800	ClassGraph	------ Found resource within accepted package: application.yml
2023-09-20T10:28:22.661+0800	ClassGraph	------ Scanning Path: D:/workspace/scan-external-tomcat-project/app/target/classes/com (took 0.010890 sec)
2023-09-20T10:28:22.662+0800	ClassGraph	-------- Scanning Path: D:/workspace/scan-external-tomcat-project/app/target/classes/com/gtx (took 0.008908 sec)
2023-09-20T10:28:22.664+0800	ClassGraph	---------- Scanning Path: D:/workspace/scan-external-tomcat-project/app/target/classes/com/gtx/myapp (took 0.006859 sec)
2023-09-20T10:28:22.667+0800	ClassGraph	------------ Scanning Path: D:/workspace/scan-external-tomcat-project/app/target/classes/com/gtx/myapp/controller (took 0.000794 sec)
2023-09-20T10:28:22.670+0800	ClassGraph	------------ Scanning Path: D:/workspace/scan-external-tomcat-project/app/target/classes/com/gtx/myapp/service (took 0.000947 sec)

These should show lines like Found classfile within accepted package: XYZ.class. Since those lines are not there in the log, your code should not find these files here:

https://github.com/601724080/scan-external-tomcat-project/blob/main/my-agent/src/main/java/org/gtx/myagent/MyAgent.java#L19

Hmm....

Any more tips as to how I can get the external branch to launch?

[lukehutch]

@601724080 ping...