scan() can not find classes in .war if acceptedPackages() contains wildcard

[etbasim]

Hello;

Currently we are using 4.4.12 and after i have updated classgraph in latest version we are having trouble with finding classes in war files if acceptedPackages has a wildcard in package name. This is working ok if the '' character is prefix or suffix of package name its working without problem. But if package name is in format of "xxx..xxx" then its not able to find the class. This option is working fine with jar files but not only working for .war files and issue occurs in also for different application servers with WebSphere Traditional and WebSphere Liberty.
As an example of the problem;

ClassLoader classLoader = Thread.currentThread().getContextClassLoader();

        final ClassGraph classGraph = new ClassGraph().enableClassInfo()
                .enableAnnotationInfo()
                .acceptPackages("com.xxx.*.internal.properties")
                .addClassLoader(classLoader);
        ScanResult scanResult = classGraph.verbose().scan(); // can find the classes in .jar but not in .war

You can find a simple producer in the following link.
https://github.com/etbasim/ClassLoaderSample

[lukehutch]

Thank you for the report, and for the complete testcase. I will try to get to this soon, although I am traveling right now with very limited spare time, so it might take a few more days.

[lukehutch]

@etbasim I finally found time to build your example project. I followed the steps on the project page, but in the last step if I do ./gradlew ear, I get only:

$ ./gradlew ear

BUILD SUCCESSFUL in 415ms
5 actionable tasks: 5 up-to-date
$

If I go to https://localhost:9080, it just shows the default OpenLiberty landing page.

What am I missing here? The instructions say

Since it's a Gradle project you just execute ear task to build the project and ear application will deploy directly after build into liberty server since it's already configured.

but that is not what seems to have happened.

[etbasim]

@lukehutch in this example scanning operation is done during war file deployment. When you build the ear file its going to liberty server dropins folder with the volume configuration in docker-compose and liberty listening dropins folder for the ear file change if there is new ear or ear updated its start deploying. RegisterWebResourceContextListener.contextInitialized() is triggered when war file is deploying and scan() is trigger in this listener. So example is doing the class scan on startup and write the scan result to System.out and verbose it. You can find it in console log.

This is the class i am looking for the scan result : com.xxx.web.internal.properties.WebTestProperties

This is the case scan can not find above class

final ClassGraph classGraph = new ClassGraph().enableClassInfo()
                .enableAnnotationInfo()
                .acceptPackages("com.xxx.*.internal.properties")
                .addClassLoader(classLoader);
        ScanResult scanResult = classGraph.verbose().scan();

This is the case scan can find the above class

final ClassGraph classGraphServlet = new ClassGraph().enableClassInfo()
                .enableAnnotationInfo()
                .acceptPackages("com.xxx.web.*");
        ScanResult scanResultServlet = classGraphServlet.scan();

To trigger the deployment again you can simple add or remove empty line build project, since its an almost empty project deployment just take a few seconds. So you don't need to trigger any page from browser but you need to trigger the deployment instead and you can find the result in container console log.

If there is anything to make your analysis easier, please let me know.

[lukehutch]

Thanks for that extra info. How do I check the container log for this? I have zero knowledge of Docker.

[etbasim]

You can see in cmd window after you execute docker-compose up or you can find it under container/logs/messages.log file under project folder.

[lukehutch]

OK, I understand what is going on here.

When scanning jarfiles, the path of every zip entry is matched against the accept/reject list, so a simple conversion of glob-style wildcards into regexps is able to implement the accept/reject criteria.

When scanning files in a directory structure (presumably in this case, including a war-file, if it is exploded/extracted onto the local filesystem), the search must proceed hierarchically.

For a hierarchical search, considerable effort is expended to prune the hierarchical traversal of the filesystem once it is known that there cannot be an accepted resource below the current directory. But we must keep scanning hierarchically if there is any possibility that an accepted resource is below the current directory. It is very difficult to make a pruned search both correct and efficient in the case of accept/reject criteria containing wildcards.

Currently ClassGraph cuts some corners to make common cases work well, such as prefix and suffix searches. But this is not enough to cover your usecase.

I don't know what the right solution is, but I suspect I need to carefully audit the whole accept/reject system to find out exactly how these criteria are used, then rethink how this should proceed.

I think I may simply switch off pruning of recursive scanning once the first wildcard is hit. This would mean that paths containing wildcards may end up scanning a lot more of the filesystem than before.

Another issue is whether I should revamp the wildcard system to support double-globs, ** -- but to do so would break existing usage, because * currently matches multiple path segments.

[etbasim]

Hello again @lukehutch,

Just i would like to check if you have any update about this issue.

[lukehutch]

Not yet, sorry, this will be a significant and complex change, and I'm currently swamped with an upcoming release of another project.

[shani88]

Hello @etbasim ,
Would you like to try the package name com.xxx.*.internal.properties.*?
Tell me if you succeed.

If there is an asterisk in the package name, it seems to work with a simple regular expression process.