Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to resolve UNC-based classpath entries #705

Closed
jdeppe-pivotal opened this issue Sep 26, 2022 · 18 comments
Closed

Unable to resolve UNC-based classpath entries #705

jdeppe-pivotal opened this issue Sep 26, 2022 · 18 comments

Comments

@jdeppe-pivotal
Copy link

Hello Luke!

I have come across a problem where UNC-based classpath entries are not resolved correctly. I think I've boiled it down to this test scenario:

public class Main {
    public static void main(String[] args) throws Exception {
        String packagesToScan = args[0];
        String interfaceClass = args[1];

        ScanResult scanResult = new ClassGraph().acceptPackages(packagesToScan)
                .ignoreParentClassLoaders()
                .overrideClasspath("\\\\192.168.216.1\\workspace\\gemfire\\geode-assembly\\build\\install\\apache-geode\\lib\\spring-shell-1.2.0.RELEASE.jar")
                .enableClassInfo()
                .enableRealtimeLogging()
                .scan(1);

        ClassInfoList classInfoList = scanResult.getClassesImplementing(interfaceClass);

        classInfoList.forEach(x -> System.out.println(x.getName()));
    }
}

When run with these args:

org.springframework.shell org.springframework.shell org.springframework.shell.core.Converter

Produces this log:

Sep 26, 2022 2:09:52 PM nonapi.io.github.classgraph.utils.LogNode flush
INFO: 2022-09-26T14:09:52.636-0700	ClassGraph	ClassGraph version 4.8.149
2022-09-26T14:09:52.637-0700	ClassGraph	Operating system: Windows 10 10.0 amd64
2022-09-26T14:09:52.637-0700	ClassGraph	Java version: 1.8.0_345 / 1.8.0_345-b01 (BellSoft)
2022-09-26T14:09:52.638-0700	ClassGraph	Java home: C:\Users\IEUser\.jdks\liberica-1.8.0_345\jre
2022-09-26T14:09:52.648-0700	ClassGraph	JRE rt.jar:
2022-09-26T14:09:52.648-0700	ClassGraph	-- C:/Users/IEUser/.jdks/liberica-1.8.0_345/jre/lib/rt.jar
2022-09-26T14:09:52.658-0700	ClassGraph	ScanSpec:
2022-09-26T14:09:52.658-0700	ClassGraph	-- packageAcceptReject: accept: ["org.springframework.shell"]; acceptPrefixes: ["", "/", "org.", "org.springframework.", "org.springframework.shell."]
2022-09-26T14:09:52.658-0700	ClassGraph	-- packagePrefixAcceptReject: acceptPrefixes: ["org.springframework.shell."]
2022-09-26T14:09:52.658-0700	ClassGraph	-- pathAcceptReject: accept: ["org/springframework/shell/"]; acceptPrefixes: ["", "/", "org/", "org/springframework/", "org/springframework/shell/"]
2022-09-26T14:09:52.658-0700	ClassGraph	-- pathPrefixAcceptReject: acceptPrefixes: ["org/springframework/shell/"]
2022-09-26T14:09:52.658-0700	ClassGraph	-- classAcceptReject: 
2022-09-26T14:09:52.658-0700	ClassGraph	-- classfilePathAcceptReject: 
2022-09-26T14:09:52.659-0700	ClassGraph	-- classPackageAcceptReject: 
2022-09-26T14:09:52.659-0700	ClassGraph	-- classPackagePathAcceptReject: 
2022-09-26T14:09:52.659-0700	ClassGraph	-- moduleAcceptReject: 
2022-09-26T14:09:52.659-0700	ClassGraph	-- jarAcceptReject: 
2022-09-26T14:09:52.659-0700	ClassGraph	-- classpathElementResourcePathAcceptReject: 
2022-09-26T14:09:52.659-0700	ClassGraph	-- libOrExtJarAcceptReject: 
2022-09-26T14:09:52.660-0700	ClassGraph	-- scanJars: true
2022-09-26T14:09:52.660-0700	ClassGraph	-- scanNestedJars: true
2022-09-26T14:09:52.660-0700	ClassGraph	-- scanDirs: true
2022-09-26T14:09:52.660-0700	ClassGraph	-- scanModules: true
2022-09-26T14:09:52.660-0700	ClassGraph	-- enableClassInfo: true
2022-09-26T14:09:52.660-0700	ClassGraph	-- enableFieldInfo: false
2022-09-26T14:09:52.660-0700	ClassGraph	-- enableMethodInfo: false
2022-09-26T14:09:52.660-0700	ClassGraph	-- enableAnnotationInfo: false
2022-09-26T14:09:52.660-0700	ClassGraph	-- enableStaticFinalFieldConstantInitializerValues: false
2022-09-26T14:09:52.660-0700	ClassGraph	-- enableInterClassDependencies: false
2022-09-26T14:09:52.661-0700	ClassGraph	-- enableExternalClasses: false
2022-09-26T14:09:52.661-0700	ClassGraph	-- enableSystemJarsAndModules: false
2022-09-26T14:09:52.661-0700	ClassGraph	-- ignoreClassVisibility: false
2022-09-26T14:09:52.661-0700	ClassGraph	-- ignoreFieldVisibility: false
2022-09-26T14:09:52.661-0700	ClassGraph	-- ignoreMethodVisibility: false
2022-09-26T14:09:52.661-0700	ClassGraph	-- disableRuntimeInvisibleAnnotations: false
2022-09-26T14:09:52.661-0700	ClassGraph	-- extendScanningUpwardsToExternalClasses: true
2022-09-26T14:09:52.661-0700	ClassGraph	-- allowedURLSchemes: null
2022-09-26T14:09:52.661-0700	ClassGraph	-- addedClassLoaders: null
2022-09-26T14:09:52.661-0700	ClassGraph	-- overrideClassLoaders: null
2022-09-26T14:09:52.661-0700	ClassGraph	-- addedModuleLayers: null
2022-09-26T14:09:52.661-0700	ClassGraph	-- overrideModuleLayers: null
2022-09-26T14:09:52.661-0700	ClassGraph	-- overrideClasspath: [\\192.168.216.1\workspace\gemfire\geode-assembly\build\install\apache-geode\lib\spring-shell-1.2.0.RELEASE.jar]
2022-09-26T14:09:52.661-0700	ClassGraph	-- classpathElementFilters: null
2022-09-26T14:09:52.661-0700	ClassGraph	-- initializeLoadedClasses: false
2022-09-26T14:09:52.662-0700	ClassGraph	-- removeTemporaryFilesAfterScan: false
2022-09-26T14:09:52.662-0700	ClassGraph	-- ignoreParentClassLoaders: true
2022-09-26T14:09:52.662-0700	ClassGraph	-- ignoreParentModuleLayers: false
2022-09-26T14:09:52.662-0700	ClassGraph	-- modulePathInfo: 
2022-09-26T14:09:52.662-0700	ClassGraph	-- maxBufferedJarRAMSize: 67108864
2022-09-26T14:09:52.662-0700	ClassGraph	-- enableMemoryMapping: false
2022-09-26T14:09:52.662-0700	ClassGraph	Number of worker threads: 1
2022-09-26T14:09:52.671-0700	ClassGraph	Finding classpath
2022-09-26T14:09:52.672-0700	ClassGraph	-- Finding classpath and modules
2022-09-26T14:09:52.693-0700	ClassGraph	---- Overriding classpath with: [\\192.168.216.1\workspace\gemfire\geode-assembly\build\install\apache-geode\lib\spring-shell-1.2.0.RELEASE.jar]
2022-09-26T14:09:52.697-0700	ClassGraph	------ Found classpath element: //192.168.216.1/workspace/gemfire/geode-assembly/build/install/apache-geode/lib/spring-shell-1.2.0.RELEASE.jar
2022-09-26T14:09:52.697-0700	ClassGraph	------ WARNING: when the classpath is overridden, there is no guarantee that the classes found by classpath scanning will be the same as the classes loaded by the context classloader
2022-09-26T14:09:52.700-0700	ClassGraph	Opening classpath elements (took 0.107643 sec)
2022-09-26T14:09:52.804-0700	ClassGraph	-- Opening classpath element file://192.168.216.1/workspace/gemfire/geode-assembly/build/install/apache-geode/lib/spring-shell-1.2.0.RELEASE.jar
2022-09-26T14:09:52.805-0700	ClassGraph	---- Opening jar: file://192.168.216.1/workspace/gemfire/geode-assembly/build/install/apache-geode/lib/spring-shell-1.2.0.RELEASE.jar
2022-09-26T14:09:52.807-0700	ClassGraph	------ Could not open jarfile file://192.168.216.1/workspace/gemfire/geode-assembly/build/install/apache-geode/lib/spring-shell-1.2.0.RELEASE.jar : java.io.IOException: Could not get logical zipfile file://192.168.216.1/workspace/gemfire/geode-assembly/build/install/apache-geode/lib/spring-shell-1.2.0.RELEASE.jar : java.io.IOException: Could not get PhysicalZipFile for path /192.168.216.1/workspace/gemfire/geode-assembly/build/install/apache-geode/lib/spring-shell-1.2.0.RELEASE.jar : java.io.FileNotFoundException: File does not exist or cannot be read: C:\192.168.216.1\workspace\gemfire\geode-assembly\build\install\apache-geode\lib\spring-shell-1.2.0.RELEASE.jar
2022-09-26T14:09:52.807-0700	ClassGraph	Finding nested classpath elements
2022-09-26T14:09:52.807-0700	ClassGraph	Final classpath element order:
2022-09-26T14:09:52.808-0700	ClassGraph	Scanning classpath elements (took 0.000397 sec)
2022-09-26T14:09:52.808-0700	ClassGraph	Masking classfiles (took 0.000017 sec)
2022-09-26T14:09:52.810-0700	ClassGraph	Scanning classfiles (took 0.000014 sec)
2022-09-26T14:09:52.810-0700	ClassGraph	Linking related classfiles (took 0.000020 sec)
2022-09-26T14:09:52.814-0700	ClassGraph	Total time: 0.113 sec

As you're probably aware, Java can open UNC-based files. If one passes something like file://192.168.216.1/workspace/somefile to this snippet:

        URI uri = new URI(args[0]);
        Path path = Paths.get(uri);

The path ends up pointing to a WindowsPath object (that looks like \\192.168.216.1\workspace\somefile) and is able to be opened and read.

Thanks for your time!

@jdeppe-pivotal
Copy link
Author

@lukehutch Any chance you could comment on this issue? Thanks!

@lukehutch
Copy link
Member

Hi @jdeppe-pivotal,

I'm sorry it took so long for me to get to this, I have been swamped for the last few months.

I checked in a change -- please test the git head version and let me know if it resolves this issue for you.

@jdeppe-pivotal
Copy link
Author

Thanks for taking a look! Unfortunately, though, the problem still seems to be there. Here is a log from a run using your most recent changes:

Nov 14, 2022 2:25:02 PM nonapi.io.github.classgraph.utils.LogNode flush
INFO: 2022-11-14T14:25:01.977-0800	ClassGraph	ClassGraph version 4.8.150-SNAPSHOT
2022-11-14T14:25:01.977-0800	ClassGraph	Operating system: Windows 10 10.0 amd64
2022-11-14T14:25:01.977-0800	ClassGraph	Java version: 1.8.0_345 / 1.8.0_345-b01 (BellSoft)
2022-11-14T14:25:01.977-0800	ClassGraph	Java home: C:\Users\IEUser\.jdks\liberica-1.8.0_345\jre
2022-11-14T14:25:01.987-0800	ClassGraph	JRE rt.jar:
2022-11-14T14:25:01.987-0800	ClassGraph	-- C:/Users/IEUser/.jdks/liberica-1.8.0_345/jre/lib/rt.jar
2022-11-14T14:25:01.996-0800	ClassGraph	ScanSpec:
2022-11-14T14:25:01.997-0800	ClassGraph	-- packageAcceptReject: accept: ["org.springframework.shell"]; acceptPrefixes: ["", "/", "org.", "org.springframework.", "org.springframework.shell."]
2022-11-14T14:25:01.997-0800	ClassGraph	-- packagePrefixAcceptReject: acceptPrefixes: ["org.springframework.shell."]
2022-11-14T14:25:01.997-0800	ClassGraph	-- pathAcceptReject: accept: ["org/springframework/shell/"]; acceptPrefixes: ["", "/", "org/", "org/springframework/", "org/springframework/shell/"]
2022-11-14T14:25:01.997-0800	ClassGraph	-- pathPrefixAcceptReject: acceptPrefixes: ["org/springframework/shell/"]
2022-11-14T14:25:01.997-0800	ClassGraph	-- classAcceptReject: 
2022-11-14T14:25:01.997-0800	ClassGraph	-- classfilePathAcceptReject: 
2022-11-14T14:25:01.997-0800	ClassGraph	-- classPackageAcceptReject: 
2022-11-14T14:25:01.997-0800	ClassGraph	-- classPackagePathAcceptReject: 
2022-11-14T14:25:01.997-0800	ClassGraph	-- moduleAcceptReject: 
2022-11-14T14:25:01.997-0800	ClassGraph	-- jarAcceptReject: 
2022-11-14T14:25:01.997-0800	ClassGraph	-- classpathElementResourcePathAcceptReject: 
2022-11-14T14:25:01.997-0800	ClassGraph	-- libOrExtJarAcceptReject: 
2022-11-14T14:25:01.998-0800	ClassGraph	-- scanJars: true
2022-11-14T14:25:01.998-0800	ClassGraph	-- scanNestedJars: true
2022-11-14T14:25:01.998-0800	ClassGraph	-- scanDirs: true
2022-11-14T14:25:01.998-0800	ClassGraph	-- scanModules: true
2022-11-14T14:25:01.998-0800	ClassGraph	-- enableClassInfo: true
2022-11-14T14:25:01.998-0800	ClassGraph	-- enableFieldInfo: false
2022-11-14T14:25:01.998-0800	ClassGraph	-- enableMethodInfo: false
2022-11-14T14:25:01.998-0800	ClassGraph	-- enableAnnotationInfo: true
2022-11-14T14:25:01.998-0800	ClassGraph	-- enableStaticFinalFieldConstantInitializerValues: false
2022-11-14T14:25:01.998-0800	ClassGraph	-- enableInterClassDependencies: false
2022-11-14T14:25:01.998-0800	ClassGraph	-- enableExternalClasses: false
2022-11-14T14:25:01.999-0800	ClassGraph	-- enableSystemJarsAndModules: false
2022-11-14T14:25:01.999-0800	ClassGraph	-- ignoreClassVisibility: false
2022-11-14T14:25:01.999-0800	ClassGraph	-- ignoreFieldVisibility: false
2022-11-14T14:25:01.999-0800	ClassGraph	-- ignoreMethodVisibility: false
2022-11-14T14:25:01.999-0800	ClassGraph	-- disableRuntimeInvisibleAnnotations: false
2022-11-14T14:25:01.999-0800	ClassGraph	-- extendScanningUpwardsToExternalClasses: true
2022-11-14T14:25:01.999-0800	ClassGraph	-- allowedURLSchemes: null
2022-11-14T14:25:01.999-0800	ClassGraph	-- addedClassLoaders: null
2022-11-14T14:25:01.999-0800	ClassGraph	-- overrideClassLoaders: null
2022-11-14T14:25:01.999-0800	ClassGraph	-- addedModuleLayers: null
2022-11-14T14:25:01.999-0800	ClassGraph	-- overrideModuleLayers: null
2022-11-14T14:25:01.999-0800	ClassGraph	-- overrideClasspath: [\\192.168.216.1\workspace\gemfire\geode-assembly\build\install\vmware-gemfire\lib\spring-shell-1.2.0.RELEASE.jar]
2022-11-14T14:25:02.000-0800	ClassGraph	-- classpathElementFilters: null
2022-11-14T14:25:02.000-0800	ClassGraph	-- initializeLoadedClasses: false
2022-11-14T14:25:02.000-0800	ClassGraph	-- removeTemporaryFilesAfterScan: false
2022-11-14T14:25:02.000-0800	ClassGraph	-- ignoreParentClassLoaders: false
2022-11-14T14:25:02.000-0800	ClassGraph	-- ignoreParentModuleLayers: false
2022-11-14T14:25:02.000-0800	ClassGraph	-- modulePathInfo: 
2022-11-14T14:25:02.001-0800	ClassGraph	-- maxBufferedJarRAMSize: 67108864
2022-11-14T14:25:02.001-0800	ClassGraph	-- enableMemoryMapping: false
2022-11-14T14:25:02.001-0800	ClassGraph	Number of worker threads: 1
2022-11-14T14:25:02.008-0800	ClassGraph	Finding classpath
2022-11-14T14:25:02.010-0800	ClassGraph	-- Finding classpath and modules
2022-11-14T14:25:02.038-0800	ClassGraph	---- Overriding classpath with: [\\192.168.216.1\workspace\gemfire\geode-assembly\build\install\vmware-gemfire\lib\spring-shell-1.2.0.RELEASE.jar]
2022-11-14T14:25:02.042-0800	ClassGraph	------ Found classpath element: //192.168.216.1/workspace/gemfire/geode-assembly/build/install/vmware-gemfire/lib/spring-shell-1.2.0.RELEASE.jar
2022-11-14T14:25:02.042-0800	ClassGraph	------ WARNING: when the classpath is overridden, there is no guarantee that the classes found by classpath scanning will be the same as the classes loaded by the context classloader
2022-11-14T14:25:02.045-0800	ClassGraph	Opening classpath elements (took 0.126889 sec)
2022-11-14T14:25:02.170-0800	ClassGraph	-- Opening classpath element file://192.168.216.1/workspace/gemfire/geode-assembly/build/install/vmware-gemfire/lib/spring-shell-1.2.0.RELEASE.jar
2022-11-14T14:25:02.170-0800	ClassGraph	---- Opening jar: file://192.168.216.1/workspace/gemfire/geode-assembly/build/install/vmware-gemfire/lib/spring-shell-1.2.0.RELEASE.jar
2022-11-14T14:25:02.172-0800	ClassGraph	------ Could not open jarfile file://192.168.216.1/workspace/gemfire/geode-assembly/build/install/vmware-gemfire/lib/spring-shell-1.2.0.RELEASE.jar : java.io.IOException: Could not get logical zipfile file://192.168.216.1/workspace/gemfire/geode-assembly/build/install/vmware-gemfire/lib/spring-shell-1.2.0.RELEASE.jar : java.io.IOException: Could not get PhysicalZipFile for path /192.168.216.1/workspace/gemfire/geode-assembly/build/install/vmware-gemfire/lib/spring-shell-1.2.0.RELEASE.jar : java.io.FileNotFoundException: File does not exist or cannot be read: C:\192.168.216.1\workspace\gemfire\geode-assembly\build\install\vmware-gemfire\lib\spring-shell-1.2.0.RELEASE.jar
2022-11-14T14:25:02.172-0800	ClassGraph	Finding nested classpath elements
2022-11-14T14:25:02.173-0800	ClassGraph	Final classpath element order:
2022-11-14T14:25:02.173-0800	ClassGraph	Scanning classpath elements (took 0.000368 sec)
2022-11-14T14:25:02.173-0800	ClassGraph	Masking classfiles (took 0.000016 sec)
2022-11-14T14:25:02.175-0800	ClassGraph	Scanning classfiles (took 0.000008 sec)
2022-11-14T14:25:02.175-0800	ClassGraph	Linking related classfiles (took 0.000036 sec)
2022-11-14T14:25:02.178-0800	ClassGraph	Total time: 0.132 sec

@jdeppe-pivotal
Copy link
Author

I left a comment on your code changes: 047e465#r89914945

@jdeppe-pivotal
Copy link
Author

jdeppe-pivotal commented Nov 15, 2022

At that point I tried simply changing Path path = Paths.get(new URI(pathElementResolved)); to Path path = Paths.get(new URI("file:" + pathElementResolved)); but it didn't make a difference. At least the exception wasn't thrown but the only visible difference was that this log line:

------ Found classpath element: //192.168.216.1/workspace/gemfire/geode-assembly/build/install/vmware-gemfire/lib/spring-shell-1.2.0.RELEASE.jar

... changed to:

------ Found classpath element: \\192.168.216.1\workspace\gemfire\geode-assembly\build\install\vmware-gemfire\lib\spring-shell-1.2.0.RELEASE.jar

lukehutch added a commit that referenced this issue Nov 15, 2022
@lukehutch
Copy link
Member

I think I finally fixed it, released in version 4.8.150. (I'll reopen if it still doesn't work for you, but it works in my own testing.)

The trick was to just do new File(uncPathStr).

@jdeppe-pivotal
Copy link
Author

jdeppe-pivotal commented Nov 15, 2022

Unfortunately it's still not working. I think there may be a couple of problems still.
When ClasspathElementZip is initialized (with a UNC path) it does receive an underlying WindowsPath but then it does: rawPath = ((Path) rawPathObj).toUri().toString(); which turns the path into file://192.168.1.1/whatevs. When this, in turn, is then passed to FastPathResolver.resolve, the file:// portion is stripped off and the semantics of a UNC path are lost.

I tried changing the assignment of rawPath to rawPath = ((Path) rawPathObj).toFile().toURI().toString(); which then produces a file:////192.168.1.1/whatevs string representation (I think more correct). However, FastPathResolver.resolver still ends up stripping off all of the file://// portion instead of just the file:// scheme part.

@jdeppe-pivotal
Copy link
Author

jdeppe-pivotal commented Nov 15, 2022

Oh! If I do rawPath = ((Path) rawPathObj).toFile().toString(); my example then works. But then that breaks one of the tests :(. Perhaps there just needs to be special handling for UNC paths in the ClasspathElementZip constructor.

@lukehutch lukehutch reopened this Nov 16, 2022
lukehutch added a commit that referenced this issue Nov 17, 2022
@lukehutch
Copy link
Member

@jdeppe-pivotal OK, I think I may have solved this finally.

The reason paths have to be converted to strings is because I'm trying to support nested jar syntax, e.g. jar:file:/c/Users/Blah/myjar.jar!nestedjar.jar. The part before ! has to be extracted, when present.

For UNC paths, some Java APIs (out of Path, File, URI, URL) require syntax like \\localhost\c$\path and some require \\localhost\c:\path, while others require just \\localhost\c\path... so conversions failed using a bunch of different methods I tried.

I think the result may finally work, please test and let me know.

@lukehutch
Copy link
Member

Ultimately the whole path handling strategy should be overhauled for ClassGraph, but I'm worried I'll break something for some users, so let's try these minimal changes to start with.

@jdeppe-pivotal
Copy link
Author

Looks like that has fixed it! Thanks very much.

lukehutch added a commit that referenced this issue Nov 17, 2022
@lukehutch
Copy link
Member

I fixed a few more issues and released as version 4.8.151, please let me know if it works for you. Thanks.

@jdeppe-pivotal
Copy link
Author

Yes! Looks good. Thanks again, Luke.

@lukehutch
Copy link
Member

Great, thank you for your persistence!

@devlauer
Copy link

@lukehutch : In your last commit a218636 an ClassCastException was introduced if an URLClassLoader is used with direct URLs to class files. In this case the pathElement is a URL an not a Path so an ClassCastException is thrown. Could you add an instanceOf check to the new if clause? Thank you in advance!

lukehutch added a commit that referenced this issue Dec 21, 2022
@lukehutch
Copy link
Member

@devlauer Thanks, fixed. I'll release this soon.

@devlauer
Copy link

@lukehutch : Thank you for the quick fix!!

@lukehutch
Copy link
Member

@devlauer You're welcome, thanks for the report.

There are probably more UNC path issues lurking, since UNC paths are weird, and the handling in the Java libraries is complex. Please report any other issues you find!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants