Target platform resolution accessing eclipse.org through HTTP instead of HTTPS

[eliericha]

Hello folks,

I'm building a Tycho-based project on a machine where only HTTPS connections are allowed. I'm using a .target file where I used HTTPS for all URLs, but I'm still seeing failed attempts to the following HTTP URL:

!ENTRY org.eclipse.equinox.p2.artifact.repository 4 0 2021-09-15 15:59:14.782
!MESSAGE Error processing mirrors URL: http://www.eclipse.org/downloads/download.php?format=xml&file=/releases/2021-03/202103171000&countryCode=us&timeZone=0&format=xml
!STACK 1
org.eclipse.core.runtime.CoreException: Unable to read repository at http://www.eclipse.org/downloads/download.php?format=xml&file=/releases/2021-03/202103171000&countryCode=us&timeZone=0&format=xml.
        at org.eclipse.equinox.internal.p2.transport.ecf.RepositoryTransport.stream(RepositoryTransport.java:196)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.getMirrorsDocument(MirrorSelector.java:294)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.computeMirrors(MirrorSelector.java:259)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.initMirrors(MirrorSelector.java:349)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.selectMirror(MirrorSelector.java:417)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.getMirrorLocation(MirrorSelector.java:327)
 ...
Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to www.eclipse.org:80 [www.eclipse.org/198.41.30.198] failed: connect timed out
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:374)
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
...
Caused by: java.net.SocketTimeoutException: connect timed out
        at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
        at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
        at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
...

I believe the resolver is trying to fetch a list of mirrors at http://www.eclipse.org/downloads/download.php?format=xml&file=/releases/2021-03/202103171000&countryCode=us&timeZone=0&format=xml so it can later on use mirrors for downloading artifacts.

I just tried the above URL with HTTPS and it also works. So:

1. Is this URL hard-coded in the Tycho codebase? Could we update it to use HTTPS?
2. Is there a way to change this URL without a Tycho update, perhaps through a Java property?
3. Is there a way to disable P2 repository mirrors globally such that the code no longer tries to access the above URL?

Thanks!

Elie Richa

[mickaelistria]

Can you please share a minimal project to reproduce this issue.

[eliericha]

Here's a tiny self-contained reproducer: tycho-repr.zip

Running mvn compile triggers the resolution of the target platform.

On a machine where HTTP requests are blocked, the process takes a long time and yields a timeout on the URL http://www.eclipse.org/downloads/download.php?format=xml&file=/releases/2021-03/202103171000&countryCode=us&timeZone=0&format=xml. Running mvn -X compile should give a backtrace similar to my initial message.

On a machine where HTTP requests are allowed, the process completes quickly and successfully.

[mickaelistria]

I see the pom file references http://download.eclipse.org/releases/latest ; what happens if you use https there instead?

[eliericha]

Apologies, my first reproducer was incorrect. I was indeed using HTTP explicitly which obviously causes the timeout.

I now updated the reproducer as follows:

1. I changed the repository URL to HTTPS
2. I added a required bundle to the MANIFEST to cause Tycho to download something from the repository

Now running mvn compile causes long waits before each bundle can be downloaded because Tycho tries to download the list of mirrors through HTTP before each attempt to download a JAR. Without -X the timeout exceptions are hidden and we just see that it takes a long time. Note the 4min elapsed before each JAR is fetched.

$ mvn -B compile | ts
Sep 16 10:30:16 [INFO] Scanning for projects...
Sep 16 10:30:18 [INFO] Computing target platform for MavenProject: com.example:tycho-repr:1.0.0-SNAPSHOT @ /home/richa/tycho-repr/pom.xml
Sep 16 10:30:19 [INFO] Fetching p2.index from https://download.eclipse.org/releases/latest/ (122B)
Sep 16 10:30:19 [INFO] Adding repository https://download.eclipse.org/releases/latest
Sep 16 10:30:20 [INFO] Fetching p2.index from https://download.eclipse.org/releases/2021-09/ (122B)
Sep 16 10:30:23 [INFO] Fetching p2.index from https://download.eclipse.org/releases/2021-09/202109151000/ (140B)
Sep 16 10:30:25 [INFO] Resolving dependencies of MavenProject: com.example:tycho-repr:1.0.0-SNAPSHOT @ /home/richa/tycho-repr/pom.xml
Sep 16 10:30:25 [INFO] Resolving class path of MavenProject: com.example:tycho-repr:1.0.0-SNAPSHOT @ /home/richa/tycho-repr/pom.xml
Sep 16 10:34:30 [INFO] Fetching org.eclipse.equinox.common_3.15.0.v20210518-0604.jar from https://download.eclipse.org/releases/2021-09/202109151000/plugins/ (139.68kB)
Sep 16 10:38:32 [INFO] Fetching org.eclipse.core.jobs_3.12.0.v20210723-1034.jar from https://download.eclipse.org/releases/2021-09/202109151000/plugins/ (108.05kB)
^C

Running mvn -X compile shows the timeout exceptions due to accessing HTTP. I think that every time I see the following pair of exceptions with different backtraces. I think the second one is more useful.

!ENTRY org.eclipse.equinox.p2.transport.ecf 2 0 2021-09-16 10:34:22.505
!MESSAGE Connection to http://www.eclipse.org/downloads/download.php?format=xml&file=/releases/2021-09/202109151000&countryCode=us&timeZone=0&format=xml failed on Connect to www.eclipse.org:80 [www.eclipse.org/198.41.30.198] failed: connect timed out. Retry attempt 0 started
!STACK 0
org.apache.http.conn.ConnectTimeoutException: Connect to www.eclipse.org:80 [www.eclipse.org/198.41.30.198] failed: connect timed out
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.eclipse.ecf.provider.filetransfer.httpclient45.HttpClientRetrieveFileTransfer.performConnect(HttpClientRetrieveFileTransfer.java:1003)
        at org.eclipse.ecf.provider.filetransfer.httpclient45.HttpClientRetrieveFileTransfer.access$0(HttpClientRetrieveFileTransfer.java:995)
        at org.eclipse.ecf.provider.filetransfer.httpclient45.HttpClientRetrieveFileTransfer$1.performFileTransfer(HttpClientRetrieveFileTransfer.java:991)
        at org.eclipse.ecf.filetransfer.FileTransferJob.run(FileTransferJob.java:76)
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

!ENTRY org.eclipse.equinox.p2.artifact.repository 4 0 2021-09-16 10:36:22.820
!MESSAGE Error processing mirrors URL: http://www.eclipse.org/downloads/download.php?format=xml&file=/releases/2021-09/202109151000&countryCode=us&timeZone=0&format=xml
!STACK 1
org.eclipse.core.runtime.CoreException: Unable to read repository at http://www.eclipse.org/downloads/download.php?format=xml&file=/releases/2021-09/202109151000&countryCode=us&timeZone=0&format=xml.
        at org.eclipse.equinox.internal.p2.transport.ecf.RepositoryTransport.stream(RepositoryTransport.java:196)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.getMirrorsDocument(MirrorSelector.java:294)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.computeMirrors(MirrorSelector.java:259)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.initMirrors(MirrorSelector.java:349)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.selectMirror(MirrorSelector.java:417)
        at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.getMirrorLocation(MirrorSelector.java:327)
        at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getMirror(SimpleArtifactRepository.java:757)
        at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.downloadArtifact(SimpleArtifactRepository.java:654)
        at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:787)
        at org.eclipse.tycho.repository.p2base.artifact.repository.RepositoryArtifactProvider.getArtifactFromOneMirror(RepositoryArtifactProvider.java:247)
        at org.eclipse.tycho.repository.p2base.artifact.repository.RepositoryArtifactProvider.getArtifactFromAnyMirror(RepositoryArtifactProvider.java:227)
        at org.eclipse.tycho.repository.p2base.artifact.repository.RepositoryArtifactProvider$1.perform(RepositoryArtifactProvider.java:202)
        at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:770)
        at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifacts(SimpleArtifactRepository.java:847)
        at org.eclipse.equinox.internal.p2.artifact.repository.CompositeArtifactRepository.getArtifacts(CompositeArtifactRepository.java:362)
        at org.eclipse.equinox.internal.p2.artifact.repository.CompositeArtifactRepository.getArtifacts(CompositeArtifactRepository.java:362)
        at org.eclipse.tycho.repository.p2base.artifact.repository.LazyArtifactRepository.getArtifacts(LazyArtifactRepository.java:102)
        at org.eclipse.tycho.repository.p2base.artifact.repository.RepositoryArtifactProvider.getArtifactFromAnyChildRepository(RepositoryArtifactProvider.java:214)
        at org.eclipse.tycho.repository.p2base.artifact.repository.RepositoryArtifactProvider.getArtifactFromAnyFormatAvailableInRepository(RepositoryArtifactProvider.java:181)
        at org.eclipse.tycho.repository.p2base.artifact.repository.RepositoryArtifactProvider.getArtifactFromAnySource(RepositoryArtifactProvider.java:167)
        at org.eclipse.tycho.repository.p2base.artifact.provider.CompositeArtifactProviderBaseImpl.getArtifact(CompositeArtifactProviderBaseImpl.java:54)
        at org.eclipse.tycho.repository.local.MirroringArtifactProvider.downloadCanonicalArtifact(MirroringArtifactProvider.java:255)
        at org.eclipse.tycho.repository.local.MirroringArtifactProvider.downloadMostSpecificNeededFormatOfArtifact(MirroringArtifactProvider.java:248)
        at org.eclipse.tycho.repository.local.MirroringArtifactProvider.downloadArtifact(MirroringArtifactProvider.java:233)
        at org.eclipse.tycho.repository.local.MirroringArtifactProvider.makeOneFormatLocallyAvailable(MirroringArtifactProvider.java:218)
        at org.eclipse.tycho.repository.local.MirroringArtifactProvider.makeLocallyAvailable(MirroringArtifactProvider.java:185)
        at org.eclipse.tycho.repository.local.MirroringArtifactProvider.getArtifactFile(MirroringArtifactProvider.java:124)
        at org.eclipse.tycho.repository.p2base.artifact.provider.CompositeArtifactProvider.getArtifactFile(CompositeArtifactProvider.java:87)
        at org.eclipse.tycho.p2.target.TargetPlatformBaseImpl.getLocalArtifactFile(TargetPlatformBaseImpl.java:130)
        at org.eclipse.tycho.p2.resolver.DefaultP2ResolutionResult.lambda$0(DefaultP2ResolutionResult.java:58)
        at org.eclipse.tycho.p2.resolver.DefaultP2ResolutionResultEntry.getLocation(DefaultP2ResolutionResultEntry.java:73)
        at org.eclipse.tycho.p2.resolver.P2DependencyResolver.lambda$newDefaultTargetPlatform$2(P2DependencyResolver.java:450)
        at org.eclipse.tycho.core.osgitools.targetplatform.ArtifactCollection.lambda$addArtifactFile$2(ArtifactCollection.java:66)
        at org.eclipse.tycho.core.osgitools.DefaultArtifactDescriptor.getLocation(DefaultArtifactDescriptor.java:62)
        at org.eclipse.tycho.core.osgitools.targetplatform.ArtifactCollection.lambda$addArtifact$3(ArtifactCollection.java:121)
        at org.eclipse.tycho.core.osgitools.DefaultArtifactDescriptor.getLocation(DefaultArtifactDescriptor.java:62)
        at org.eclipse.tycho.core.osgitools.targetplatform.ArtifactCollection.lambda$addArtifact$3(ArtifactCollection.java:121)
        at org.eclipse.tycho.core.osgitools.DefaultArtifactDescriptor.getLocation(DefaultArtifactDescriptor.java:62)
        at org.eclipse.tycho.core.osgitools.EquinoxResolver.newState(EquinoxResolver.java:296)
        at org.eclipse.tycho.core.osgitools.EquinoxResolver.newResolvedState(EquinoxResolver.java:91)
        at org.eclipse.tycho.core.osgitools.OsgiBundleProject.getResolverState(OsgiBundleProject.java:326)
        at org.eclipse.tycho.core.osgitools.OsgiBundleProject.resolveClassPath(OsgiBundleProject.java:196)
        at org.eclipse.tycho.core.resolver.DefaultTychoResolver.resolveProject(DefaultTychoResolver.java:173)
        at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.lambda$resolveProjects$0(TychoMavenLifecycleParticipant.java:127)

I hope this helps. Let me know if you need more information.

[mickaelistria]

The artifacts.* files in https://download.eclipse.org/releases/2021-09/202109151000/ do contain a reference to the mirror list in the http form. I think it can be perceived as a bug in the p2 repository which should be full https. Please report this issue at https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Community&component=Cross-Project

[eliericha]

Done.

Thanks for the quick response!