[MNG-7432] Resolver session contains non-MavenWorkspaceReader #695

laeubi · 2022-03-15T09:27:15Z

Signed-off-by: Christoph Läubrich christoph@laeubi-soft.de

Following this checklist to help us incorporate your
contribution quickly and easily:

Make sure there is a JIRA issue filed
for the change (usually before you start working on it). Trivial changes like typos do not
require a JIRA issue. Your pull request should address just this issue, without
pulling in other changes.
Each commit in the pull request should have a meaningful subject line and body.
Format the pull request title like [MNG-XXX] SUMMARY, where you replace MNG-XXX
and SUMMARY with the appropriate JIRA issue. Best practice is to use the JIRA issue
title in the pull request title and in the first line of the commit message.
Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
Run mvn clean verify to make sure basic checks pass. A more thorough check will
be performed on your pull request automatically.
You have run the Core IT successfully.

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.

To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004
In any other case, please file an Apache Individual Contributor License Agreement.

laeubi · 2022-03-15T09:28:31Z

FYI @famod @olamy @michael-o

laeubi · 2022-03-15T09:29:45Z

~~I have copied the code from org.eclipse.aether.util.repository.ChainedWorkspaceReader because it is a final class.~~

I'm using a delegation to the original org.eclipse.aether.util.repository.ChainedWorkspaceReadernow.

famod · 2022-03-15T21:35:25Z

I've just created a ticket for this: https://issues.apache.org/jira/browse/MNG-7432

Signed-off-by: Christoph Läubrich <christoph@laeubi-soft.de>

laeubi · 2022-03-18T10:13:54Z

@michael-o do you like to review this?

michael-o · 2022-03-18T10:36:34Z

@michael-o do you like to review this?

I need to understand the cause first because there seems to be some quirk, no?

laeubi · 2022-03-18T12:01:34Z

Not sure, at least this does restores the previous behavior.
The problem is caused by the following (we have noticed something similar in Tycho as well):

MavenWorkspaceReader has the method findModel
But the chained reader has not implemented this
The reactor reader implement it on the other hand
at two places in maven it is explicitly queried for the MavenWorkspaceReader#findModel what produces slightly different results:
Lets assume we have a dependency build by the current reactor and referenced in another project, then the model might be read again with different outcome (e.g. because of https://www.mojohaus.org/flatten-maven-plugin/plugin-info.html used)
When using profiles, it seems to modify the in memory model returned by MavenWorkspaceReader#findModel, but if that is read from disk the additions from the profile are missing

The fix do the following:

return the ReactorWorspaceReader if it is the only WSR (as in Maven 3.8.4)
implement MavenWorkspaceReader#findModel in a chained mode

this should fix all this issues.

basil · 2022-03-19T22:34:52Z

@olamy The regression introduced in Maven 3.8.5 in 6f14196, which is resolved in this unmerged and unreleased pull request, is causing all Jenkins core PR builds to fail.

basil · 2022-03-19T22:44:22Z

A more minimal (and therefore likely less controversial) alternative to the fix proposed in this PR is:

diff --git a/maven-core/src/main/java/org/apache/maven/DefaultMaven.java b/maven-core/src/main/java/org/apache/maven/DefaultMaven.java
index 56a42b724..965345394 100644
--- a/maven-core/src/main/java/org/apache/maven/DefaultMaven.java
+++ b/maven-core/src/main/java/org/apache/maven/DefaultMaven.java
@@ -377,8 +377,15 @@ private void setupWorkspaceReader( MavenSession session, DefaultRepositorySystem
             }
             workspaceReaders.add( workspaceReader );
         }
-        WorkspaceReader[] readers = workspaceReaders.toArray( new WorkspaceReader[0] );
-        repoSession.setWorkspaceReader( new ChainedWorkspaceReader( readers ) );
+        if ( workspaceReaders.size() == 1 )
+        {
+            repoSession.setWorkspaceReader( workspaceReaders.get( 0 ) );
+        }
+        else
+        {
+            WorkspaceReader[] readers = workspaceReaders.toArray( new WorkspaceReader[0] );
+            repoSession.setWorkspaceReader( new ChainedWorkspaceReader( readers ) );
+        }
 
     }

This fixes the problem for me by restoring the old behavior from Maven 3.8.4.

laeubi · 2022-03-20T05:49:06Z

This fixes the problem for me

but will instantly fail once another workspace reader is registered again ;-)

basil · 2022-03-20T05:57:41Z

I do not really know or care what a workspace reader is. I just want my builds to work again.

laeubi · 2022-03-20T06:06:23Z

I do not really know or care what a workspace reader is. I just want my builds to work again.

You can use maven 3.8.4 in the meantime.

basil · 2022-03-20T06:19:03Z

I think that writing "restoring the old behavior from Maven 3.8.4" should have made it clear that I already knew that.

laeubi · 2022-03-20T07:33:52Z

I think that writing "restoring the old behavior from Maven 3.8.4" should have made it clear that I already knew that.

Sure but if you simply "do not really care" the easiest and fastest way is using 3.8.4. For 3.8.5 support for additional Readers was added because there are people who care, and applying a fix that would only "restoring the old behavior" (that was already broken but probably never noticed) won't help much.

I also don't see that the fix is controversial just that @michael-o want to better understand the cause, so if you can provide additional information why your build fails e.g. by providing a minimal reproducer, integration or unit test or you can confirm that it is the same problem described by @famod then it might help to make more progress here.

basil · 2022-03-20T17:13:29Z

so if you can provide

Sorry, I lack the time and interest to pursue this further. I have already reverted back to Maven 3.8.4.

jglick · 2022-03-21T14:57:50Z

I suppose

maven/maven-core/src/main/java/org/apache/maven/internal/aether/MavenChainedWorkspaceReader.java

Lines 99 to 102 in af9c75e

    
           if ( readers.length == 1 ) 
        
           { 
        
               return readers[0]; 
        
           }

in this PR effectively restores previous behavior in the normal case, just like #695 (comment)?

laeubi · 2022-03-21T17:32:48Z

I suppose

maven/maven-core/src/main/java/org/apache/maven/internal/aether/MavenChainedWorkspaceReader.java

Lines 99 to 102 in af9c75e

if ( readers.length == 1 )

{

return readers[0];

}

in this PR effectively restores previous behavior in the normal case, just like #695 (comment)?

No, this is just a shorthand in case there is only one WSR.

laeubi · 2022-03-29T02:12:17Z

@michael-o just wondering inf we can proceed here? If its fine for 3.8.x I'll prepare PRs for 2.9.x and 4.x as well ... I also wonder if it would be suitable to release a 3.8.6 version or do we have to wait for 2.9.0?

jglick

Seems like a straightforward fix. If I understand correctly, one or both of

maven/maven-resolver-provider/src/main/java/org/apache/maven/repository/internal/DefaultArtifactDescriptorReader.java

Lines 208 to 215 in b3d5c12

    
           if ( workspace instanceof MavenWorkspaceReader ) 
        
           { 
        
               model = ( (MavenWorkspaceReader) workspace ).findModel( pomArtifact ); 
        
               if ( model != null ) 
        
               { 
        
                   return model; 
        
               } 
        
           }

or

maven/maven-core/src/main/java/org/apache/maven/project/artifact/MavenMetadataSource.java

Lines 187 to 190 in 69d6c6d

    
           if ( workspace instanceof MavenWorkspaceReader ) 
        
           { 
        
               model = ( (MavenWorkspaceReader) workspace ).findModel( RepositoryUtils.toArtifact( artifact ) ); 
        
           }

broke in 3.8.5? So merging this towards 3.8.6 would restore the 3.8.4 behavior.

laeubi · 2022-03-29T12:24:05Z

If I understand correctly, one or both of

Exactly!

michael-o · 2022-03-29T12:32:41Z

Is there an IT for this? If there would be, I'd check this and add for 3.8.6+.

laeubi · 2022-03-29T13:11:21Z

Is there an IT for this? If there would be, I'd check this and add for 3.8.6+.

The jira has a link to a reproducer maybe this could be integrated an in itest or give a hint how to create one?

jglick · 2022-03-29T13:12:43Z

For reference, the Jenkins build error was at

maven/maven-resolver-provider/src/main/java/org/apache/maven/repository/internal/DefaultArtifactDescriptorReader.java

Line 302 in 3599d34

throw new ArtifactDescriptorException( result );

famod · 2022-03-30T21:27:38Z

@laeubi

Is there an IT for this? If there would be, I'd check this and add for 3.8.6+.

The jira has a link to a reproducer maybe this could be integrated an in itest or give a hint how to create one?

ITs are here: https://github.com/apache/maven-integration-testing

cstamas · 2022-04-02T13:32:52Z

maven-core/src/main/java/org/apache/maven/internal/aether/MavenChainedWorkspaceReader.java

+    public static WorkspaceReader of( Collection<WorkspaceReader> workspaceReaderCollection )
+    {
+        WorkspaceReader[] readers = workspaceReaderCollection.toArray( new WorkspaceReader[0] );
+        if ( readers.length == 1 )


I'd personally just remove this length check, and make it always wrapped (is simpler).

I think the complexity is manageable given that in most cases there is only one WSP and we save useless loop calls in that case when it is used across the maven build.

~~Still, the check could happen then on collection, not after conversion to array, no?~~ nvm, am fine with it as it is

Yes, I just used the "toArray" as we accept a collection and then getting the first element from collection requires to iterate over the collection, but if size is larger requires an toArray as well... in the end it dosn't make much a difference given that this only happens once per maven invocation.

cstamas · 2022-04-02T13:33:30Z

Added some explanations here: https://issues.apache.org/jira/browse/MNG-7432?focusedCommentId=17516300&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17516300

cstamas · 2022-04-02T15:27:11Z

Created UT that IMHO should cover the issue #706
Please reuse it freely, no need to keep author etc (just lift it from there)

This UT makes sure that DefaultMaven sets up proper type of WorkspaceReader in RepositorySystemSession. For example, this UT fails with current maven-3.8.x and maven-3.9.x branches.

laeubi · 2022-04-04T14:30:38Z

@cstamas I have now fetched your test as well into this PR.

cstamas · 2022-04-07T06:24:59Z

@michael-o so what's blocking here? Should I take over?

michael-o · 2022-04-07T06:28:06Z

Feel free to take over the entire ticket handling with IT, etc.

cstamas · 2022-04-07T06:31:55Z

Ok, but IMHO no need for IT here, as UT covers nicely (again: when resolver runs "within maven", the session.workspaceReader MUST be instanceof MavenWorkspaceReader to get expected -- stuff from reactor -- result, that's all)

michael-o · 2022-04-07T07:24:26Z

Ok, but IMHO no need for IT here, as UT covers nicely (again: when resolver runs "within maven", the session.workspaceReader MUST be instanceof MavenWorkspaceReader to get expected -- stuff from reactor -- result, that's all)

So, you even wouldn't consider to absorb the reproducer as an IT?

cstamas · 2022-04-07T07:27:15Z

As I commented on JIRA, I consider the reproducer WRONG USECASE, and if we lift that into ITs we would put it in concrete...

michael-o · 2022-04-07T07:38:16Z

As I commented on JIRA, I consider the reproducer WRONG USECASE, and if we lift that into ITs we would put it in concrete...

I go with that explanation

As Resolver session contains non-MavenWorkspaceReader, the reactor models (already resolved w/ profiles applied) are re-built when using Resolver within Mojo, instead to get them via ReactorReader as expected. The rebuilt models will lack explicit (-P on CLI) profiles applied, as resolver itself is not maven aware, hence there is no way to "tell" resolver to apply them. Building reactor models w/ profiles applied is NOT done using resolver, but by Maven when loading up reactor, as profiles are NOT applied for downstream transitive dependencies (see discussion on MNG-1388 why). Signed-off-by: Christoph Läubrich <christoph@laeubi-soft.de> Co-authored-by: Christoph Läubrich <christoph@laeubi-soft.de> Co-authored-by: Tamas Cservenak <tamas@cservenak.net>

…#695) As Resolver session contains non-MavenWorkspaceReader, the reactor models (already resolved w/ profiles applied) are re-built when using Resolver within Mojo, instead to get them via ReactorReader as expected. The rebuilt models will lack explicit (-P on CLI) profiles applied, as resolver itself is not maven aware, hence there is no way to "tell" resolver to apply them. Building reactor models w/ profiles applied is NOT done using resolver, but by Maven when loading up reactor, as profiles are NOT applied for downstream transitive dependencies (see discussion on MNG-1388 why). Signed-off-by: Christoph Läubrich <christoph@laeubi-soft.de> Co-authored-by: Christoph Läubrich <christoph@laeubi-soft.de> Co-authored-by: Tamas Cservenak <tamas@cservenak.net>

laeubi force-pushed the quickfix_quarkus branch 2 times, most recently from 6aea07d to 3a72f60 Compare March 15, 2022 09:55

laeubi mentioned this pull request Mar 18, 2022

Bump maven-version from 3.8.4 to 3.8.5 eclipse-tycho/tycho#789

Merged

laeubi changed the title ~~Quick Fix for https://github.com/quarkusio/quarkus/pull/24285~~ [MNG-7432] Dependencies from profile not picked up via -P<profileName> Mar 18, 2022

MNG-7432 - Dependencies from profile not picked up via -P<profileName>

af9c75e

Signed-off-by: Christoph Läubrich <christoph@laeubi-soft.de>

laeubi force-pushed the quickfix_quarkus branch from 3a72f60 to af9c75e Compare March 18, 2022 10:12

basil mentioned this pull request Mar 19, 2022

Upgrade to Maven 3.8.5 blocks Jenkins core development jenkins-infra/helpdesk#2835

Closed

jglick approved these changes Mar 29, 2022

View reviewed changes

cstamas reviewed Apr 2, 2022

View reviewed changes

cstamas approved these changes Apr 2, 2022

View reviewed changes

[MNG-7432] Proposed UT for maven-3.9.x and 3.8.x

bdae903

This UT makes sure that DefaultMaven sets up proper type of WorkspaceReader in RepositorySystemSession. For example, this UT fails with current maven-3.8.x and maven-3.9.x branches.

laeubi requested a review from cstamas April 4, 2022 14:30

cstamas approved these changes Apr 4, 2022

View reviewed changes

laeubi mentioned this pull request Apr 7, 2022

Provide a Tycho Workspace Reader eclipse-tycho/tycho#586

Merged

cstamas changed the title ~~[MNG-7432] Dependencies from profile not picked up via -P<profileName>~~ [MNG-7432] Resolver session contains non-MavenWorkspaceReader Apr 8, 2022

cstamas merged commit 957b5e3 into apache:maven-3.8.x Apr 8, 2022

jglick mentioned this pull request Jun 3, 2022

Maven385 apache/maven-mvnd#616

Merged

gnodet mentioned this pull request Oct 5, 2022

Make sure RAT plugin ignores files in the target folder #814

Merged

gnodet mentioned this pull request May 16, 2023

[MNG-7785] Clean usage of SessionData #1111

Merged

gnodet mentioned this pull request May 24, 2023

MNG 7587 3.9.x #1120

Closed

	if ( workspace instanceof MavenWorkspaceReader )
	{
	model = ( (MavenWorkspaceReader) workspace ).findModel( pomArtifact );
	if ( model != null )
	{
	return model;
	}
	}

[MNG-7432] Resolver session contains non-MavenWorkspaceReader #695

[MNG-7432] Resolver session contains non-MavenWorkspaceReader #695

Conversation

laeubi commented Mar 15, 2022 • edited

laeubi commented Mar 15, 2022

laeubi commented Mar 15, 2022 • edited

famod commented Mar 15, 2022

laeubi commented Mar 18, 2022

michael-o commented Mar 18, 2022

laeubi commented Mar 18, 2022

basil commented Mar 19, 2022

basil commented Mar 19, 2022

laeubi commented Mar 20, 2022

basil commented Mar 20, 2022

laeubi commented Mar 20, 2022

basil commented Mar 20, 2022

laeubi commented Mar 20, 2022

basil commented Mar 20, 2022

jglick commented Mar 21, 2022

laeubi commented Mar 21, 2022

laeubi commented Mar 29, 2022

jglick left a comment

Choose a reason for hiding this comment

laeubi commented Mar 29, 2022 • edited

michael-o commented Mar 29, 2022

laeubi commented Mar 29, 2022

jglick commented Mar 29, 2022

famod commented Mar 30, 2022 • edited

cstamas Apr 2, 2022 • edited

Choose a reason for hiding this comment

laeubi Apr 2, 2022

Choose a reason for hiding this comment

cstamas Apr 2, 2022 • edited

Choose a reason for hiding this comment

laeubi Apr 2, 2022

Choose a reason for hiding this comment

cstamas commented Apr 2, 2022

cstamas commented Apr 2, 2022

laeubi commented Apr 4, 2022

cstamas commented Apr 7, 2022

michael-o commented Apr 7, 2022

cstamas commented Apr 7, 2022

michael-o commented Apr 7, 2022

cstamas commented Apr 7, 2022

michael-o commented Apr 7, 2022

laeubi commented Mar 15, 2022 •

edited

laeubi commented Mar 15, 2022 •

edited

laeubi commented Mar 29, 2022 •

edited

famod commented Mar 30, 2022 •

edited

cstamas Apr 2, 2022 •

edited

cstamas Apr 2, 2022 •

edited