Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AxisTest fails when Git plugin is on the test classpath #615

Open
basil opened this issue Jul 31, 2021 · 6 comments
Open

AxisTest fails when Git plugin is on the test classpath #615

basil opened this issue Jul 31, 2021 · 6 comments
Labels
help wanted Extra attention is needed tests

Comments

@basil
Copy link
Member

basil commented Jul 31, 2021

See #251 (comment). It would be good to get to the bottom of this and remove the AxisTest exclusion.
@basil basil added the bug Something isn't working label Jul 31, 2021
@basil basil added tests and removed bug Something isn't working labels Sep 1, 2023
@basil
Copy link
Member Author

basil commented Nov 9, 2023

The AxisTest exclusion ought to be removed, as its presence has led to breaking changes not being discovered quickly enough in the recent past. The blocking issue is a PCT-only failure of AxisTest#submitInvalidAxisValue:

org.htmlunit.ElementNotFoundException: elementName=[input] attributeName=[name] attributeValue=[_.valueString]
	at org.htmlunit.html.HtmlForm.getInputByName(HtmlForm.java:636)
	at hudson.matrix.AxisTest.submitInvalidAxisValue(AxisTest.java:94)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.jvnet.hudson.test.JenkinsRule$1.evaluate(JenkinsRule.java:607)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.lang.Thread.run(Thread.java:833)

The problem can be reproduced outside of PCT by checking out matrix-project-plugin and adding the Git plugin to test scope

diff --git a/pom.xml b/pom.xml
index a9bdf84..7d13f0c 100644
--- a/pom.xml
+++ b/pom.xml
@@ -74,6 +74,11 @@
             <artifactId>mockito-core</artifactId>
             <scope>test</scope>
         </dependency>
+        <dependency>
+            <groupId>org.jenkins-ci.plugins</groupId>
+            <artifactId>git</artifactId>
+            <scope>test</scope>
+        </dependency>
     </dependencies>
     <dependencyManagement>
         <dependencies>

then running mvn clean verify -Dtest=AxisTest#submitInvalidAxisValue. It seems like this is a simple test issue that should be relatively easy to fix. Once a fix has been integrated and released, this long-standing exclusion can finally be removed.

@basil basil added the help wanted Extra attention is needed label Nov 9, 2023
@basil basil changed the title AxisTest fails with cryptic PCT-only errors AxisTest#submitInvalidAxisValue fails when Git plugin is on the test classpath Nov 9, 2023
@basil
Copy link
Member Author

basil commented Nov 9, 2023

FYI @MarkEWaite since the above failure only occurs when the Git plugin is on the test classpath

@MarkEWaite
Copy link
Contributor

It appears that the test relies on a lookup of the input form by name but the name is ambiguous when the git plugin is loaded. The test finds an input for by name("_.name"). Without the git plugin loaded, there is only one form with that name on the page. With the git plugin loaded, there is another form with that same name and the form from the git plugin is selected instead of the form from the matrix project plugin.

Maybe an XPath selection could be used instead of finding the input form by name?

I'll need to do more searching as time allows.

@basil
Copy link
Member Author

basil commented Nov 28, 2023

Maybe an XPath selection could be used instead of finding the input form by name?

Seems like a good idea. For what it's worth, I think ChatGPT is pretty good at writing XPath queries.

MarkEWaite added a commit to MarkEWaite/matrix-project-plugin that referenced this issue Dec 10, 2023
@MarkEWaite
Fix AxisTest#submitInvalidAxisValue for PCT / plugin BOM
2932673 
jenkinsci/bom#615 describes the problem in
more detail.

When the git plugin is loaded along with the matrix project plugin,
there are multiple input fields on the job configuration page with the
name `_.name`.  The previous code waited until there was at least one
input field with the name `_.name`, but instead it needs to wait for
the arrival of an additional input field with the name `_.name`.

Adds a new `setName` method that sets the value of the `_.name` field
that was created by this plugin.  Previously, it would set the first
`_.name` field.  Setting the first `_.name` field works when running
the matrix plugin tests without the git plugin loaded, but fails when
the git plugin is loaded.

The combination of waiting for the correct `_.name` field to appear and
setting the value on the correct `_.name` field allows oone of the four
tests to pass when the git plugin is loaded.  The remaining tests need
more investigation to pass when the git plugin is loaded.

Also removes a nearly silent skip of the tests when the input does not
appear within the timeout.

Also increases the time between retries so that my fast computer needs
only two or three retries.  I confirmed on a much slower computer
(Intel Core i5-2410M CPU @ 2.30GHz) that even on that slow computer,
it found the necessary field in 8 tries or less.  The upper bound of 18
tries should be more than enough for all the test environments.

The retry period should not be increased beyond the current ~300ms
because one of the tests is using a form that does not include `_.name`.
That test falls through to the end of the retry period.
@MarkEWaite MarkEWaite mentioned this issue Dec 11, 2023
Merged
6 tasks
MarkEWaite added a commit to MarkEWaite/matrix-project-plugin that referenced this issue Dec 13, 2023
@MarkEWaite
Fix AxisTest#submitInvalidAxisValue for PCT / plugin BOM
3196c13 
jenkinsci/bom#615 describes the problem in
more detail.

When the git plugin is loaded along with the matrix project plugin,
there are multiple input fields on the job configuration page with the
name `_.name`.  The previous code waited until there was at least one
input field with the name `_.name`, but instead it needs to wait for
the arrival of an additional input field with the name `_.name`.

Adds a new `setName` method that sets the value of the `_.name` field
that was created by this plugin.  Previously, it would set the first
`_.name` field.  Setting the first `_.name` field works when running
the matrix plugin tests without the git plugin loaded, but fails when
the git plugin is loaded.

The combination of waiting for the correct `_.name` field to appear and
setting the value on the correct `_.name` field allows oone of the four
tests to pass when the git plugin is loaded.  The remaining tests need
more investigation to pass when the git plugin is loaded.

Also removes a nearly silent skip of the tests when the input does not
appear within the timeout.

Also increases the time between retries so that my fast computer needs
only two or three retries.  I confirmed on a much slower computer
(Intel Core i5-2410M CPU @ 2.30GHz) that even on that slow computer,
it found the necessary field in 8 tries or less.  The upper bound of 18
tries should be more than enough for all the test environments.

The retry period should not be increased beyond the current ~300ms
because one of the tests is using a form that does not include `_.name`.
That test falls through to the end of the retry period.
basil pushed a commit to jenkinsci/matrix-project-plugin that referenced this issue Dec 19, 2023
@MarkEWaite
Fix AxisTest#submitInvalidAxisValue for PCT / plugin BOM (#184)
0389fc6 
jenkinsci/bom#615 describes the problem in
more detail.

When the git plugin is loaded along with the matrix project plugin,
there are multiple input fields on the job configuration page with the
name `_.name`.  The previous code waited until there was at least one
input field with the name `_.name`, but instead it needs to wait for
the arrival of an additional input field with the name `_.name`.

Adds a new `setName` method that sets the value of the `_.name` field
that was created by this plugin.  Previously, it would set the first
`_.name` field.  Setting the first `_.name` field works when running
the matrix plugin tests without the git plugin loaded, but fails when
the git plugin is loaded.

The combination of waiting for the correct `_.name` field to appear and
setting the value on the correct `_.name` field allows oone of the four
tests to pass when the git plugin is loaded.  The remaining tests need
more investigation to pass when the git plugin is loaded.

Also removes a nearly silent skip of the tests when the input does not
appear within the timeout.

Also increases the time between retries so that my fast computer needs
only two or three retries.  I confirmed on a much slower computer
(Intel Core i5-2410M CPU @ 2.30GHz) that even on that slow computer,
it found the necessary field in 8 tries or less.  The upper bound of 18
tries should be more than enough for all the test environments.

The retry period should not be increased beyond the current ~300ms
because one of the tests is using a form that does not include `_.name`.
That test falls through to the end of the retry period.
@basil basil mentioned this issue Dec 19, 2023
Merged
@basil basil changed the title AxisTest#submitInvalidAxisValue fails when Git plugin is on the test classpath AxisTest fails when Git plugin is on the test classpath Dec 19, 2023
@basil
Copy link
Member Author

basil commented Dec 19, 2023

As of #2767 at least 1 out of 4 tests is running on a regular basis. Now this I call progress.

@MarkEWaite
Copy link
Contributor

As of #2767 at least 1 out of 4 tests is running on a regular basis. Now this I call progress.

Thanks so much for #2767 and the adjustment of the exclusions so that one of the tests is being watched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted Extra attention is needed tests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@basil @MarkEWaite