Feature request: Do not fail the Maven build for a failed test #534

kaqqao · 2016-12-09T15:15:40Z

When running npm test (for example) and if it fails, the whole Maven build will fail too.
Would it be possible to make the plugin act differently for goals invoked during test and integration-test phases so that failures in those are only reported as test failures?

If you can provide some guidance on if/how this can be achieved, I'd gladly take on the task of implementing it.

The text was updated successfully, but these errors were encountered:

eirslett · 2016-12-09T15:29:41Z

Try this:

<maven.test.failure.ignore>true</maven.test.failure.ignore>

kaqqao · 2016-12-09T16:14:10Z

Thanks for a such a fast response!
That was my first guess as well :) Alas, it doesn't make a difference.

Here's my config:

<plugin>
				<groupId>com.github.eirslett</groupId>
				<artifactId>frontend-maven-plugin</artifactId>
				<configuration>
            		<installDirectory>target</installDirectory>
        		</configuration>
				<executions>
					... install-node-and-npm and <npm install> go here
					<execution>
						<id>npm test</id>
						<phase>test</phase>
						<goals>
							<goal>npm</goal>
						</goals>
						<configuration>
							<arguments>test</arguments>
						</configuration>
					</execution>
				</executions>
			</plugin>

If my assumption is correct, to support this it would be enough not to throw MojoFailureException in the presence of the maven.test.failure.ignore flag, if the mojo is attached to one of the test phases.

EDIT
I found out this is supported as described, but only in Karma mojo. As I am running Protractor tests (via npm test), I don't benefit from it. For similar cases, I believe this flag should be supported ubiquitously, during the test and integration-test phase of all mojos.

Kepro · 2017-02-09T21:37:30Z

@eirslett, any other idea? :) if junit fail some test it's just a warning but this failing whole build and that is not good :(

kaqqao · 2017-02-10T00:20:55Z

I've just made a pull request that closes this issue by adding ubiquitous support for maven.test.failure.ignore as described above: #564

EDIT
Seems like a similar issue and pull request already exist: #552 & #553
That one adds some more features (an option to ignore errors outside of test phase), so feel free to reject mine in favor of that one.

kaqqao · 2017-09-30T17:03:10Z

While the feature has been added, the opposite behavior is now broken i.e. the builds now never seem to fail, even with maven.test.failure.ignore set to false.

eirslett · 2017-09-30T17:08:05Z

Even with the latest version of the plugin?

cliffmeyers · 2017-10-06T15:08:49Z

@kaqqao @eirslett

I ran into this issue yesterday. The behavior does appear broken in 1.4 (failures during the test phase are always ignored). In 1.6 it works as expected. In these examples, I've introduced a failing test case that is run via gulp + gulp-jest.

Run a gulp build that finishes with exit code 1:

macbook-15:blueocean-dashboard cmeyers$ mvn clean install
...
[ERROR] Test Suites: 1 failed, 2 skipped, 26 passed, 27 of 29 total
[ERROR] Tests:       1 failed, 8 skipped, 149 passed, 158 total
[ERROR] Snapshots:   0 total
[ERROR] Time:        26.889s
[ERROR] Ran all test suites.
[INFO] [10:52:58] tests failed or errored!
[ERROR] 
[ERROR] npm ERR! Darwin 15.6.0
[ERROR] npm ERR! argv "/Users/cmeyers/Development/code/blueocean/blueocean-dashboard/node/node" "/Users/cmeyers/Development/code/blueocean/blueocean-dashboard/node/node_modules/npm/bin/npm-cli.js" "run" "mvntest"
[ERROR] npm ERR! node v6.4.0
[ERROR] npm ERR! npm  v3.10.3
[ERROR] npm ERR! code ELIFECYCLE
[ERROR] npm ERR! blueocean-dashboard@0.0.1 mvntest: `gulp test`
[ERROR] npm ERR! Exit status 1
[ERROR] npm ERR! 
[ERROR] npm ERR! Failed at the blueocean-dashboard@0.0.1 mvntest script 'gulp test'.
[ERROR] npm ERR! Make sure you have the latest version of node.js and npm installed.
[ERROR] npm ERR! If you do, this is most likely a problem with the blueocean-dashboard package,
[ERROR] npm ERR! not with npm itself.
[ERROR] npm ERR! Tell the author that this fails on your system:
[ERROR] npm ERR!     gulp test
[ERROR] npm ERR! You can get information on how to open an issue for this project with:
[ERROR] npm ERR!     npm bugs blueocean-dashboard
[ERROR] npm ERR! Or if that isn't available, you can get their info via:
[ERROR] npm ERR!     npm owner ls blueocean-dashboard
[ERROR] npm ERR! There is likely additional logging output above.
[ERROR] 
[ERROR] npm ERR! Please include the following file with any support request:
[ERROR] npm ERR!     /Users/cmeyers/Development/code/blueocean/blueocean-dashboard/npm-debug.log
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

Run again with the maven.test.failure.ignore property set:

mvn clean test -Dmaven.test.failure.ignore
...
[ERROR] Test Suites: 1 failed, 2 skipped, 26 passed, 27 of 29 total
[ERROR] Tests:       1 failed, 8 skipped, 149 passed, 158 total
[ERROR] Snapshots:   0 total
[ERROR] Time:        25.227s
[ERROR] Ran all test suites.
[INFO] [10:56:22] tests failed or errored!
[ERROR] 
[ERROR] npm ERR! Darwin 15.6.0
[ERROR] npm ERR! argv "/Users/cmeyers/Development/code/blueocean/blueocean-dashboard/node/node" "/Users/cmeyers/Development/code/blueocean/blueocean-dashboard/node/node_modules/npm/bin/npm-cli.js" "run" "mvntest"
[ERROR] npm ERR! node v6.4.0
[ERROR] npm ERR! npm  v3.10.3
[ERROR] npm ERR! code ELIFECYCLE
[ERROR] npm ERR! blueocean-dashboard@0.0.1 mvntest: `gulp test`
[ERROR] npm ERR! Exit status 1
[ERROR] npm ERR! 
[ERROR] npm ERR! Failed at the blueocean-dashboard@0.0.1 mvntest script 'gulp test'.
[ERROR] npm ERR! Make sure you have the latest version of node.js and npm installed.
[ERROR] npm ERR! If you do, this is most likely a problem with the blueocean-dashboard package,
[ERROR] npm ERR! not with npm itself.
[ERROR] npm ERR! Tell the author that this fails on your system:
[ERROR] npm ERR!     gulp test
[ERROR] npm ERR! You can get information on how to open an issue for this project with:
[ERROR] npm ERR!     npm bugs blueocean-dashboard
[ERROR] npm ERR! Or if that isn't available, you can get their info via:
[ERROR] npm ERR!     npm owner ls blueocean-dashboard
[ERROR] npm ERR! There is likely additional logging output above.
[ERROR] 
[ERROR] npm ERR! Please include the following file with any support request:
[ERROR] npm ERR!     /Users/cmeyers/Development/code/blueocean/blueocean-dashboard/npm-debug.log
[ERROR] There are test failures.
Failed to run task: 'npm run mvntest' failed.
com.github.eirslett.maven.plugins.frontend.lib.TaskRunnerException: 'npm run mvntest' failed.
	at com.github.eirslett.maven.plugins.frontend.lib.NodeTaskExecutor.execute(NodeTaskExecutor.java:63)
	at com.github.eirslett.maven.plugins.frontend.mojo.NpmMojo.execute(NpmMojo.java:62)
	at com.github.eirslett.maven.plugins.frontend.mojo.AbstractFrontendMojo.execute(AbstractFrontendMojo.java:94)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutionException: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
	at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.execute(ProcessExecutor.java:82)
	at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.executeAndRedirectOutput(ProcessExecutor.java:64)
	at com.github.eirslett.maven.plugins.frontend.lib.NodeExecutor.executeAndRedirectOutput(NodeExecutor.java:29)
	at com.github.eirslett.maven.plugins.frontend.lib.NodeTaskExecutor.execute(NodeTaskExecutor.java:58)
	... 24 more
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
	at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
	at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
	at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.execute(ProcessExecutor.java:74)
	... 27 more
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Also thanks to the contributor for helpful output like this:

[INFO] --- frontend-maven-plugin:1.6:npm (npm install) @ blueocean-dashboard ---
[INFO] testFailureIgnore property is ignored in non test phases
[INFO] Running 'npm install' in /Users/cmeyers/Development/code/blueocean/blueocean-dashboard

Two small suggestions that I could open a PR for if you're willing @eirslett

Perhaps we should add a similar logging statement during test phases that indicate the value of testFailureIgnore and whether failures will result in a broken build, and
The stack trace is logged when test failures are ignored, but not in the other case (see first snippet). Should it be logged in both cases?

This was referenced Feb 10, 2017

closes #534: Do not fail the Maven build for a failed test #564

Closed

Request Feature - Add failsafe mode to generally manage integration-test #552

Closed

jcarsique mentioned this issue Feb 20, 2017

add failOnError and integration-test support #572

Merged

hnuttin mentioned this issue Jul 20, 2017

Maven build won't fail when integration-test fails #631

Closed

cliffmeyers mentioned this issue Oct 6, 2017

bug / maven frontend plugin 1.6 jenkinsci/blueocean-plugin#1464

Merged

7 tasks

yurloc mentioned this issue Oct 16, 2019

Run lint in compile phase to fail Maven build on Jenkins kiegroup/optaweb-vehicle-routing#206

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature request: Do not fail the Maven build for a failed test #534

Feature request: Do not fail the Maven build for a failed test #534

kaqqao commented Dec 9, 2016 •

edited

eirslett commented Dec 9, 2016

kaqqao commented Dec 9, 2016 •

edited

Kepro commented Feb 9, 2017

kaqqao commented Feb 10, 2017 •

edited

kaqqao commented Sep 30, 2017

eirslett commented Sep 30, 2017

cliffmeyers commented Oct 6, 2017

Feature request: Do not fail the Maven build for a failed test #534

Feature request: Do not fail the Maven build for a failed test #534

Comments

kaqqao commented Dec 9, 2016 • edited

eirslett commented Dec 9, 2016

kaqqao commented Dec 9, 2016 • edited

Kepro commented Feb 9, 2017

kaqqao commented Feb 10, 2017 • edited

kaqqao commented Sep 30, 2017

eirslett commented Sep 30, 2017

cliffmeyers commented Oct 6, 2017

kaqqao commented Dec 9, 2016 •

edited

kaqqao commented Dec 9, 2016 •

edited

kaqqao commented Feb 10, 2017 •

edited