Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SUREFIRE-2232] StatelessXmlReporter: handle failed test result without a throwable #716

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

dr29bart
Copy link

A regression bug appeared in 3.0.0-M6:

A testNG test class has a listener which updates results from SUCCESS to FAILURE:

@Override
public void onTestSuccess(ITestResult result) {
    result.setStatus(ITestResult.FAILURE);
    result.getTestContext().getPassedTests().removeResult(result);
    result.getTestContext().getFailedTests().addResult(result);
}

Surefire fails to process a failed test result without a throwable and reports 0 total tests.

ForkStarter IOException: java.util.NoSuchElementException.
org.apache.maven.plugin.surefire.booterclient.output.MultipleFailureException: java.util.NoSuchElementException
	at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.<init>(ThreadedStreamConsumer.java:59)
	at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer.<init>(ThreadedStreamConsumer.java:107)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:546)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:285)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:250) 
...
Suppressed: java.util.NoSuchElementException
		at java.base/java.util.StringTokenizer.nextToken(StringTokenizer.java:347)
		at org.apache.maven.plugin.surefire.report.StatelessXmlReporter.getTestProblems(StatelessXmlReporter.java:454)
		at org.apache.maven.plugin.surefire.report.StatelessXmlReporter.serializeTestClassWithoutRerun(StatelessXmlReporter.java:221)
		at org.apache.maven.plugin.surefire.report.StatelessXmlReporter.serializeTestClass(StatelessXmlReporter.java:211)
		at org.apache.maven.plugin.surefire.report.StatelessXmlReporter.testSetCompleted(StatelessXmlReporter.java:161)
		at org.apache.maven.plugin.surefire.report.StatelessXmlReporter.testSetCompleted(StatelessXmlReporter.java:85)
		at org.apache.maven.plugin.surefire.report.TestSetRunListener.testSetCompleted(TestSetRunListener.java:193)
		at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$TestSetCompletedListener.handle(ForkClient.java:143)

The fix is to set a hardcoded value - UndefinedException as error type in case stack trace is empty.

checklist Following this checklist to help us incorporate your contribution quickly and easily:
  • Make sure there is a JIRA issue filed
    for the change (usually before you start working on it). Trivial changes like typos do not
    require a JIRA issue. Your pull request should address just this issue, without
    pulling in other changes.
  • Each commit in the pull request should have a meaningful subject line and body.
  • Format the pull request title like [SUREFIRE-XXX] - Fixes bug in ApproximateQuantiles,
    where you replace SUREFIRE-XXX with the appropriate JIRA issue. Best practice
    is to use the JIRA issue title in the pull request title and in the first line of the
    commit message.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Run mvn clean install to make sure basic checks pass. A more thorough check will
    be performed on your pull request automatically.
  • You have run the integration tests successfully (mvn -Prun-its clean install).

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.

To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

ppw.addAttribute("type", new StringTokenizer(stackTrace).nextToken());
ppw.addAttribute(
"type",
isBlank(stackTrace) ? "UndefinedException" : new StringTokenizer(stackTrace).nextToken());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you reuse the approach from line 451?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean indexOf approach or to use value as is?

I didn't want to leave a chance for type to be null. What non-null value can be used here?

The stackTrace StatelessXmlReporter#439 var may be null:

public String getStackTrace(boolean trimStackTrace) {
        StackTraceWriter w = original.getStackTraceWriter();
        return w == null ? null : (trimStackTrace ? w.writeTrimmedTraceToString() : w.writeTraceToString());
    }

At the same time surefire's XSD requires type

XSD
<xs:element name="error" nillable="true" minOccurs="0" maxOccurs="1">
                                <xs:complexType>
                                    <xs:simpleContent>
                                        <xs:extension base="xs:string">
                                            <xs:attribute name="message" type="xs:string"/>
                                            <xs:attribute name="type" type="xs:string" use="required"/>
                                        </xs:extension>
                                    </xs:simpleContent>
                                </xs:complexType>
                            </xs:element>

The goal is to handle stack traces like:

// has message
junit.framework.ComparisonFailure: 
Expected :fail at foo
Actual   :faild at foo

// does not have message
java.lang.AssertionError
	at org.junit.Assert.fail(Assert.java:87)
	at org.junit.Assert.fail(Assert.java:96)

// or stack trace is missing (empty String)

Please, let me know if this alternative is more preferable:

Option A
if (report.getStackTraceWriter() != null) {
            //noinspection ThrowableResultOfMethodCallIgnored
            SafeThrowable t = report.getStackTraceWriter().getThrowable();
            if (t != null) {
                int delimiter = StringUtils.indexOfAny(stackTrace, ":", "\t", "\n", "\r", "\f");
                String type = delimiter == -1 ? stackTrace : stackTrace.substring(0, delimiter);
                ppw.addAttribute("type", Objects.toString(type, "UndefinedException"));
            }
        }
Option B
SafeThrowable t = report.getStackTraceWriter().getThrowable();
            if (t != null) {
                if (t.getMessage() != null) {
                    int delimiter = stackTrace.indexOf(":");
                    String type = delimiter == -1 ? stackTrace : stackTrace.substring(0, delimiter);
                    ppw.addAttribute("type", type);
                } else {
                    ppw.addAttribute(
                            "type",
                            isBlank(stackTrace) ? stackTrace : new StringTokenizer(stackTrace).nextToken());
                }
            }
Option C
SafeThrowable t = report.getStackTraceWriter().getThrowable();
            if (t != null) {
                if (t.getMessage() != null) {
                    int delimiter = stackTrace.indexOf(":");
                    String type = delimiter == -1 ? stackTrace : stackTrace.substring(0, delimiter);
                    ppw.addAttribute("type", type);
                } else {
                    int delimiter = StringUtils.indexOfAny(stackTrace, "\t", "\n", "\r", "\f");
                    String type = delimiter == -1 ? stackTrace : stackTrace.substring(0, delimiter);
                    ppw.addAttribute("type", type);
                }
            }
</details>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants