New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Junit logger produces report which Jenxins xunit-plugin considers invalid. #3913
Comments
The "X" in "XML" stands for "Extensible". This means, at least IMO, that any consumer of the JUnit XML produced by PHPUnit must ignore attributes it does not handle. There is no (official) standard for JUnit XML and there is no official XSD for JUnit XML. See #3162 for more details. I am sick and tired of this situation and am strongly thinking about abandoning "JUnit XML" and instead create PHPUnit's XML logfile format for test results. |
Technically, we already have that with the PHPUnit coverage xml. |
Yes, we "just" need to make it possible to create a logfile using that format when no code coverage is collected/processed. |
When @localheinz and I are done with the Event subsystem, we can have a look at that :) |
Thanks @sebastianbergmann. For now it looks like our options are:
|
The PR to xunit-plugin has been merged, so this should be fixed in a future release of that. |
Just a nudge to notify watchers that the xunit-plugin has been updated and this is now working for me using phpunit 8.5.2 |
@sebastianbergmann what do you think about having an integration test against Jenkins XSDs? https://github.com/jenkinsci/xunit-plugin/tree/master/src/main/resources/org/jenkinsci/plugins/xunit/types |
I am tired of "playing catchup" with various "forks" of JUnit XML. Just because the Jenkins CI project has an XSD does not make this XSD official. There is no "official JUnit XML". This has been discussed in #3162, for instance. To be honest, I would like to eventually deprecate and remove PHPUnit's ability to generate "JUnit XML" logfiles once #4321 has been implemented. |
Summary
Report generated with
--log-junit
includes an attribute 'warnings' of element 'testsuite'. According to https://github.com/jenkinsci/xunit-plugin/blob/master/src/main/resources/org/jenkinsci/plugins/xunit/types/phpunit-2.0.xsd#L89 this is not an allowed attribute.Depending on build config, this can cause a build failure in Jenkins, with a message like the following showing up in the log:
At line 3 of file:/var/lib/jenkins/workspace/myproject/build/logs/report.xml:cvc-complex-type.3.2.2: Attribute 'warnings' is not allowed to appear in element 'testsuite'
However I'm not sure if there is any really authoritative xsd for the junit format available, so potentially this could something to fix in the xunit-plugin instead or as well as fixing here. I have also made a PR to xunit-plugin.
This issue is new in PHPUnit 8.4.2 , apparently introduced in e7b8576 .
Current behavior
report.xml file is generated with line as shown below:
<testsuite name="Test Suite" tests="201" assertions="221" errors="0" warnings="0" failures="0" skipped="0" time="0.498413">
How to reproduce
Run
./bin/phpunit --log-junit report.xml
Expected behavior
report.xml to be generated without the warnings attribute, as in PHPUnit 8.4.1:
<testsuite name="Test Suite" tests="201" assertions="221" errors="0" failures="0" skipped="0" time="0.498413">
The text was updated successfully, but these errors were encountered: