Add additional information to JUnit reports

[simonwang384]

Description

Nightwatch currently generates a JUnit report with data that fits most people needs but there are some use cases where the current JUnit report isn't sufficient enough mainly regarding attaching properties to test suites and test cases. Our main use case currently for this is a need for the base64 of the screenshot of when the test failed.

Suggested solution

Here is a link on common JUnit report formats https://github.com/testmoapp/junitxml?tab=readme-ov-file. The extension we are going to want is adding properties with property. This way we can attach more properties related to a test suite and test case. Ideally we just have the current Nightwatch JUnit reporter create these fields for us based on what information we need in the report.

Alternatives / Workarounds

Create a custom JUnit reporter that extends current JUnit reporting functionality. Doesn't seem like there is a easy way of doing this besides copying the current JUnit reporter. If we can consolidate this into the main JUnit reporter that would be great.

Additional Information

I understand this solution is outside the traditional JUnit report format but if we can get more information in the JUnit report from the Nightwatch test suite and case that can be beneficial for apps that parse test reports for information.

If this is outside of Nightwatch's control would it be okay to copy the current JUnit reporting functionality and create our own custom JUnit reporter from that?

Edit: Looks like the BaseReporter is exported so it wouldn't be too hard to create our own custom JUnit reporter but I guess I want to know if this something we should do as its own package or add it in the core Nightwatch project

[github-actions]

Thank you for setting this as an enhancement. One of the product folk will triage this again to help see when we can fit this in an upcoming sprint.

[beatfactor]

hey @simonwang384, thanks for raising this. Could you give us an example of how the enhanced report will look like? Screenshots are added to the JUnit report as attachments currently, but you want them as base64 encoded data?

[simonwang384]

Hey @beatfactor, I have an example report here:
example junit.txt

Yes. Our CI doesn't export the attachments from the run and it's not an easy process for us. If we did export the images it's not a easy process to map the image to the test case in the report. We want the image in base64 and other metadata like (steps, tags, etc...) in the JUnit report so we can just work with the report. The attachment in the JUnit report is just the path to the image which isn't enough for us sadly.

[simonwang384]

Created this custom reporter for now https://www.npmjs.com/package/nightwatch-extended-junit-reporter?activeTab=readme if anyone comes along this issue and also wants to use this