Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for NUnit files #289

Merged
merged 4 commits into from Jun 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/ci-cd.yml
Expand Up @@ -202,7 +202,7 @@ jobs:
id: test-results
if: always()
run: |
docker run --workdir $GITHUB_WORKSPACE --rm -e INPUT_CHECK_NAME -e INPUT_JUNIT_FILES -e INPUT_XUNIT_FILES -e INPUT_TRX_FILES -e INPUT_TIME_UNIT -e INPUT_GITHUB_TOKEN -e INPUT_GITHUB_RETRIES -e INPUT_COMMIT -e INPUT_COMMENT_TITLE -e INPUT_FAIL_ON -e INPUT_REPORT_INDIVIDUAL_RUNS -e INPUT_DEDUPLICATE_CLASSES_BY_FILE_NAME -e INPUT_IGNORE_RUNS -e INPUT_HIDE_COMMENTS -e INPUT_COMMENT_ON_PR -e INPUT_COMMENT_MODE -e INPUT_COMPARE_TO_EARLIER_COMMIT -e INPUT_PULL_REQUEST_BUILD -e INPUT_EVENT_FILE -e INPUT_EVENT_NAME -e INPUT_TEST_CHANGES_LIMIT -e INPUT_CHECK_RUN_ANNOTATIONS -e INPUT_CHECK_RUN_ANNOTATIONS_BRANCH -e INPUT_SECONDS_BETWEEN_GITHUB_READS -e INPUT_SECONDS_BETWEEN_GITHUB_WRITES -e INPUT_JSON_FILE -e INPUT_JSON_THOUSANDS_SEPARATOR -e INPUT_JOB_SUMMARY -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e GITHUB_STEP_SUMMARY -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "$RUNNER_TEMP":"$RUNNER_TEMP" -v "$GITHUB_WORKSPACE":"$GITHUB_WORKSPACE" enricomi/publish-unit-test-result-action:latest
docker run --workdir $GITHUB_WORKSPACE --rm -e INPUT_CHECK_NAME -e INPUT_JUNIT_FILES -e INPUT_NUNIT_FILES -e INPUT_XUNIT_FILES -e INPUT_TRX_FILES -e INPUT_TIME_UNIT -e INPUT_GITHUB_TOKEN -e INPUT_GITHUB_RETRIES -e INPUT_COMMIT -e INPUT_COMMENT_TITLE -e INPUT_FAIL_ON -e INPUT_REPORT_INDIVIDUAL_RUNS -e INPUT_DEDUPLICATE_CLASSES_BY_FILE_NAME -e INPUT_IGNORE_RUNS -e INPUT_HIDE_COMMENTS -e INPUT_COMMENT_ON_PR -e INPUT_COMMENT_MODE -e INPUT_COMPARE_TO_EARLIER_COMMIT -e INPUT_PULL_REQUEST_BUILD -e INPUT_EVENT_FILE -e INPUT_EVENT_NAME -e INPUT_TEST_CHANGES_LIMIT -e INPUT_CHECK_RUN_ANNOTATIONS -e INPUT_CHECK_RUN_ANNOTATIONS_BRANCH -e INPUT_SECONDS_BETWEEN_GITHUB_READS -e INPUT_SECONDS_BETWEEN_GITHUB_WRITES -e INPUT_JSON_FILE -e INPUT_JSON_THOUSANDS_SEPARATOR -e INPUT_JOB_SUMMARY -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e GITHUB_STEP_SUMMARY -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "$RUNNER_TEMP":"$RUNNER_TEMP" -v "$GITHUB_WORKSPACE":"$GITHUB_WORKSPACE" enricomi/publish-unit-test-result-action:latest
env:
INPUT_GITHUB_TOKEN: ${{ github.token }}
INPUT_CHECK_NAME: Test Results (Docker Image)
Expand Down Expand Up @@ -409,7 +409,7 @@ jobs:
- name: Checkout
uses: actions/checkout@v3

- name: Copy test files
- name: Copy test result files
run: cp -rv python/test/files test-files

- name: Prepare publish action from this branch
Expand All @@ -423,6 +423,7 @@ jobs:
check_name: Test Results (Test Files)
fail_on: nothing
junit_files: "test-files/junit-xml/**/*.xml"
nunit_files: "test-files/nunit/**/*.xml"
xunit_files: "test-files/xunit/**/*.xml"
trx_files: "test-files/trx/**/*.trx"
json_file: "tests.json"
Expand Down
6 changes: 4 additions & 2 deletions README.md
Expand Up @@ -13,7 +13,7 @@
[![Test Results](https://gist.githubusercontent.com/EnricoMi/612cb538c14731f1a8fefe504f519395/raw/badge.svg)](https://gist.githubusercontent.com/EnricoMi/612cb538c14731f1a8fefe504f519395/raw/badge.svg)

This [GitHub Action](https://github.com/actions) analyses test result files and
publishes the results on GitHub. It supports the JUnit XML and TRX file formats, and runs on Linux, macOS and Windows.
publishes the results on GitHub. It supports the TRX file format and JUnit, NUnit and XUnit XML formats, and runs on Linux, macOS and Windows.

You can add this action to your GitHub workflow for ![Ubuntu Linux](https://badgen.net/badge/icon/Ubuntu?icon=terminal&label) (e.g. `runs-on: ubuntu-latest`) runners:

Expand All @@ -23,6 +23,7 @@ You can add this action to your GitHub workflow for ![Ubuntu Linux](https://badg
if: always()
with:
junit_files: "test-results/junit/**/*.xml"
nunit_files: "test-results/nunit/**/*.xml"
xunit_files: "test-results/xunit/**/*.xml"
trx_files: "test-results/**/*.trx"
```
Expand All @@ -36,6 +37,7 @@ and ![Windows](https://badgen.net/badge/icon/Windows?icon=windows&label) (e.g. `
if: always()
with:
junit_files: "test-results/junit/**/*.xml"
nunit_files: "test-results/nunit/**/*.xml"
xunit_files: "test-results/xunit/**/*.xml"
trx_files: "test-results/**/*.trx"
```
Expand Down Expand Up @@ -204,7 +206,7 @@ The list of most notable options:

|Option|Default Value|Description|
|:-----|:-----:|:----------|
|`junit_files`<br/>`xunit_files`<br/>`trx_files`|At least one of these `*_files` must be set.|File patterns of JUnit XML, XUnit XML, and TRX test result files, respectively. Supports `*`, `**`, `?`, and `[]`. Use multiline string for multiple patterns. Patterns starting with `!` exclude the matching files. There have to be at least one pattern starting without a `!`.|
|`junit_files`<br/>`nunit_files`<br/>`xunit_files`<br/>`trx_files`|At least one of these `*_files` must be set.|File patterns of JUnit XML, NUnit XML, XUnit XML, and TRX test result files, respectively. Supports `*`, `**`, `?`, and `[]`. Use multiline string for multiple patterns. Patterns starting with `!` exclude the matching files. There have to be at least one pattern starting without a `!`.|
|`check_name`|`"Test Results"`|An alternative name for the check result.|
|`comment_title`|same as `check_name`|An alternative name for the pull request comment.|
|`comment_mode`|`always`|The action posts comments to pull requests that are associated with the commit. Set to:<br/>`always` - always comment<br/>`changes` - comment when changes w.r.t. the target branch exist<br/>`changes in failures` - when changes in the number of failures and errors exist<br/>`changes in errors` - when changes in the number of (only) errors exist<br/>`failures` - when failures or errors exist<br/>`errors` - when (only) errors exist<br/>`off` - to not create pull request comments.|
Expand Down
3 changes: 3 additions & 0 deletions action.yml
Expand Up @@ -32,6 +32,9 @@ inputs:
junit_files:
description: 'File patterns of JUnit XML test result files. Supports *, **, ?, and []. Use multiline string for multiple patterns. Patterns starting with ! exclude the matching files. There have to be at least one pattern starting without a `!`.'
required: false
nunit_files:
description: 'File patterns of NUnit XML test result files. Supports *, **, ?, and []. Use multiline string for multiple patterns. Patterns starting with ! exclude the matching files. There have to be at least one pattern starting without a `!`.'
required: false
xunit_files:
description: 'File patterns of XUnit XML test result files. Supports *, **, ?, and []. Use multiline string for multiple patterns. Patterns starting with ! exclude the matching files. There have to be at least one pattern starting without a `!`.'
required: false
Expand Down
4 changes: 4 additions & 0 deletions composite/action.yml
Expand Up @@ -32,6 +32,9 @@ inputs:
junit_files:
description: 'File patterns of JUnit XML test result files. Supports *, **, ?, and []. Use multiline string for multiple patterns. Patterns starting with ! exclude the matching files. There have to be at least one pattern starting without a `!`.'
required: false
nunit_files:
description: 'File patterns of NUnit XML test result files. Supports *, **, ?, and []. Use multiline string for multiple patterns. Patterns starting with ! exclude the matching files. There have to be at least one pattern starting without a `!`.'
required: false
xunit_files:
description: 'File patterns of XUnit XML test result files. Supports *, **, ?, and []. Use multiline string for multiple patterns. Patterns starting with ! exclude the matching files. There have to be at least one pattern starting without a `!`.'
required: false
Expand Down Expand Up @@ -160,6 +163,7 @@ runs:
# deprecated
FILES: ${{ inputs.files }}
JUNIT_FILES: ${{ inputs.junit_files }}
NUNIT_FILES: ${{ inputs.nunit_files }}
XUNIT_FILES: ${{ inputs.xunit_files }}
TRX_FILES: ${{ inputs.trx_files }}
TIME_UNIT: ${{ inputs.time_unit }}
Expand Down
6 changes: 3 additions & 3 deletions python/publish/junit.py
Expand Up @@ -174,9 +174,9 @@ def create_junitxml(filepath: str, tree: JUnitTree) -> Union[JUnitXml, JUnitXmlE
for suite in (junit if junit._tag == "testsuites" else [junit])]

suite_tests = sum([suite.tests for result_file, suite in suites if suite.tests])
suite_skipped = sum([suite.skipped + suite.disabled for result_file, suite in suites if suite.skipped])
suite_failures = sum([suite.failures for result_file, suite in suites if suite.failures])
suite_errors = sum([suite.errors for result_file, suite in suites if suite.errors])
suite_skipped = sum([suite.skipped + suite.disabled for result_file, suite in suites if suite.skipped and not math.isnan(suite.skipped)])
suite_failures = sum([suite.failures for result_file, suite in suites if suite.failures and not math.isnan(suite.failures)])
suite_errors = sum([suite.errors for result_file, suite in suites if suite.errors and not math.isnan(suite.errors)])
suite_time = int(sum([suite.time for result_file, suite in suites
if suite.time and not math.isnan(suite.time)]) * time_factor)

Expand Down
29 changes: 29 additions & 0 deletions python/publish/nunit.py
@@ -0,0 +1,29 @@
import os
import pathlib
from typing import Iterable, Callable

from lxml import etree
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blacklist: Using etree to parse untrusted XML data is known to be vulnerable to XML attacks. Replace etree with the equivalent defusedxml package.

(at-me in a reply with help or ignore)


Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opt.semgrep.python.lang.security.use-defused-xml.use-defused-xml: Found use of the native Python XML libraries, which is vulnerable to XML external entity (XXE)
attacks. The Python documentation recommends the 'defusedxml' library instead if the XML being
loaded is untrusted.

(at-me in a reply with help or ignore)


Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]


from publish.junit import JUnitTreeOrException, ParsedJUnitFile

with (pathlib.Path(__file__).parent / 'xslt' / 'nunit3-to-junit.xslt').open('r', encoding='utf-8') as r:
transform_nunit_to_junit = etree.XSLT(etree.parse(r))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blacklist: Using lxml.etree.parse to parse untrusted XML data is known to be vulnerable to XML attacks. Replace lxml.etree.parse with its defusedxml equivalent function.

(at-me in a reply with help or ignore)


Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]



def parse_nunit_files(files: Iterable[str],
progress: Callable[[ParsedJUnitFile], ParsedJUnitFile] = lambda x: x) -> Iterable[ParsedJUnitFile]:
"""Parses nunit files."""
def parse(path: str) -> JUnitTreeOrException:
"""Parses an nunit file and returns either a JUnitTree or an Exception."""
if not os.path.exists(path):
return FileNotFoundError(f'File does not exist.')
if os.stat(path).st_size == 0:
return Exception(f'File is empty.')

try:
nunit = etree.parse(path)
return transform_nunit_to_junit(nunit)
except BaseException as e:
return e

return [progress((result_file, parse(result_file))) for result_file in files]
1 change: 1 addition & 0 deletions python/publish/publisher.py
Expand Up @@ -45,6 +45,7 @@ class Settings:
fail_on_failures: bool
# one of these *_files_glob must be set
junit_files_glob: Optional[str]
nunit_files_glob: Optional[str]
xunit_files_glob: Optional[str]
trx_files_glob: Optional[str]
time_factor: float
Expand Down
91 changes: 91 additions & 0 deletions python/publish/xslt/nunit-to-junit.xslt
@@ -0,0 +1,91 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- based on https://github.com/syl20bnr/nunit-plugin/blob/be8193421b60c45366e9f6c9d0ff91b3aa852244/src/main/resources/hudson/plugins/nunit/nunit-to-junit.xsl -->
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" indent="yes" />

<xsl:template match="/test-results">
<testsuites>
<xsl:for-each select="test-suite//results//test-case[1]">

<xsl:for-each select="../..">
<xsl:variable name="firstTestName"
select="results//test-case[1]//@name" />

<xsl:variable name="assembly">
<xsl:choose>
<xsl:when test="substring($firstTestName, string-length($firstTestName)) = ')'">
<xsl:value-of select="substring-before($firstTestName, concat('.', @name))"></xsl:value-of>
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="concat(substring-before($firstTestName, @name), @name)" />
</xsl:otherwise>
</xsl:choose>
</xsl:variable>
<!--
<xsl:variable name="assembly"
select="concat(substring-before($firstTestName, @name), @name)" />
-->

<!-- <redirect:write file="{$outputpath}/TEST-{$assembly}.xml">-->

<testsuite name="{$assembly}"
tests="{count(*/test-case)}" time="{@time}"
failures="{count(*/test-case/failure)}" errors="0"
skipped="{count(*/test-case[@executed='False'])}">
<xsl:for-each select="*/test-case">
<xsl:variable name="testcaseName">
<xsl:choose>
<xsl:when test="contains(./@name, concat($assembly,'.'))">
<xsl:value-of select="substring-after(./@name, concat($assembly,'.'))"/><!-- We either instantiate a "15" -->
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="./@name"/><!-- ...or a "20" -->
</xsl:otherwise>
</xsl:choose>
</xsl:variable>

<testcase classname="{$assembly}"
name="{$testcaseName}">
<xsl:if test="@time!=''">
<xsl:attribute name="time"><xsl:value-of select="@time" /></xsl:attribute>
</xsl:if>

<xsl:variable name="generalfailure"
select="./failure" />

<xsl:if test="./failure">
<xsl:variable name="failstack"
select="count(./failure/stack-trace/*) + count(./failure/stack-trace/text())" />
<failure>
<xsl:choose>
<xsl:when test="$failstack &gt; 0 or not($generalfailure)">
MESSAGE:
<xsl:value-of select="./failure/message" />
+++++++++++++++++++
STACK TRACE:
<xsl:value-of select="./failure/stack-trace" />
</xsl:when>
<xsl:otherwise>
MESSAGE:
<xsl:value-of select="$generalfailure/message" />
+++++++++++++++++++
STACK TRACE:
<xsl:value-of select="$generalfailure/stack-trace" />
</xsl:otherwise>
</xsl:choose>
</failure>
</xsl:if>
<xsl:if test="@executed='False'">
<skipped>
<xsl:attribute name="message"><xsl:value-of select="./reason/message"/></xsl:attribute>
</skipped>
</xsl:if>
</testcase>
</xsl:for-each>
</testsuite>
<!-- </redirect:write>-->
</xsl:for-each>
</xsl:for-each>
</testsuites>
</xsl:template>
</xsl:stylesheet>