Nunit test failure message is missing

[mikz]

Given the following example XML:

<?xml version="1.0" encoding="utf-8"?>
<test-run id="2" testcasecount="352" result="Failed(Child)" total="352" passed="349" failed="3" inconclusive="0" skipped="0" asserts="0" engine-version="3.5.0.0" clr-version="4.0.30319.42000" start-time="2022-09-20 21:16:48Z" end-time="2022-09-20 21:31:37Z" duration="888.5251836">
    <test-suite type="TestSuite" id="1000" name="workspace" fullname="workspace" runstate="Runnable" testcasecount="3688" result="Failed" site="Child" start-time="2022-09-20 21:16:48Z" end-time="2022-09-20 21:31:37Z" duration="888.525184" total="352" passed="349" failed="3" inconclusive="0" skipped="0" asserts="0">
        <properties>
            <property name="isRoot" value="True" />
        </properties>
        <failure>
            <message><![CDATA[One or more child tests had errors]]></message>
        </failure>
        <test-suite type="Assembly" id="1361" name="MafiaPigs.EditorTests.dll" fullname="/github/workspace/Library/ScriptAssemblies/MafiaPigs.EditorTests.dll" runstate="Runnable" testcasecount="352" result="Failed" site="Child" start-time="2022-09-20 21:16:48Z" end-time="2022-09-20 21:31:37Z" duration="888.385199" total="352" passed="349" failed="3" inconclusive="0" skipped="0" asserts="0">
            <properties>
                <property name="_PID" value="1140" />
                <property name="_APPDOMAIN" value="Unity Child Domain" />
                <property name="platform" value="EditMode" />
                <property name="isAssembly" value="True" />
                <property name="EditorOnly" value="True" />
                <property name="RequiresPlayMode" value="False" />
            </properties>
            <failure>
                <message><![CDATA[One or more child tests had errors]]></message>
            </failure>
            <test-suite type="TestSuite" id="1362" name="MP" fullname="MP" runstate="Runnable" testcasecount="352" result="Failed" site="Child" start-time="2022-09-20 21:16:48Z" end-time="2022-09-20 21:31:37Z" duration="888.339072" total="352" passed="349" failed="3" inconclusive="0" skipped="0" asserts="0">
                <properties />
                <failure>
                    <message><![CDATA[One or more child tests had errors]]></message>
                </failure>
                <test-suite type="TestSuite" id="1363" name="Tests" fullname="MP.Tests" runstate="Runnable" testcasecount="352" result="Failed" site="Child" start-time="2022-09-20 21:16:48Z" end-time="2022-09-20 21:31:37Z" duration="888.307736" total="352" passed="349" failed="3" inconclusive="0" skipped="0" asserts="0">
                    <properties />
                    <failure>
                        <message><![CDATA[One or more child tests had errors]]></message>
                    </failure>
                    <test-suite type="TestFixture" id="1003" name="AssetValidatorTest" fullname="MP.Tests.AssetValidatorTest" classname="MP.Tests.AssetValidatorTest" runstate="Runnable" testcasecount="351" result="Failed" site="Child" start-time="2022-09-20 21:16:48Z" end-time="2022-09-20 21:31:36Z" duration="887.960312" total="351" passed="348" failed="3" inconclusive="0" skipped="0" asserts="0">
                        <properties />
                        <failure>
                            <message><![CDATA[One or more child tests had errors]]></message>
                        </failure>
                        <output/>
                        <test-suite type="ParameterizedMethod" id="1062" name="ValidateSceneContainer" fullname="MP.Tests.AssetValidatorTest.ValidateSceneContainer" classname="MP.Tests.AssetValidatorTest" runstate="Runnable" testcasecount="56" result="Failed" site="Child" start-time="2022-09-20 21:24:21Z" end-time="2022-09-20 21:28:24Z" duration="242.562296" total="56" passed="53" failed="3" inconclusive="0" skipped="0" asserts="0">
                            <properties />
                            <failure>
                                <message><![CDATA[One or more child tests had errors]]></message>
                            </failure>
                            <test-case id="1017" name="ValidateSceneContainer(&quot;Assets/Scenes/Grid/GridTest.unity&quot;)" fullname="MP.Tests.AssetValidatorTest.ValidateSceneContainer(&quot;Assets/Scenes/Grid/GridTest.unity&quot;)" methodname="ValidateSceneContainer" classname="MP.Tests.AssetValidatorTest" runstate="Runnable" seed="78278819" result="Failed" label="Error" start-time="2022-09-20 21:25:49Z" end-time="2022-09-20 21:25:51Z" duration="2.117365" asserts="0">
                                <properties />
                                <failure>
                                    <message><![CDATA[Zenject.ZenjectException : Zenject Validation Failed!  See errors below for details.]]></message>
                                    <stack-trace><![CDATA[  at Zenject.Internal.ZenUnityEditorUtil.ValidateCurrentSceneSetup () [0x0009c] in /github/workspace/Assets/ThirdParty/Zenject/Source/Editor/ZenUnityEditorUtil.cs:82
  at MP.Tests.AssetValidatorTest.ValidateSceneContainer (System.String scenePath) [0x00009] in /github/workspace/Assets/Tests/EditorMode/AssetValidatorTest.cs:58
  at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
  at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <b67d2f60bf2548a58dc569b37fe71c3d>:0 ]]></stack-trace>
                                </failure>
                            </test-case>
                        </test-suite>
                    </test-suite>
                </test-suite>
            </test-suite>
        </test-suite>
    </test-suite>
</test-run>

The GitHub check should show Zenject.ZenjectException : Zenject Validation Failed! See errors below for details. plus the stack trace. But it just shows the stack trace.

The junit <failure message="" should be set correctly with this transform:

publish-unit-test-result-action/python/publish/xslt/nunit3-to-junit.xslt

Lines 118 to 120 in 60624fd

	<failure message="{./message}">
	<xsl:value-of select="./stack-trace"/>
	</failure>

But somehow it still does not appear on GitHub.

[mikz]

Traced it to

publish-unit-test-result-action/python/publish/__init__.py

Line 716 in 60624fd

message = case.message if case.result in ['skipped', 'disabled'] else case.content

[EnricoMi]

Right, that was some special logic coming from JUnit files. Maybe this should be generalized and case.message and case.content should always be used.

[mikz]

I've went with and it works for me.

https://github.com/timewarpinc/publish-unit-test-result-action/blob/53edcbb45c9c18eae417ffee30642e588e589ef1/python/publish/__init__.py#L711-L718

[EnricoMi]

I think this has to be fixed in a different place. That message is used to identify identical test cases. The message used in the annotations is defined elsewhere, and should be changed there.

I have created a fix, please test the fix as follows (includes fix for #354):

uses: EnricoMi/publish-unit-test-result-action/composite@branch-stdout-and-stderr

[EnricoMi]

This has been fixed in #358.