Skip to content

feat(xUnitXmlPublisher): Include assertion message in test output#208

Open
stefanbesler wants to merge 2 commits into
tcunit:masterfrom
stefanbesler:feature/xUnitXmlPublisher_AssertionFailure
Open

feat(xUnitXmlPublisher): Include assertion message in test output#208
stefanbesler wants to merge 2 commits into
tcunit:masterfrom
stefanbesler:feature/xUnitXmlPublisher_AssertionFailure

Conversation

@stefanbesler

@stefanbesler stefanbesler commented Mar 18, 2023

Copy link
Copy Markdown
Contributor

This PR write the actual assertion failure to the output of xUnitXmlPublisher. Previously only the message from test was written into message.

@dfreiberger fyi

@stefanbesler stefanbesler force-pushed the feature/xUnitXmlPublisher_AssertionFailure branch from 55a1946 to c4f6242 Compare November 18, 2023 16:37
@stefanbesler stefanbesler force-pushed the feature/xUnitXmlPublisher_AssertionFailure branch from c4f6242 to c7c8563 Compare November 18, 2023 16:40
@stefanbesler

stefanbesler commented Nov 18, 2023

Copy link
Copy Markdown
Contributor Author

Maybe some explanation on how to verify this:

  • Enable xUnit publishing by setting the following TcUnit parameters in TcUnitVerifier.plcproj
    • XUNITENABLEPUBLISH = TRUE

image

If any test fails, the C:\tcunit_xunit_testresults.xml will now correctly show the EXP and ACT values in the assertion message

<testsuites disabled="" failures="116" tests="144" time="6.1428994">
   <testsuite id="0" name="PRG_TEST.PrimitiveTypes" tests="46" failures="23" time="1.4893e-3">
      <testcase name="Test_ANY_Equals" classname="PRG_TEST.PrimitiveTypes" time="2.35e-5" status="PASS" />
      <testcase name="Test_ANY_Differ_DataType" classname="PRG_TEST.PrimitiveTypes" time="1.47e-5" status="FAIL">
         <failure message="Values differ; EXP: (Type class = INT), ACT: (Type class = UINT)" type="Array2D_LREAL" />
      </testcase>
      <testcase name="Test_BOOL_Equals" classname="PRG_TEST.PrimitiveTypes" time="3.8e-6" status="PASS" />
      <testcase name="Test_BOOL_Differ" classname="PRG_TEST.PrimitiveTypes" time="1.18e-5" status="FAIL">
         <failure message="Values differ; EXP: TRUE, ACT: FALSE" type="Array2D_LREAL" />
      </testcase>
      <testcase name="Test_BYTE_Equals" classname="PRG_TEST.PrimitiveTypes" time="4.4e-6" status="PASS" />
      <testcase name="Test_BYTE_Differ" classname="PRG_TEST.PrimitiveTypes" time="1.14e-5" status="FAIL">
         <failure message="Values differ; EXP: 0xAB, ACT: 0xCD" type="Array2D_LREAL" />
      </testcase>

Without the PR, failure > message only shows "Values differ"

@sagatowski

sagatowski commented Nov 23, 2023

Copy link
Copy Markdown
Member

I'll start to look into this as soon as your previous PR is fixed/completed.

See: #210 & #226

@sagatowski

Copy link
Copy Markdown
Member

@stefanbesler Have you noticed the problems described in #226?

@dfreiberger

Copy link
Copy Markdown
Contributor

@stefanbesler thanks for this, it will be good to have these more granular results.

@TimSalzmann

Copy link
Copy Markdown

@stefanbesler, Hi, I really want to have this feature. If I can help, please let me know

@sagatowski

Copy link
Copy Markdown
Member

I agree, this is a great feature.
I do however have to get the stuff fixed that will be included in version 1.3 first.
For this milestone (https://github.com/tcunit/TcUnit/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+1.3.0.0%22) there is only one ticket left to be solved, and this one has to be solved first before I look into integrating more PRs.

@dfreiberger & @TimSalzmann, you are of course welcome to help me figure out the problem!

@sagatowski sagatowski added this to the Release 1.4.0.0 milestone Nov 30, 2023
@tcunit tcunit deleted a comment from beslst Dec 2, 2023
@tcunit tcunit deleted a comment from beslst Dec 2, 2023
@tcunit tcunit deleted a comment from beslst Dec 2, 2023
@tcunit tcunit deleted a comment from beslst Dec 2, 2023
@sagatowski

Copy link
Copy Markdown
Member

Realized that the discussion in this PR went very OffTopic, cleared it up. Sorry for that!

@stefanbesler I'm working on the last pieces to release 1.3. After that, I will look into this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants