Skip to content

TR-22322: Fixed --verify data verification false failures from TestRail's legacy html field wrapping - #475

Open
acuanico-tr-galt wants to merge 1 commit into
release/1.16.1from
fix/TR-22322
Open

acuanico-tr-galt wants to merge 1 commit into
release/1.16.1from
fix/TR-22322

Conversation

@acuanico-tr-galt

@acuanico-tr-galt acuanico-tr-galt commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Issue being resolved: #471

Solution description

TRCLI sends is_legacy: True on add_case requests to trigger Markdown -> HTML conversion. On TestRail v9.8.1+, this also reformats plain-text fields like custom_automation_id; --verify falsely flagged a successfully-created case as corrupted and rolled it back.

  • Only affects the fallback comparator (fields not already special-cased as estimate/description/comment).
  • Genuine mismatches still fail, even when HTML-wrapped (verified by negative test).
  • Regex only unwraps a single outer paragraph pair - doesn't mask legitimately differing HTML content.

Changes

  • trcli/api/api_response_verify.py:92-103 - on strict-equality failure, if both values are str, retries after stripping a single outer paragraph tags. Non-string types never enter the fallback, so True/False or 1/2 still compare strictly.

  • tests/test_response_verify.py adds positive (tolerates_legacy_html_wrapping) and negative (still_fails_on_real_mismatch) parametrized tests.

Potential impacts

None.

Steps to test

  • Upload a sample JUnit xml test result file with some new cases, use with --verify flag and --case-matcher auto (must use against TestRail v9.8.1+)

PR Tasks

  • PR reference added to issue
  • README updated
  • Unit tests added/updated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant