Skip to content

feat(evidence): expose collision-safe failure artifact records #152

Closed

Description

@BenSheridanEdwards

Problem

Depends on #151: this follow-up should be reviewed and merged only after the supported runner-config/hook work in #151.

NativeProof's built-in failure capture still cannot give a reporter-neutral consumer the evidence files belonging to a failed test:

  • failureEvidenceName hashes only parent + title, so the same named test collides across spec files, projects, and retry attempts.
  • captureState intentionally returns page-source XML text, not the PNG/XML paths, and the generated afterTest hook does not expose those paths.

This leaves generic reporters unable to associate the exact files NativeProof actually saved without guessing filenames. #140 fixed sanitization/truncation collisions, but not this identity/association gap.

Acceptance criteria

  • Add a minimal config-level callback for built-in failure evidence; no Allure dependency and no new test DSL.
  • The callback receives a structured record identifying project, file, fullName, and retry attempt, plus the exact PNG/XML paths actually produced.
  • Failure filenames include the complete identity, so same-title tests differ across files, projects, and attempts.
  • Preserve the public captureState(prefix): Promise<string> API.
  • Preserve built-in default capture before the existing consumer afterTest hook.
  • Passing tests do not invoke the failure-evidence callback.
  • Capture failures do not report fictitious paths and do not replace the original test failure.

Proof required

  • Exact feat(config): expose supported runner options #151 baseline demonstrates file/project/retry collisions and no path association.
  • Regression tests prove unique paths for file/project/attempt and callback records matching files actually saved.
  • Sabotage/baseline run proves the regression test fails without the identity/path fix.
  • npm run check, npm test, npm run build, and a fresh packed-consumer type/load proof pass.
  • No device claim unless a device lane is actually run.

Activity

  1. BenSheridanEdwards commented on Sep 15, 2026

    @BenSheridanEdwards
    OwnerAuthor

    Implementation is open in #153, stacked explicitly on #151. It has not been merged or released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions