Skip to content

Add generated filesystem round-trip and mutation property tests #56

Description

@plx

Parent epic

#31

Problem and impact

The existing suite has no model-level assertion that a generated guide can be parsed and verifies the filesystem it came from. That missing invariant allowed raw syntax-sensitive filenames, directory symlinks, empty roots, and zero indentation to ship despite all 127 tests passing.

Required properties

Build bounded generated temporary filesystem trees and assert, for every supported case:

filesystem -> dump -> parse/check -> verify(same filesystem) == success

Then mutate the tree and assert expected failure unless the guide explicitly permits omission:

  • delete a represented item;
  • rename it;
  • change file to directory or vice versa;
  • change nesting;
  • add/remove entries around commented and uncommented placeholders; and
  • alter case/normalization on relevant filesystems.

The generator corpus must include ordinary nesting, wide/deep trees, grammar metacharacters, choices/escapes, Unicode, whitespace, empty directories, exclusions, and every approved/rejected symlink/special-entry class. Unsupported names/types must generate a precise rejection property rather than be excluded from generation.

Required red-before-fix evidence

Seed deterministic regressions for:

  • report plus report#draft;
  • bracketed filename;
  • literal ...;
  • directory symlink;
  • empty root/file-valued root;
  • --indent 0; and
  • malformed hierarchy fixture where appropriate.

Show those seeds fail against audited revision bf40806 before relying on randomized cases.

Implementation guidance

  • Use deterministic seeds and bounded sizes so the normal suite is reproducible and fast.
  • Separate portable names from platform-specific generators/capability checks.
  • Keep output outside the enumerated root to avoid self-inclusion.
  • Exercise both dump stdout and init file paths.
  • Compare an independent filesystem model to the parsed guide; do not only call production code in a tautological cycle.
  • Persist every discovered minimal counterexample as a named regression.

Acceptance criteria

  • Deterministic historical seeds are red on the audited revision and green on the candidate.
  • Supported generated trees round-trip exactly.
  • Unsupported items always produce the approved error without partial misleading output.
  • Filesystem mutations are detected according to partial-list/placeholder semantics.
  • Tests run in bounded time on all supported platforms.
  • Seeds/counterexamples are easy to reproduce from CI output.

Validation commands

cargo test --all-targets --all-features --locked
cargo test --all-targets --all-features --locked --release

Run a larger iteration count in scheduled/manual CI and record seed/case counts.

Relationships

Non-goals

Property tests supplement but do not replace each defect ticket's focused regression.

Approved #35 filesystem trust handoff

The maintainer-approved filesystem trust decision is recorded in PR #85 and the normative docs/v0.2-contract.md. It is a binding input to this issue, but it does not make the current runtime conformant.

Generated round-trip and mutation properties must wait for the relevant #43, #45, #49, and #51 behavior; the native blocker graph records the newly required #45/#49/#51 edges. Cover supported regular trees and the relevant generated/input/delivery rejection classes, explicit exclusions, root aliases, unresolved-parent spellings, sentinel non-disclosure, and create-new failure behavior without duplicating the focused owners' row suites.

Model a stable tree and observed mutation failures. Do not describe randomized mutation coverage as proof of hostile-concurrent-replacement safety or a sandbox.

Preserve the exact normative outcomes and focused ownership. If this issue changes runtime behavior, capture its own red-before-change evidence; coordinating work must not claim another issue's trust rows. Never claim sandbox or hostile-concurrent-replacement safety.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0Critical, urgent issuescomponent: test-infraFixtures, property tests, fuzzing, coverage, and benchmarksdiscovered-by-codexIssues discovered via review by codex.domain: correctnessBehavioral correctness and accurate validation resultsdomain: reliabilityFail-closed, deterministic, and robust operational behaviorproduction-readinessPart of the audited path from prototype to production-ready userelease-blockerMust close before the production-readiness release gate can passrisk: cross-platformBehavior depends on operating-system or filesystem semanticsrisk: false-successCan report success without validating the intended statetarget: v0.2.0Must be resolved or explicitly waived before the v0.2 production audittestingAdds or strengthens test infrastructure and assuranceworkflow:production-readinessIncluded in automatic production-readiness work selectionworkflow:production-readiness-leafIndependently actionable work item in the production-readiness sequence

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions