Skip to content

test: pin the behaviours mutation testing and coverage left unconstrained - #150

Open
justin13888 wants to merge 8 commits into
masterfrom
test/134-pin-unconstrained-mutant-behaviours
Open

justin13888 wants to merge 8 commits into
masterfrom
test/134-pin-unconstrained-mutant-behaviours

Conversation

@justin13888

@justin13888 justin13888 commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds tests that name a behaviour for each mutant the sample left alive, and for each uncovered failure or cleanup path listed in #134. Production code is unchanged: every hunk is under #[cfg(test)] or in tests/.

  • src/plan/decide.rs
    • BX-TEST-F4: a_region_write_is_recorded_as_a_region_and_not_the_whole_file applies an interactive [[env]] over a user's ~/.zshrc. It asserts that the ledger records Mechanism::Region { comment: '#' }, not Own (invariant 1).
    • BX-TEST-F5: a_symlink_create_names_every_directory_apply_creates_for_it (decide_link) and a_tracked_copy_put_onto_this_machine_names_every_directory_apply_creates (track_onto_machine) assert the exact "creates ~/.tool 0755, ~/.tool/deep 0755" announcement (invariant 7).
    • BX-TEST-F3: an_agreement_of_up_to_64_kib_is_kept_whole_and_a_larger_one_as_its_digest fixes the whole-bytes/digest switch at a literal 65 536 bytes, and checks the fingerprint round trip on both sides of it.
    • BX-TEST-F18: an_environment_d_line_is_judged_as_exported now asserts the refusal names line 1: SCRATCH_HOME, where before it only checked is_some().
  • src/env_guard.rs
    • BX-TEST-F1: a_whole_word_opening_with_a_tilde_before_a_break_is_read_from_its_first_slash. Words like ~:notes/todo.md and ~=notes/todo.md are admitted rather than refused as another user's home. The issue's suggested ~other/x is not usable as the example: its first piece is already refused before the whole word is read, so it cannot tell the mutant apart from the original.
    • BX-TEST-F2: bashs_removal_of_a_word_ending_in_a_braced_reference_is_read checks that ${CARGO_HOME} and /opt/${TOOL} are read as removals, and that their unbraced forms are refused.
  • src/doctor/state.rs (new test module), BX-TEST-F9: a_ledger_or_fingerprints_that_cannot_be_read_is_named_with_the_reason covers a directory sitting where the file belongs. a_state_directory_that_cannot_be_listed_says_a_set_aside_copy_may_be_unseen covers a state directory at mode 0300.
  • src/plan/external.rs, BX-TEST-F10:
    • nested-repo refusal and an unreadable checkout (a_directory_inside_another_checkout_is_not_its_own_checkout)
    • not_forward's git error arm (a_git_failure_counting_local_commits_is_named)
    • every outcome of missing_dirs (missing_dirs_names_every_absent_parent_and_refuses_one_that_cannot_hold_a_clone)
    • a parent that cannot be made at apply, which stops the clone with nothing made or recorded (a_parent_that_cannot_be_made_at_apply_stops_the_clone_and_records_nothing)
    • not_forward's Ok(Err(_)) arm, where git answers the local-commit count with something that is not a number (a_local_commit_count_git_answers_with_something_other_than_a_number_is_not_counted)
  • src/fs/link.rs, BX-TEST-F11: a_staged_link_reports_what_it_replaces_and_what_it_holds covers the accessors at 174-194. a_link_whose_directory_cannot_be_opened_is_not_published covers publish refusing when the directory cannot be opened: nothing is renamed and the temporary link is removed.
  • src/config/merge.rs, src/config/values.rs, src/init.rs, BX-TEST-F18: where a test asserted only is_ok(), it now asserts the kept answer and its file, the resolved value and the empty root set, and the Prepared fields. values_in_the_local_layer_are_fine is renamed to values_in_the_local_layer_are_kept_as_answered.
  • tests/acceptance.rs, BX-TEST-F7: every_activation_the_example_declares_ran_the_bench_stub requires all four stub markers (__bench_{mise,starship,zoxide,fzf}_loaded) to be set in both zsh and bash, so a host binary cannot satisfy the row.

Tests that change a directory's mode first check whether the mode takes effect, through testing::skip_unconstructible, as the repository's other permission tests do.

Validation

Each targeted mutant fails against the new tests and passes on the original. I ran cargo-mutants in place, filtered to the new tests, each run starting with an unmutated baseline that passed:

  • cargo mutants --in-place --no-shuffle --file src/env_guard.rs --re 'env_guard.rs:1908:20: replace match guard piece with true|env_guard.rs:2331:30: replace \+ with \*' -- --lib -- bashs_removal_of_a_word_ending a_whole_word_opening_with_a_tilde gave 2 mutants tested: 2 caught.
  • cargo mutants --in-place --no-shuffle --file src/plan/decide.rs --re 'decide.rs:(74:45: replace \* with \+|707:13|992:9|1424:9)' -- --lib -- an_agreement_of_up_to_64 a_symlink_create_names a_tracked_copy_put_onto a_region_write_is_recorded caught all four target mutants: 74:45 replace * with +, 707:13 delete match arm Attach::Region{comment}, 992:9 delete match arm Action::Create in decide_link and 1424:9 delete match arm Action::Create in track_onto_machine. The regex also matched two unrelated delete field declared mutants at 235/253. Those were run only against this filtered test set, so their result says nothing either way.
  • BX-TEST-F7 was mutated by hand. With bench/stubs/fzf made non-executable, the host's /usr/bin/fzf answered instead, bx apply still converged, and the new acceptance test failed at its own assertion. Restoring the stub made it pass again.

Local gates (the pre-push hook) at a72aa42:

  • cargo fmt --check: pass.
  • cargo clippy --all-targets -- -D warnings: pass.
  • cargo test: pass.
  • cargo llvm-cov --ignore-filename-regex 'src/main\.rs' --fail-under-lines 80: pass, 97.34% lines.

Coverage gaps

These lines are still uncovered. Each is an error-mapping closure for a filesystem call that can only be made to fail by fault injection, which would need a production seam. That would be a behaviour change, so it is out of scope here:

  • src/plan/external.rs 443-445 and 515-518: remove_dir_all failing on an interrupted or partial clone. The cleanup itself is exercised: a_failed_clone_is_blocked_and_leaves_nothing_behind asserts it leaves nothing behind.
  • src/plan/external.rs 474-478: a created directory that is not portable, which missing_dirs cannot produce because it stays under the home.
  • src/fs/link.rs 230-232: the rename or directory fsync failing once the directory is open.

Risks and rollout

None. Only tests change.

Decisions taken

  1. How survivors are closed
    Taken: a behaviour-named test shown to fail on the mutant and pass on the original (accepted unrebutted)
    Rejected: declaring them equivalent - none has an argument for equivalence
    Reverses: n/a
  2. Uncovered error-mapping closures for remove_dir_all, rename and fsync failures
    Taken: left uncovered and named under Coverage gaps
    Rejected: a fault-injection seam in production code - the issue's surface is internal and allows no behaviour change; a split issue - the seam is a design change, not a remainder of this test work
    Reverses: add a test-only failure seam to fs::link::StagedLink::publish and plan::external::clone, then cover the closures

Issue

Closes #134

Unresolved review notes

C1

not_forward's Ok(Err(_)) arm in src/plan/external.rs, where git answers the local-commit count with something that is not a number, is now covered and pinned by a_local_commit_count_git_answers_with_something_other_than_a_number_is_not_counted. The test runs not_forward with a stub git on PATH that prints many. It asserts the exact note "{rev} is not a fast-forward from {head}; bx left it as it is", which differs from the zero-count arm's note. Validated by cargo test --lib plan::external (28 passed) and the pre-push gate (fmt, clippy, coverage 97.34% lines). Re-checked at 19b7293: under cargo llvm-cov --lib --text -- plan::external::tests::a_local_commit_count, line 342 of src/plan/external.rs (Ok(Err(_)) => ...) has 1 hit. The Summary now lists this test under src/plan/external.rs.
Resolved at: 19b7293

@justin13888 justin13888 added the de-slop Repository cleanup: documentation, structure, and test adequacy label Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

de-slop Repository cleanup: documentation, structure, and test adequacy

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pin the behaviours mutation testing and coverage show no test constrains

1 participant