test: pin the behaviours mutation testing and coverage left unconstrained - #150
Open
justin13888 wants to merge 8 commits into
Open
justin13888 wants to merge 8 commits into
justin13888 wants to merge 8 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds tests that name a behaviour for each mutant the sample left alive, and for each uncovered failure or cleanup path listed in #134. Production code is unchanged: every hunk is under
#[cfg(test)]or intests/.src/plan/decide.rsa_region_write_is_recorded_as_a_region_and_not_the_whole_fileapplies an interactive[[env]]over a user's~/.zshrc. It asserts that the ledger recordsMechanism::Region { comment: '#' }, notOwn(invariant 1).a_symlink_create_names_every_directory_apply_creates_for_it(decide_link) anda_tracked_copy_put_onto_this_machine_names_every_directory_apply_creates(track_onto_machine) assert the exact "creates ~/.tool 0755, ~/.tool/deep 0755" announcement (invariant 7).an_agreement_of_up_to_64_kib_is_kept_whole_and_a_larger_one_as_its_digestfixes the whole-bytes/digest switch at a literal 65 536 bytes, and checks the fingerprint round trip on both sides of it.an_environment_d_line_is_judged_as_exportednow asserts the refusal namesline 1: SCRATCH_HOME, where before it only checkedis_some().src/env_guard.rsa_whole_word_opening_with_a_tilde_before_a_break_is_read_from_its_first_slash. Words like~:notes/todo.mdand~=notes/todo.mdare admitted rather than refused as another user's home. The issue's suggested~other/xis not usable as the example: its first piece is already refused before the whole word is read, so it cannot tell the mutant apart from the original.bashs_removal_of_a_word_ending_in_a_braced_reference_is_readchecks that${CARGO_HOME}and/opt/${TOOL}are read as removals, and that their unbraced forms are refused.src/doctor/state.rs(new test module), BX-TEST-F9:a_ledger_or_fingerprints_that_cannot_be_read_is_named_with_the_reasoncovers a directory sitting where the file belongs.a_state_directory_that_cannot_be_listed_says_a_set_aside_copy_may_be_unseencovers a state directory at mode 0300.src/plan/external.rs, BX-TEST-F10:a_directory_inside_another_checkout_is_not_its_own_checkout)not_forward's git error arm (a_git_failure_counting_local_commits_is_named)missing_dirs(missing_dirs_names_every_absent_parent_and_refuses_one_that_cannot_hold_a_clone)a_parent_that_cannot_be_made_at_apply_stops_the_clone_and_records_nothing)not_forward'sOk(Err(_))arm, where git answers the local-commit count with something that is not a number (a_local_commit_count_git_answers_with_something_other_than_a_number_is_not_counted)src/fs/link.rs, BX-TEST-F11:a_staged_link_reports_what_it_replaces_and_what_it_holdscovers the accessors at 174-194.a_link_whose_directory_cannot_be_opened_is_not_publishedcoverspublishrefusing when the directory cannot be opened: nothing is renamed and the temporary link is removed.src/config/merge.rs,src/config/values.rs,src/init.rs, BX-TEST-F18: where a test asserted onlyis_ok(), it now asserts the kept answer and its file, the resolved value and the empty root set, and thePreparedfields.values_in_the_local_layer_are_fineis renamed tovalues_in_the_local_layer_are_kept_as_answered.tests/acceptance.rs, BX-TEST-F7:every_activation_the_example_declares_ran_the_bench_stubrequires all four stub markers (__bench_{mise,starship,zoxide,fzf}_loaded) to be set in both zsh and bash, so a host binary cannot satisfy the row.Tests that change a directory's mode first check whether the mode takes effect, through
testing::skip_unconstructible, as the repository's other permission tests do.Validation
Each targeted mutant fails against the new tests and passes on the original. I ran cargo-mutants in place, filtered to the new tests, each run starting with an unmutated baseline that passed:
cargo mutants --in-place --no-shuffle --file src/env_guard.rs --re 'env_guard.rs:1908:20: replace match guard piece with true|env_guard.rs:2331:30: replace \+ with \*' -- --lib -- bashs_removal_of_a_word_ending a_whole_word_opening_with_a_tildegave2 mutants tested: 2 caught.cargo mutants --in-place --no-shuffle --file src/plan/decide.rs --re 'decide.rs:(74:45: replace \* with \+|707:13|992:9|1424:9)' -- --lib -- an_agreement_of_up_to_64 a_symlink_create_names a_tracked_copy_put_onto a_region_write_is_recordedcaught all four target mutants:74:45 replace * with +,707:13 delete match arm Attach::Region{comment},992:9 delete match arm Action::Create in decide_linkand1424:9 delete match arm Action::Create in track_onto_machine. The regex also matched two unrelateddelete field declaredmutants at 235/253. Those were run only against this filtered test set, so their result says nothing either way.bench/stubs/fzfmade non-executable, the host's/usr/bin/fzfanswered instead,bx applystill converged, and the new acceptance test failed at its own assertion. Restoring the stub made it pass again.Local gates (the pre-push hook) at
a72aa42:cargo fmt --check: pass.cargo clippy --all-targets -- -D warnings: pass.cargo test: pass.cargo llvm-cov --ignore-filename-regex 'src/main\.rs' --fail-under-lines 80: pass, 97.34% lines.Coverage gaps
These lines are still uncovered. Each is an error-mapping closure for a filesystem call that can only be made to fail by fault injection, which would need a production seam. That would be a behaviour change, so it is out of scope here:
src/plan/external.rs443-445 and 515-518:remove_dir_allfailing on an interrupted or partial clone. The cleanup itself is exercised:a_failed_clone_is_blocked_and_leaves_nothing_behindasserts it leaves nothing behind.src/plan/external.rs474-478: a created directory that is not portable, whichmissing_dirscannot produce because it stays under the home.src/fs/link.rs230-232: the rename or directoryfsyncfailing once the directory is open.Risks and rollout
None. Only tests change.
Decisions taken
Taken: a behaviour-named test shown to fail on the mutant and pass on the original (accepted unrebutted)
Rejected: declaring them equivalent - none has an argument for equivalence
Reverses: n/a
remove_dir_all, rename andfsyncfailuresTaken: left uncovered and named under Coverage gaps
Rejected: a fault-injection seam in production code - the issue's surface is internal and allows no behaviour change; a split issue - the seam is a design change, not a remainder of this test work
Reverses: add a test-only failure seam to
fs::link::StagedLink::publishandplan::external::clone, then cover the closuresIssue
Closes #134
Unresolved review notes
C1
not_forward'sOk(Err(_))arm insrc/plan/external.rs, where git answers the local-commit count with something that is not a number, is now covered and pinned bya_local_commit_count_git_answers_with_something_other_than_a_number_is_not_counted. The test runsnot_forwardwith a stubgitonPATHthat printsmany. It asserts the exact note "{rev}is not a fast-forward from{head}; bx left it as it is", which differs from the zero-count arm's note. Validated bycargo test --lib plan::external(28 passed) and the pre-push gate (fmt, clippy, coverage 97.34% lines). Re-checked at 19b7293: undercargo llvm-cov --lib --text -- plan::external::tests::a_local_commit_count, line 342 ofsrc/plan/external.rs(Ok(Err(_)) => ...) has 1 hit. The Summary now lists this test undersrc/plan/external.rs.Resolved at: 19b7293