fix(bin): recover legacy child endpoint records during forced secondmate teardown - #8
Merged
Merged
Conversation
…ate teardown Forced teardown of a secondmate home refused outright on the first child task record written before the endpoint_task_id binding existed, stranding the whole home with no supported lifecycle action. Both recursive preflights over a secondmate's own children called the bare offline endpoint validator, which cannot settle that one record shape and returns the recoverable code 2. The stated reason for not wiring the live resolver in - that a recursive preflight holds no per-child metadata lock - no longer held: preflight_descendant_task_locks already acquires every descendant task's control and metadata lock and holds them until teardown exits. Route both loops through one shared validate_child_task_endpoint, which upgrades a legacy record through fm_backend_resolve_task_endpoint only while this teardown provably holds that exact child's own metadata lock, reads the live label proof as the child home (zellij and cmux scope runtime titles by owning home tag), and defers rather than deciding on the cheap pre-lock pass. Every other refusal, and a legacy record the live endpoint cannot prove, still refuses with nothing mutated. Also guard fm_backend_zellij_tab_matches_label against an unread tab list: an absent or unreachable zellij produces no output, and `jq -e` over empty input exits 0 on jq 1.6, so the label proof could pass vacuously and bind an endpoint nothing was read from.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Fix teardown of legacy child task records inside a secondmate home without weakening any landed-work or ownership guard.
Task as given:
What was found (deliberate decisions a reviewer reading only the diff would not know):
Root cause. Forced teardown of a secondmate home refused outright on the first child task record written before the endpoint_task_id binding existed. Both recursive preflights over a secondmate's own children - validate_firstmate_home_children_removal and preflight_firstmate_home_herdr_children - called the bare offline fm_backend_validate_task_endpoint, which cannot settle that one record shape and returns the recoverable code 2; both loops treated any nonzero as terminal. Reproduced end to end with a fixture secondmate home holding a legacy zellij child: "REFUSED: legacy Zellij endpoint metadata for task childtask lacks an exact task binding". The same fixture with endpoint_task_id present tears down cleanly, which is the counterfactual that isolates the cause.
The in-code KNOWN LIMITATION comments claimed the resolver could not be wired in because "a recursive preflight that currently takes no locks at all" cannot hold the child's own metadata lock that fm_backend_resolve_task_endpoint's write requires. That claim was stale: preflight_descendant_task_locks (added in commit 2d2be63, 2026-08-08) already acquires every descendant task's control lock and metadata lock and holds them until teardown exits, while the limitation comment was written later (5543cd0, 2026-08-20) without accounting for it. Confirming that stale claim, rather than accepting the brief's hypothesis that new per-child locking had to be built, is what made the minimal fix possible. Both stale comment blocks were removed rather than left to mislead.
The shared fix. One new helper validate_child_task_endpoint is now the single endpoint-identity test both recursive preflights run over a child record. It upgrades ONLY the one recoverable legacy shape through fm_backend_resolve_task_endpoint, and only while teardown provably holds that exact child's own metadata lock. That proof is structural, not positional: teardown_holds_descendant_meta_lock checks membership of the child's own meta-lock path in DESCENDANT_LOCK_PATHS, so the guard does not depend on which call site asks or on call ordering. The cheap pre-lock pass therefore DEFERS the recoverable shape (returns 2) instead of deciding it: refusing there would strand a record the locked pass can prove, and accepting there would authorize an unproven endpoint. The herdr preflight, which only ever runs after the lock preflight, refuses on that deferral code, so it fails closed if it ever saw an unlocked record.
Deliberate choice on scoping the live proof: the label proof is read as the CHILD home (FM_HOME/FM_ROOT set to the child home, FM_ROOT_OVERRIDE unset), not the parent. zellij and cmux scope their runtime titles by owning home tag, and forced cleanup already re-scopes its zellij child kills the same way for the same reason. This is why the resolver call is wrapped in a subshell and the record is then re-validated offline in the caller: the subshell would otherwise lose the FM_BACKEND_VALIDATED_* output globals.
Deliberate choice on diagnostics: the pre-lock pass suppresses the validator's stderr until the code is known, then re-emits any refusal verbatim by calling the validator a second time. Without that, a successful recovery printed a "lacks an exact task binding" REFUSED line followed by the recovery note, which reads as a failure to an operator. No refusal is hidden - only the deferral is silent.
Second defect found and fixed while proving the first. fm_backend_zellij_tab_matches_label never checked that it had actually read a tab list. An absent or unreachable zellij prints nothing, and
jq -eover EMPTY input exits 0 on jq 1.6 (verified live on this machine), so the identity proof passed vacuously and the resolver would have bound an endpoint that nothing was read from - exactly the "do not make a record appear provable when it cannot be proven" boundary this task must not weaken. Guarded with an explicit failure on a failed CLI call and on empty output. This is a pre-existing hole on the top-level teardown path too, so fixing it at the shared matcher rather than at the new call site is intentional. herdr's and cmux's matchers were checked and already refuse on their CLI's nonzero exit; they were left alone rather than speculatively hardened.Safety boundaries deliberately preserved and asserted by tests: an unprovable legacy child still refuses with the child record, the secondmate home, the parent record, and the child worktree's unlanded work all intact and no binding written; a child whose metadata lock is held elsewhere refuses without any runtime read and without binding; a current-format child is still decided offline with no label re-derivation.
Regression coverage added (colocated, extending existing suites, behavior-only - no assertions over implementation source):
Both new groups were verified to FAIL against the pre-fix scripts and pass after, so they are genuine regressions.
Validation already run locally: bin/fm-lint.sh clean; the three touched suites green (fm-secondmate-safety 81 ok, fm-backend-zellij 65 ok, fm-teardown-endpoint-safety 8 ok); bin/fm-test-run.sh --changed --base HEAD ran 22 suites with exactly two failures, BOTH reproduced identically with my script changes reverted and therefore baseline, not regressions:
Scope was kept deliberately narrow: no operator documentation changed because no maintained prose surface documented the limitation (grepped), and no AGENTS.md change because the task produced no repo-wide durable knowledge - the jq-1.6 fact lives in the code comment where it is load-bearing.
What Changed
bin/fm-teardown.sh: both recursive child preflights (validate_firstmate_home_children_removalandpreflight_firstmate_home_herdr_children) now route through one sharedvalidate_child_task_endpointhelper instead of calling the bare offline endpoint validator, so a child record predating theendpoint_task_idbinding no longer makes a forced secondmate-home teardown refuse outright. The helper upgrades only that one recoverable legacy shape viafm_backend_resolve_task_endpoint, read as the child's own home, and only whenteardown_holds_descendant_meta_lockproves the child's own metadata-lock path is inDESCENDANT_LOCK_PATHS; the cheap pre-lock pass defers that shape rather than deciding it, and the herdr preflight refuses on the deferral. Stale KNOWN LIMITATION comments claiming a lock could not be held were removed, and the pre-lock pass suppresses then re-emits validator stderr verbatim so a successful recovery no longer prints a REFUSED line.bin/backends/zellij.sh:fm_backend_zellij_tab_matches_labelnow fails when the zellij CLI call fails or returns no output, instead of passing vacuously becausejq -eover empty input exits 0 - previously an absent or unreachable zellij could "prove" a label nothing was read from.tests/fm-secondmate-safety.test.shadds four end-to-end forced-teardown cases (provable legacy child recovers, unprovable legacy child refuses with record/home/parent/unlanded work intact and no binding written, current-format child stays offline, foreign metadata lock refuses without a runtime read);tests/fm-backend-zellij.test.shadds no-tab-list cases for both no-read shapes;docs/configuration.mdanddocs/verification/runtime-backends.mdreplace the "known limitation / left for a follow-up" text with the new contract and coverage.Risk Assessment
✅ Low: Bounded shared-helper fix whose fail-closed behavior I traced end to end (deferral is always re-decided under the child's own meta lock, and any still-unlocked record refuses at the herdr preflight), with behavior-level regression coverage for both the recovered and the refused paths and no safety guard weakened.
Testing
Targeted validation, not a full-suite run. Built a fixture secondmate home holding one legacy-format zellij child record and drove
bin/fm-teardown.sh domain --forceexactly as an operator would, at both the base and target commits: at base ba5f4c4 the run refuses with "legacy Zellij endpoint metadata for task child lacks an exact task binding", makes zero live endpoint reads and strands the whole home; at HEAD the same fixture reads the live tab list, re-derives the binding from the endpoint's own fm-child label and cleans the home up. The three safety boundaries were exercised in the same transcript and all hold: an unprovable label refuses with the child record, the secondmate record, the parent home and the child worktree's unlanded work all intact and no endpoint_task_id written; a child whose own metadata lock is held by another process refuses with zero runtime reads and no binding; a current-format child is still decided offline with no label re-derivation. The three touched suites (fm-secondmate-safety, fm-backend-zellij, fm-teardown-endpoint-safety) pass in full, and both new test groups were verified to fail against the reverted pre-fix scripts, so they are genuine regressions. One environment fix was needed: this machine's global git init.templateDir installs pre-commit hooks into every test-created repo, which breaks the unrelated treehouse-lease case, so the runs neutralized it via GIT_TEMPLATE_DIR; CI is unaffected. The pre-existing tests/fm-teardown.test.sh herdr-child-preflight failure reproduces identically with the change reverted and is reported as a baseline failure. No UI surface is involved, so evidence is CLI transcripts plus post-run filesystem state rather than screenshots.Evidence: Forced secondmate teardown, target commit 40d7ec9 - four legacy-child cases, full operator transcript plus post-run state
Source: Forced secondmate teardown, target commit 40d7ec9 - four legacy-child cases, full operator transcript plus post-run state
Evidence: Same fixture at base ba5f4c4 - the failure being fixed
Source: Same fixture at base ba5f4c4 - the failure being fixed
Evidence: Reproduction driver used to produce both transcripts (builds the fixture with the repo's own test helpers)
Source: Reproduction driver used to produce both transcripts (builds the fixture with the repo's own test helpers)
Evidence: Before/after contrast, CASE 1 (legacy child whose live tab proves its own fm-child label)
Evidence: Safety boundaries held at HEAD (cases 2 and 4)
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
bin/fm-teardown.sh:2104- Recovery at the locked pass (bin/fm-teardown.sh:2414 -> validate_child_task_endpoint:2104) durably appends endpoint_task_id to a child record before later preflights conclude. If a LATER refusal fires (e.g. teardown_herdr_preflight_target for a sibling child, or preflight_firstmate_home_herdr_children), the operator sees a message ending in "nothing was changed" / "forced teardown changed nothing" even though one child's metadata record was already rewritten. Same tradeoff the top-level path at bin/fm-teardown.sh:451 already documents and accepts (metadata-only, never overwrites an existing binding), so this is a message-honesty nit, not a safety hole.bin/backends/zellij.sh:286- fm_backend_zellij_pane_exists (bin/backends/zellij.sh:285-286) has the exact vacuous-proof shape this change fixed one function below:jq -e '[...]|length>0'over empty stdin exits 0 on jq 1.6 (re-verified here), so an unreadable list-panes answer "proves" the pane exists. Reachability is narrow - its only caller, fm_backend_zellij_target_ready:391, first passes fm_backend_zellij_session_exists (a real read), so it needs list-sessions to succeed while list-panes returns nothing - and it is pre-existing, untouched code outside this task's endpoint-identity path. Noted, not a merge blocker.tests/fm-teardown.test.sh- Pre-existing baseline failure, reproduced identically at base ba5f4c4 and at HEAD:bin/fm-test-run.sh tests/fm-teardown.test.shfails withnot ok - herdr-child-preflight: refusal did not explain its non-mutating boundary. Not caused by this change (verified by reverting bin/fm-teardown.sh and bin/backends/zellij.sh to base and re-running: same single failure). Author states another crewmate owns that file concurrently, so it is deliberately not fixed here; user decides whether to accept it as baseline.bin/fm-backend.sh:670- On a SUCCESSFUL legacy-child recovery the operator still seesREFUSED: legacy Zellij endpoint metadata for task child lacks an exact task binding; preserving task state.printed immediately before thenote: ... re-derived its binding ...line and theteardown domain completeline (see evidence fixed-40d7ec9.txt, CASE 1). The change's own pre-lock stderr suppression works; this line comes from the unsuppressed validator call inside fm_backend_resolve_task_endpoint (bin/fm-backend.sh:670), which is untouched pre-existing behavior shared with the top-level teardown path. Informational only - it slightly under-delivers the intent's stated diagnostics goal, but no behavior or safety boundary is affected.bin/fm-test-run.sh tests/fm-secondmate-safety.test.sh tests/fm-backend-zellij.test.sh tests/fm-teardown-endpoint-safety.test.sh(all green; ran with GIT_TEMPLATE_DIR neutralized to an empty dir so the machine-global git init.templateDir does not inject pre-commit hooks into test-created repos)Regression proof:git checkout ba5f4c4 -- bin/fm-teardown.sh bin/backends/zellij.shthen re-ranbin/fm-test-run.sh tests/fm-secondmate-safety.test.sh tests/fm-backend-zellij.test.sh- exactly the two new groups fail (forced teardown refused a legacy child whose live endpoint proves its own label,an unreachable zellij must not prove a tab label); files restored to HEAD afterwardsManual end-to-end CLI reproduction via /tmp/no-mistakes-evidence/01M0R8QDJ9S0QA2CA2YCJW6PRE/legacy-child-e2e.sh - builds a fixture secondmate home with one legacy zellij child record and runsbin/fm-teardown.sh domain --forcefor four cases (provable legacy child, unprovable label, current-format child, child metadata lock held by another process), captured at both base ba5f4c4 and HEAD 40d7ec9Post-run state assertions in each case: secondmate home presence, child .meta presence, child worktree unlanded-work sentinel, presence of anendpoint_task_id=binding, and the count of live zellij endpoint readsBaseline check:bin/fm-test-run.sh tests/fm-teardown.test.shat HEAD and again with bin/fm-teardown.sh + bin/backends/zellij.sh reverted to ba5f4c4 - identical single failureherdr-child-preflightConfirmed the jq version the zellij guard's comment depends on:jq --version-> jq-1.6Worktree left clean (git status --porcelainempty); temp fixture dirs under /tmp removeddocs/configuration.md:90- Change intent asserted no prose surface documented the legacy-child teardown limitation; two did (docs/configuration.md, docs/verification/runtime-backends.md). Both fixed here, but the author's grep-based scoping claim in the PR body remains inaccurate and may want correcting before merge.✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.