fix(bin): honor evidence-backed backend deviations - #2875
Open
karotkriss wants to merge 6 commits into
Open
Conversation
Inherited config convergence is primary-authoritative, so a secondmate home that pinned config/backend after repeated silent worker deaths had that pin reverted at the next sync, with nothing said in either direction. The home believed it had mitigated a risk it had not. Keep the primary authoritative and add the one sanctioned, opt-in escape: a home may write config/<item>.deviation beside a declared deviable item (backend today). The deviating value stays in the config item itself, where every consumer already reads it, so the record only states that this home holds its own value there and why. The record must carry an evidence line. Where the values actually differ, convergence leaves the home's value alone and names the divergence back to the primary as a SECONDMATE_SYNC line carrying both values and the evidence, on every sync. A record that cannot be honored - a non-deviable item, no evidence, an unsafe or oversized file - is reported with its reason and the item converges to the primary value. The primary revokes by removing the record. Remote homes behave identically through their own receiver, whose deviation result the pushing primary relays as the same divergence line.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains. Reviews (2): Last reviewed commit: "no-mistakes: apply CI fixes" | Re-trigger Greptile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Fix GitHub issue #2832: inherited config/backend convergence silently reverts a secondmate home's evidence-based local backend pin. A secondmate home hit repeated silent worker deaths that correlated with the auto-detected experimental backend and set its local config/backend to the verified backend as a deliberate, evidence-backed deviation. The primary's inherited-config convergence reverted that file within the hour, working exactly as documented, and the home only noticed because it checked which backend a NEW worker actually landed on rather than trusting the file it had written. There was no sanctioned way for a home to hold a deliberate local deviation, and a silent revert of a safety-motivated setting is the worst failure shape: the home believes it mitigated a risk it has not.
Required fix shape, decided before implementation: the primary STAYS authoritative and no default behavior changes. Add an explicit deviation record the secondmate may write next to the config item, with a REQUIRED evidence line, which convergence honors AND reports back to the primary as a named divergence line at sync time. Visible, revocable, and never silent in either direction. Colocated tests must cover honoring the record, surfacing the divergence, and the primary revoking it.
Decisions and tradeoffs made while implementing:
The PR must use 'Fixes #2832' and must state the VISION.md angle explicitly: authority stays explicit and is never inferred (this is a record the home writes, never a value the primary can push, and inheritance stays primary-authoritative for every home that holds no record); the interface stays honest under load, so silence never hides a failure, a decision, or a risk (both honoring and reverting are now named to the primary); and new capability ships as an opt-in option to enable, never as a default-behavior change.
What Changed
config/backend.deviationrecords so local and remote secondmate homes can retain a verified backend value until the record is removed or rejected.SECONDMATE_SYNC:lines across bootstrap, spawn, and config-push flows, with coverage for local and remote convergence, unsafe records, quiet agreement, and revocation.VISION.mdcontracts: authority remains explicit because the home writes the record, inheritance remains primary-authoritative without one, divergence never silently hides a decision or risk, and the new capability changes no default behavior. Fixes Inherited config/backend silently reverts a secondmate's evidence-based local override #2832.Risk Assessment
✅ Low: The deviation contract is consistently enforced across local, remote, launch, bootstrap, and mid-session convergence paths, with prior silent and unsafe cases resolved.
Testing
No prior baseline command output was supplied. The focused deviation regression suite and complete remote secondmate lifecycle E2E passed, while manual local and remote sync checks captured reviewer-visible evidence proving the hold, named relay, quiet agreement, persisted state, and primary revocation.
Evidence: Backend deviation sync transcript
Source: Backend deviation sync transcript
Shows local and remote deviations retainingtmux, the primary-visibleSECONDMATE_SYNCline with both values and evidence, quiet agreement, and revocation restoringherdr.Evidence: Persisted remote deviated backend
Source: Persisted remote deviated backend
tmuxEvidence: Persisted local backend after revocation
Source: Persisted local backend after revocation
herdrPipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 4 issues found → auto-fixed (3) ✅
bin/fm-remote-inherit.sh:174- The accepted criterion says, "The divergence line is only printed where convergence would actually have changed the destination." For a remoteputwhere both values aretmux, this callsdeviation_holdsbefore the equality check, reports a false divergence, and exits. Theabsentpath similarly reports when both sides are absent. Compare destination state first, then report only a real divergence with the bounded local and primary values.bin/fm-remote-inherit.sh:174- An honored remote deviation exits beforecommit_generation, leaving the replay barrier stale. After an unknown-completion reorder, a newer generation can be honored without being recorded; once the record is revoked, a delayed older payload can pass the stale generation check and overwrite the current primary value. Commit every validated generation before any deviation exit.bin/fm-bootstrap.sh:535- The accepted criterion requires honoring and rejection to be named to the primary on every sync. This relay loop runs only when the entire multi-item remote push succeeds. Ifconfig/backendemits a deviation and a later item fails, the failure branch reduces the captured output to its first line and silently loses the divergence. Relay captured deviation lines before branching on the overall command status.bin/fm-config-push.sh:152- The accepted criterion says the pushing primary relays remotedeviation:anddeviation-rejected:output as the sameSECONDMATE_SYNCdivergence line. The mid-session remote config-push path only indents the receiver's raw output, so this supported convergence path never produces the required primary-facing line. Apply the same relay formatting used by bootstrap.🔧 Fix: Commit honored remote generations before divergence exits
5 errors still open:
bin/fm-remote-inherit.sh:175- The criterion says divergence is reported only when convergence would change the destination. A remoteputwith matching primary and local values callsdeviation_holdsbefore comparing them, so it falsely reports a divergence.absentlikewise reports when both sides are absent. Compare destination state before evaluating the record.bin/fm-remote-inherit.sh:140- Remote divergence output omits the bounded local and primary values required by the documented identical behavior. For localtmuxagainst primaryherdr, this emits onlyheld locally: <evidence>. Include both values, usingabsencewhere appropriate.bin/fm-bootstrap.sh:535- Remote deviation lines are relayed only when the entire multi-item push succeeds. Ifconfig/backendreports a deviation and a later item fails, the failure branch retains only the first output line and hides the divergence. Relay captured deviation lines before branching on overall status.bin/fm-config-push.sh:152- The required pushing-primary relay is absent from mid-session remote config push. This path only indents rawdeviation:output instead of emitting the requiredSECONDMATE_SYNC:divergence line. Apply the same relay format as bootstrap.bin/fm-spawn.sh:533- Remote launch and recovery convergence discards all receiver output. An existing remote home withconfig/backend.deviationis honored duringfm-spawn, but the pushing primary receives no named divergence because stdout is redirected to/dev/null. Relay deviation and rejection records before continuing launch.🔧 Fix: Relay remote deviations consistently across convergence paths
1 error still open:
bin/fm-config-inherit-lib.sh:208- An otherwise valid record is honored without validating the held config artifact. For example, a symlinked or directory-valuedconfig/backendreaches this function, is reported asdeviated, and remains in place without error, while the remote receiver rejects the same unsafe destination. This also bypasses the local path's prior replacement/error behavior. Only honor an absent destination or an ordinary, non-symlinked, single-link file; otherwise reject the deviation and continue normal convergence.🔧 Fix: Reject unsafe local deviation values before convergence
✅ Re-checked - no issues remain.
✅ **Test** - passed
✅ No issues found.
bash tests/fm-config-deviation.test.shbash tests/fm-remote-secondmate-lifecycle-e2e.test.shManual local convergence usingpropagate_inheritable_config, followed by persistedconfig/backendinspectionManual remote receiver checks usingFM_HOME=... ./bin/fm-remote-inherit.sh put config/backend <bytes> <sha256> <generation> < payload, including differing and agreeing valuesManual primary relay throughfm_config_relay_remote_deviations iosManual deviation revocation by removingconfig/backend.deviation, reconverging, and inspecting the resulting backend and report status✅ **Document** - passed
✅ No issues found.
🔧 Fix: Captain: fix ShellCheck payload redirection warnings
1 warning still open:
✅ **Push** - passed
✅ No issues found.