Skip to content

fix(bin): honor evidence-backed backend deviations - #2875

Open
karotkriss wants to merge 6 commits into
kunchenguid:mainfrom
karotkriss:fm/fm-2832-backend-deviation
Open

fix(bin): honor evidence-backed backend deviations#2875
karotkriss wants to merge 6 commits into
kunchenguid:mainfrom
karotkriss:fm/fm-2832-backend-deviation

Conversation

@karotkriss

Copy link
Copy Markdown
Contributor

Intent

Fix GitHub issue #2832: inherited config/backend convergence silently reverts a secondmate home's evidence-based local backend pin. A secondmate home hit repeated silent worker deaths that correlated with the auto-detected experimental backend and set its local config/backend to the verified backend as a deliberate, evidence-backed deviation. The primary's inherited-config convergence reverted that file within the hour, working exactly as documented, and the home only noticed because it checked which backend a NEW worker actually landed on rather than trusting the file it had written. There was no sanctioned way for a home to hold a deliberate local deviation, and a silent revert of a safety-motivated setting is the worst failure shape: the home believes it mitigated a risk it has not.

Required fix shape, decided before implementation: the primary STAYS authoritative and no default behavior changes. Add an explicit deviation record the secondmate may write next to the config item, with a REQUIRED evidence line, which convergence honors AND reports back to the primary as a named divergence line at sync time. Visible, revocable, and never silent in either direction. Colocated tests must cover honoring the record, surfacing the divergence, and the primary revoking it.

Decisions and tradeoffs made while implementing:

  • The record is config/.deviation beside the item. The deviating VALUE deliberately stays in config/ itself, where every consumer already reads it, so no consumer changes and there is no second source of truth for the value; the record only states that this home holds its own value there and why.
  • The deviable set is a narrow declared allowlist (FM_DEVIABLE_CONFIG), containing only 'backend' today, rather than making every inherited item deviable. Granting the escape hatch to items nobody asked about would assume consent. This mirrors the existing FM_SESSION_SCOPED_INHERITABLE_CONFIG declaration idiom in the same file.
  • The divergence line is only printed where convergence would actually have changed the destination, so a record whose value already agrees with the primary reports nothing and does not add routine noise to the session-start digest.
  • A record that cannot be honored - a non-deviable item, no evidence line, a symlinked, non-ordinary, hardlinked, or oversized file - is reported with its concrete reason and the item converges to the primary value, so refusing is not silent either.
  • Reported values and evidence are bounded and control-character stripped so one oversized or hostile local file cannot flood the primary's session-start digest.
  • Remote secondmate homes are covered in their own receiver (bin/fm-remote-inherit.sh), which honors the record and reports 'deviation:'/'deviation-rejected:', with the pushing primary relaying that as the same SECONDMATE_SYNC divergence line. Shipping the hatch for local homes only would leave it silently absent on remote homes, which is the exact failure shape the issue is about.
  • The one-owner documentation rule was followed: the secondmate-provisioning skill owns the full deviation contract, docs/configuration.md and AGENTS.md carry only pointers and a status-name update.

The PR must use 'Fixes #2832' and must state the VISION.md angle explicitly: authority stays explicit and is never inferred (this is a record the home writes, never a value the primary can push, and inheritance stays primary-authoritative for every home that holds no record); the interface stays honest under load, so silence never hides a failure, a decision, or a risk (both honoring and reverting are now named to the primary); and new capability ships as an opt-in option to enable, never as a default-behavior change.

What Changed

  • Add opt-in, evidence-backed config/backend.deviation records so local and remote secondmate homes can retain a verified backend value until the record is removed or rejected.
  • Surface honored and rejected deviations as bounded SECONDMATE_SYNC: lines across bootstrap, spawn, and config-push flows, with coverage for local and remote convergence, unsafe records, quiet agreement, and revocation.
  • Preserve the VISION.md contracts: authority remains explicit because the home writes the record, inheritance remains primary-authoritative without one, divergence never silently hides a decision or risk, and the new capability changes no default behavior. Fixes Inherited config/backend silently reverts a secondmate's evidence-based local override #2832.

Risk Assessment

✅ Low: The deviation contract is consistently enforced across local, remote, launch, bootstrap, and mid-session convergence paths, with prior silent and unsafe cases resolved.

Testing

No prior baseline command output was supplied. The focused deviation regression suite and complete remote secondmate lifecycle E2E passed, while manual local and remote sync checks captured reviewer-visible evidence proving the hold, named relay, quiet agreement, persisted state, and primary revocation.

Evidence: Backend deviation sync transcript

Source: Backend deviation sync transcript

Shows local and remote deviations retaining tmux, the primary-visible SECONDMATE_SYNC line with both values and evidence, quiet agreement, and revocation restoring herdr.

LOCAL SYNC: evidence-backed home value differs from primary
SECONDMATE_SYNC: secondmate home /home/cmckay/.no-mistakes/evidence/01M0QGSVN0P3CRWKQTRNRPY79Q/deviation-sync-state/local-second: config/backend held locally at "tmux" against primary "herdr": four workers died on herdr; tmux verified stable
persisted config/backend: tmux

REMOTE SYNC: receiver reports divergence and primary relay names it
receiver: deviation: config/backend held locally at "tmux" against primary "herdr": four workers died on herdr; tmux verified stable
SECONDMATE_SYNC: secondmate ios: deviation: config/backend held locally at "tmux" against primary "herdr": four workers died on herdr; tmux verified stable
persisted config/backend: tmux

REMOTE SYNC: primary now agrees, so ordinary unchanged output has no divergence
receiver: unchanged: config/backend

LOCAL SYNC: primary revokes by removing the home record
persisted config/backend: herdr
report status: pushed
Evidence: Persisted remote deviated backend

Source: Persisted remote deviated backend

tmux

tmux
Evidence: Persisted local backend after revocation

Source: Persisted local backend after revocation

herdr

herdr

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 4 issues found → auto-fixed (3) ✅
  • 🚨 bin/fm-remote-inherit.sh:174 - The accepted criterion says, "The divergence line is only printed where convergence would actually have changed the destination." For a remote put where both values are tmux, this calls deviation_holds before the equality check, reports a false divergence, and exits. The absent path similarly reports when both sides are absent. Compare destination state first, then report only a real divergence with the bounded local and primary values.
  • 🚨 bin/fm-remote-inherit.sh:174 - An honored remote deviation exits before commit_generation, leaving the replay barrier stale. After an unknown-completion reorder, a newer generation can be honored without being recorded; once the record is revoked, a delayed older payload can pass the stale generation check and overwrite the current primary value. Commit every validated generation before any deviation exit.
  • 🚨 bin/fm-bootstrap.sh:535 - The accepted criterion requires honoring and rejection to be named to the primary on every sync. This relay loop runs only when the entire multi-item remote push succeeds. If config/backend emits a deviation and a later item fails, the failure branch reduces the captured output to its first line and silently loses the divergence. Relay captured deviation lines before branching on the overall command status.
  • 🚨 bin/fm-config-push.sh:152 - The accepted criterion says the pushing primary relays remote deviation: and deviation-rejected: output as the same SECONDMATE_SYNC divergence line. The mid-session remote config-push path only indents the receiver's raw output, so this supported convergence path never produces the required primary-facing line. Apply the same relay formatting used by bootstrap.

🔧 Fix: Commit honored remote generations before divergence exits
5 errors still open:

  • 🚨 bin/fm-remote-inherit.sh:175 - The criterion says divergence is reported only when convergence would change the destination. A remote put with matching primary and local values calls deviation_holds before comparing them, so it falsely reports a divergence. absent likewise reports when both sides are absent. Compare destination state before evaluating the record.
  • 🚨 bin/fm-remote-inherit.sh:140 - Remote divergence output omits the bounded local and primary values required by the documented identical behavior. For local tmux against primary herdr, this emits only held locally: <evidence>. Include both values, using absence where appropriate.
  • 🚨 bin/fm-bootstrap.sh:535 - Remote deviation lines are relayed only when the entire multi-item push succeeds. If config/backend reports a deviation and a later item fails, the failure branch retains only the first output line and hides the divergence. Relay captured deviation lines before branching on overall status.
  • 🚨 bin/fm-config-push.sh:152 - The required pushing-primary relay is absent from mid-session remote config push. This path only indents raw deviation: output instead of emitting the required SECONDMATE_SYNC: divergence line. Apply the same relay format as bootstrap.
  • 🚨 bin/fm-spawn.sh:533 - Remote launch and recovery convergence discards all receiver output. An existing remote home with config/backend.deviation is honored during fm-spawn, but the pushing primary receives no named divergence because stdout is redirected to /dev/null. Relay deviation and rejection records before continuing launch.

🔧 Fix: Relay remote deviations consistently across convergence paths
1 error still open:

  • 🚨 bin/fm-config-inherit-lib.sh:208 - An otherwise valid record is honored without validating the held config artifact. For example, a symlinked or directory-valued config/backend reaches this function, is reported as deviated, and remains in place without error, while the remote receiver rejects the same unsafe destination. This also bypasses the local path's prior replacement/error behavior. Only honor an absent destination or an ordinary, non-symlinked, single-link file; otherwise reject the deviation and continue normal convergence.

🔧 Fix: Reject unsafe local deviation values before convergence
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • bash tests/fm-config-deviation.test.sh
  • bash tests/fm-remote-secondmate-lifecycle-e2e.test.sh
  • Manual local convergence using propagate_inheritable_config, followed by persisted config/backend inspection
  • Manual remote receiver checks using FM_HOME=... ./bin/fm-remote-inherit.sh put config/backend <bytes> <sha256> <generation> < payload, including differing and agreeing values
  • Manual primary relay through fm_config_relay_remote_deviations ios
  • Manual deviation revocation by removing config/backend.deviation, reconverging, and inspecting the resulting backend and report status
✅ **Document** - passed

✅ No issues found.

⚠️ **Lint** - 1 warning
  • ⚠️ linter found issues (exit code 1)

🔧 Fix: Captain: fix ShellCheck payload redirection warnings
1 warning still open:

  • ⚠️ linter found issues (exit code 1)
✅ **Push** - passed

✅ No issues found.

Inherited config convergence is primary-authoritative, so a secondmate
home that pinned config/backend after repeated silent worker deaths had
that pin reverted at the next sync, with nothing said in either
direction. The home believed it had mitigated a risk it had not.

Keep the primary authoritative and add the one sanctioned, opt-in escape:
a home may write config/<item>.deviation beside a declared deviable item
(backend today). The deviating value stays in the config item itself,
where every consumer already reads it, so the record only states that
this home holds its own value there and why. The record must carry an
evidence line.

Where the values actually differ, convergence leaves the home's value
alone and names the divergence back to the primary as a SECONDMATE_SYNC
line carrying both values and the evidence, on every sync. A record that
cannot be honored - a non-deviable item, no evidence, an unsafe or
oversized file - is reported with its reason and the item converges to
the primary value. The primary revokes by removing the record. Remote
homes behave identically through their own receiver, whose deviation
result the pushing primary relays as the same divergence line.
@greptile-apps

greptile-apps Bot commented Aug 23, 2026

Copy link
Copy Markdown

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Reviews (2): Last reviewed commit: "no-mistakes: apply CI fixes" | Re-trigger Greptile

Comment thread bin/fm-remote-inherit.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inherited config/backend silently reverts a secondmate's evidence-based local override

1 participant