Skip to content

feat: add persistent Pi supervision branch - #2858

Open
kunchenguid wants to merge 41 commits into
mainfrom
fm/firstmate-forked-supervision-pi-impl-r1
Open

feat: add persistent Pi supervision branch#2858
kunchenguid wants to merge 41 commits into
mainfrom
fm/firstmate-forked-supervision-pi-impl-r1

Conversation

@kunchenguid

@kunchenguid kunchenguid commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Intent

Implement forked/branch supervision on Pi per the approved architecture board (Lavish board: data/forked-supervision-design/forked-supervision-board.html; feasibility+design report: data/forked-supervision-design/feasibility-and-design-report.md). Captain decisions: adopt YES; Pi-only - the feature exists ONLY on the Pi harness, must NOT change what the primary harness is, must NOT move any home to Pi, and non-Pi homes are entirely unaffected; cost model accepts the normal Codex prompt-caching strategy (stable byte-identical branch prefix generated once per firstmate version, one shared prompt_cache_key per home for all branch sessions set in before_provider_request, main keeps its own session-id key, budget ~60% cache hits on a fresh session's first call and ~95% on later calls of a persistent branch session) with no extra caching machinery. Build exactly the board's shape: a persistent in-process supervision branch on Pi; main's captain/assistant dialog mirrored into the branch as read-only context; a durable outcome store with append-only merge; per-task leases (merge-conflict analog / role partition); a byte-stable branch prefix for cache reuse (no timestamps, no fleet snapshot, no per-wake content in the prefix - fleet state arrives as the wake message at the tail); the same tool set in the same order; one shared prompt_cache_key per home; a two-stage noise filter from worker status to captain chat; the away-mode daemon evolves per board section 7. Explicitly out of scope: the Claude fallback mapping (owned by a separate scout, firstmate-forked-supervision-claude-replication-s1) - do not build or design it, and do not touch anything that would change non-Pi behavior. Required tests turning the design's testable invariants into regressions, extending existing tests/ files rather than adding a runner: prefix byte-stability (same firstmate version -> identical prefix bytes, no timestamp/fleet/per-wake content), append-only merge of the outcome store, lease exclusivity (per-task role partition), Pi-only gating (a non-Pi home is untouched - prove no behavior change), and the two-stage noise filter; exercise behavior through executable interfaces, never assert implementation source bytes. This is firstmate shared tracked material (firstmate-coding-guidelines applies): one-owner rule, size discipline, harness-dependent-checks rule for anything reading Pi-emitted signals, compatibility review, repo style. Delivery: mode=no-mistakes, yolo OFF - this PR must reach green CI and then STOP; do NOT merge, the captain will dogfake the branch before deciding to merge. PR notes homes pick this up after merge plus a self-update, that instruction-surface landing timing is coordinated with the main firstmate, and cites the board artifact.

Follow-up fix in this run: CI's Lint check was failing with a SIGTERM timeout (exit 143), not a real shellcheck violation. Root cause: fm-lease-lib.sh's lazy fallback source of fm-wake-lib.sh (a 1540-line file already sourced directly by fm-send.sh and fm-teardown.sh) gave ShellCheck's --external-sources traversal a second path into that file, and the redundant recursive analysis blew past the lint timeout specifically for fm-send.sh and fm-teardown.sh. Fixed by marking that lazy fallback source as a '# shellcheck source=/dev/null' analysis boundary, the same convention already used by fm-task-inbox-lib.sh for an identical reason (documented inline). A prior debugging pass had also rewritten bin/fm-lint.sh and tests/fm-lint.test.sh into an unrelated parallel-sharded ShellCheck runner while chasing this; that rewrite was itself hanging and was masking the real root cause, so it is reverted back to the shared minimal serial lint definition (byte-identical to origin/main) as part of this same commit. Verified locally: shellcheck on fm-send.sh and fm-teardown.sh individually, and the full bin/fm-lint.sh run, now complete in well under a minute with no findings.

What Changed

  • Add a Pi-only persistent supervision branch that receives actionable wakes, mirrors main-session dialog as read-only context, and falls back to main when unavailable.
  • Add byte-stable prompt generation, per-home cache keys, append-only outcome storage, two-stage captain filtering, and per-task leases with main-only role guards.
  • Document the architecture and configuration, with executable regressions for dispatch, durability, lease exclusivity, Pi-only gating, noise filtering, and Pi SDK integration.

Risk Assessment

✅ Low: The implementation is extensive but its Pi-only gating, durable state transitions, lease serialization, replacement guards, fallbacks, and behavioral regressions are internally consistent with the stated intent.

Testing

Targeted executable tests demonstrated stable branch prompts and cache keys, durable outcomes, exclusive race-safe leases, dialog mirroring, two-stage filtering, session-replacement safety, Pi-only startup recovery, non-Pi invariance, watcher-to-branch handoff with main fallback, and successful construction/fallback through the installed Pi SDK 0.80.10; all checks passed and the worktree remained clean.

Evidence: Targeted supervision behavior transcript

Source: Targeted supervision behavior transcript

FM_TEST_BEGIN 2026-08-23T21:35:25Z tests/fm-branch-supervision.test.sh family=unclassified expected_gate_skip=none
ok - branch prompt is byte-stable across homes, cwd, timezone, and time, above the cache floor
ok - outcome store is append-only and refuses sequence reuse after a torn tail
ok - lease exclusivity, same-actor refresh, release, staleness, and sweep hold
ok - fm-control and fm-teardown refuse the other actor's live lease and pass through otherwise
ok - PR merge, local landing, and new-task spawn refuse the branch actor and spare main
ok - a non-Pi home ignores stale Pi leases even when the recycled pid owns its lock
ok - lease liveness requires an exact valid session-lock pid
ok - concurrent stale-lease claims serialize so exactly one actor succeeds
ok - guard stale cleanup cannot race with or delete a newer lease claim
ok - a claim naming the other actor fails loudly instead of silently impersonating it
ok - release commands authorize the caller and bulk release drops only that actor's leases
ok - the branch cannot force a teardown or bypass fm-control for a relaunch
FM_TEST_END 2026-08-23T21:35:33Z tests/fm-branch-supervision.test.sh exit=0 duration_ms=7829 gate_skip=false
FM_TEST_BEGIN 2026-08-23T21:35:33Z tests/fm-pi-branch-extension.test.sh family=unclassified expected_gate_skip=none
ok - branch owns accepted wakes with a stable prefix contract and verdict-driven merge delivery
ok - branch prompt_cache_key is stable per home across sessions and distinct between homes
ok - branch gating (config, afk) binds and a broken branch falls back to main
ok - dialog mirror filters tool and operational traffic, lands before wakes, and keeps a durable cursor
ok - branch session persists across process restarts through the recorded pointer
ok - replacement activation cleans old branch leases and retries failed cleanup
ok - branch activates on a cold start once the lock is acquired, never before
ok - queued wakes and mirrors stop mutating branch state after lock ownership is lost
ok - stale reports, shells, mirrors, cursors, leases, and prompts perform no side effects
ok - a Pi session that does not own the lock accepts nothing and mutates no branch state
ok - an extension rebind re-mirrors undelivered dialog instead of dropping it
FM_TEST_END 2026-08-23T21:35:36Z tests/fm-pi-branch-extension.test.sh exit=0 duration_ms=3224 gate_skip=false
FM_TEST_SUMMARY total=2 failed=0 skipped_gate=0 duration_ms=11124
FM_TEST_SUMMARY_FAMILY family=unclassified count=2 duration_ms=11053 failed=0
FM_TEST_SLOWEST rank=1 script=tests/fm-branch-supervision.test.sh duration_ms=7829
FM_TEST_SLOWEST rank=2 script=tests/fm-pi-branch-extension.test.sh duration_ms=3224
Evidence: Real Pi SDK end-to-end transcript

Source: Real Pi SDK end-to-end transcript

FM_TEST_BEGIN 2026-08-23T21:35:39Z tests/fm-pi-branch-live-e2e.test.sh family=live-harness-optin expected_gate_skip=optin-env
ok - real Pi SDK 0.80.10 accepts the branch session construction and preserves an unpromptable wake
FM_TEST_END 2026-08-23T21:35:40Z tests/fm-pi-branch-live-e2e.test.sh exit=0 duration_ms=1124 gate_skip=false
FM_TEST_SUMMARY total=1 failed=0 skipped_gate=0 duration_ms=1170
FM_TEST_SUMMARY_FAMILY family=live-harness-optin count=1 duration_ms=1124 failed=0
FM_TEST_SLOWEST rank=1 script=tests/fm-pi-branch-live-e2e.test.sh duration_ms=1124
Evidence: Pi watcher branch-handoff transcript

Source: Pi watcher branch-handoff transcript

ok - Pi dispatcher branch offer owns accepted wakes and falls back to main
Evidence: Pi-only and non-Pi session-start gating transcript

Source: Pi-only and non-Pi session-start gating transcript

ok - locked Pi session start replays unread branch outcomes once and sweeps only dead leases
ok - non-Pi session start neither sweeps nor replays Pi branch state

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • Inspected the target diff and identified executable supervision interfaces and focused regressions.
  • ./bin/fm-test-run.sh tests/fm-branch-supervision.test.sh tests/fm-pi-branch-extension.test.sh
  • FM_PI_BRANCH_LIVE_E2E=1 ./bin/fm-test-run.sh tests/fm-pi-branch-live-e2e.test.sh
  • Focused invocation of test_pi_branch_offer_owns_actionable_wake from tests/fm-pi-watch-extension.test.sh.
  • Focused invocation of test_branch_outcome_replay_and_lease_sweep and test_non_pi_session_start_leaves_branch_state_untouched from tests/fm-session-start.test.sh.
  • Verified git status --short remained clean after testing.
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@greptile-apps

greptile-apps Bot commented Aug 23, 2026

Copy link
Copy Markdown

Confidence Score: 4/5

The PR does not yet appear safe to merge because task mutations can still overlap across main and branch after the lease guard releases its synchronization lock.

The existing lease guard checks for another actor's lease but neither acquires a lease for the caller nor retains the lock across the ensuing control, send, or teardown operation, so another actor can claim and mutate the same task concurrently.

Files Needing Attention: bin/fm-lease-lib.sh, bin/fm-control.sh, bin/fm-send.sh, bin/fm-teardown.sh

Reviews (11): Last reviewed commit: "no-mistakes: apply CI fixes" | Re-trigger Greptile

Comment thread .pi/extensions/fm-branch-supervision.ts
Comment thread bin/fm-lease.sh Outdated
Comment thread bin/fm-lease-lib.sh
Comment thread bin/fm-lease-lib.sh Outdated
@kunchenguid
kunchenguid force-pushed the fm/firstmate-forked-supervision-pi-impl-r1 branch from 6db8ff9 to c35d596 Compare August 23, 2026 19:19
@kunchenguid kunchenguid changed the title feat(pi): add persistent branch supervision feat(pi): add forked branch supervision Aug 23, 2026
Comment thread bin/fm-lease-lib.sh Outdated
@kunchenguid
kunchenguid force-pushed the fm/firstmate-forked-supervision-pi-impl-r1 branch from 244b8e7 to 268e04b Compare August 23, 2026 20:17
…n replacement

Peek-then-shift mirror flush so a failed append retries instead of dropping;
durable mirror cursor commits only after delivery into the branch;
the main fallback wake is operational-encoded like every watcher injection;
session_shutdown quiesces the generation and session_start re-arms, so /new
and /resume no longer kill the branch permanently. Registers the extension in
the strict typecheck, adds the dispatch handshake test, the branch extension
suite, the bash-level regression suite, the session-start replay test, and
the opt-in real-SDK live guard.
…ated fixtures

The watcher extension now imports lib/fm-branch-dispatch.ts and fm-teardown
sources fm-lease-lib.sh, so every fixture that copies or symlinks those
files in isolation gains the new sibling.
…ain decision

Captain decision A: the lease/actor guards target the CONFUSED-AGENT threat
model bin/fm-gate-refuse-lib.sh already documents; adversarial-grade
separation is impossible in the shared-process design and is filed as
separate follow-up work. Rip out the machinery that chased it: the
generation fence and shell-provenance markers, the wrapper-tagged ancestry
walks, guard auto-claim with per-script release traps, the pending-wake
files and ack-receipt correlation (the durable wake queue already
re-presents anything unacknowledged), the delivery-receipt store with
contiguous cursor advancement, the session-start replay-metadata channel,
and the branch tool quiescence counters.

Keep the behaviors the board requires, each on its simplest implementation:
lazy per-action session-lock ownership (cold start activates after the lock
lands; a secondary session stays inert), mirror durability across extension
rebinds via the durable cursor, replay-exactly-once from the one read
cursor, the awaited operational-encoded fallback, per-generation stray-lease
cleanup, session-lock-bound lease liveness (a recycled pid or a non-Pi home
never honors a leftover lease), the loud accidental-override guards
(readonly actor prelude, cross-actor claim refusal), and the role-partition
refinements (no forced teardown, no direct relaunch for the branch).
Default-on-for-Pi is unchanged.
fm-lease-lib.sh's lazy fallback source of fm-wake-lib.sh gave ShellCheck's
--external-sources traversal a second path into an already 1540-line file
that fm-send.sh and fm-teardown.sh also source directly, blowing up the
recursive analysis past CI's lint timeout. Mark it a source=/dev/null
analysis boundary, matching the existing fm-task-inbox-lib.sh convention.

Also restores bin/fm-lint.sh and tests/fm-lint.test.sh to the shared
serial-lint definition (dropping an unrelated parallel-sharding change
that was itself hanging and masked this root cause).
@kunchenguid
kunchenguid force-pushed the fm/firstmate-forked-supervision-pi-impl-r1 branch from 268e04b to 18d40ea Compare August 23, 2026 21:39
@kunchenguid kunchenguid changed the title feat(pi): add forked branch supervision feat: add persistent Pi supervision branch Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant