Skip to content

feat(bin): add decision-tier classifier and default-with-veto CLI - #2862

Open
pramendra wants to merge 9 commits into
kunchenguid:mainfrom
pramendra:fm/fleet-ws1-decision-tiers
Open

feat(bin): add decision-tier classifier and default-with-veto CLI#2862
pramendra wants to merge 9 commits into
kunchenguid:mainfrom
pramendra:fm/fleet-ws1-decision-tiers

Conversation

@pramendra

Copy link
Copy Markdown

Intent

Workstream 1 of the fleet engineering plan (data/fleet-engineering-plan.html, ranked first by measured leverage, 'closes C, the actual 10x lever'): tier every decision into three classes - auto (consistent with precedent; firstmate acts and logs it), default-with-veto (firstmate states a recommendation and a default, acts unless the captain objects within a stated window, and the action is reversible), and hard stop (merges, destructive or irreversible actions, scope expansion, client-facing semantics - these always escalate, unconditionally). Build the MECHANISM (the classifier, the default-and-timeout machinery, the logging) as a library/policy module usable by firstmate's own escalation logic, with the three tiers as an explicit, documented, extensible classification. Do NOT wire it to actually change firstmate's live behavior yet (that activation is the captain's call) - make the mechanism real, tested, and ready to be turned on. Done when: a session's escalation count is a reported, measurable quantity, and every tier-2 decision carries a stated default that expires into action rather than blocking indefinitely. Acceptance criteria: the described mechanism is built and demonstrably works (show it, don't just assert it); prove any check added can fail by mutating what it guards and showing it go red; claim only what was measured, stating plainly what was NOT built and why scope was deliberately narrowed. This is firstmate's own shared tracked material (architecture work) - firstmate-coding-guidelines applies; keep scope to exactly what is asked, no speculative generality, no control planes beyond what is described.

What Changed

  • Add bin/fm-decision-tier-lib.sh, a sourced-only library implementing the single-owner category→tier classification table (auto / default-veto / hard-stop, failing closed to hard-stop on unrecognized categories), a 9-field TAB-separated decision-record log shape, default-with-veto status derivation (pending/vetoed/expired/unknown) computed purely from the log and a supplied epoch, and an aggregate report emitting counters including escalation_count. Mutators (log_auto, log_hard_stop, open_default, veto) refuse when a category's tier disagrees with the requested operation or when a veto is attempted outside the pending window. The library is not called from firstmate's live escalation path.
  • Add bin/fm-decision-tier.sh, a CLI over the library exposing classify, categories, auto, hard-stop, open, veto, status, and report subcommands, with a --now <epoch> override on time-sensitive subcommands for deterministic use.
  • Add tests/fm-decision-tier-lib.test.sh and tests/fm-decision-tier.test.sh covering the classification table, record/log mutators, status derivation, and CLI subcommands; list both new scripts in docs/scripts.md's toolbelt table; add the generated .serena/project.yml and .serena/.gitignore.

Risk Assessment

✅ Low: A self-contained, unwired library + CLI with no call sites elsewhere in the codebase; ran both new test suites (28 assertions) and they pass, and a targeted mutation of the fail-closed default arm correctly turned the corresponding assertion red, confirming the guard is real rather than vacuous.

Testing

Ran both new focused suites (tests/fm-decision-tier-lib.test.sh, tests/fm-decision-tier.test.sh) directly — all assertions pass; verified the fail-closed classification guard is a real, mutation-provable check by flipping its default case and confirming both suites go red on that exact assertion, then restored the file cleanly; and produced an end-to-end CLI transcript exercising the full auto/default-veto/hard-stop lifecycle (including refusal of mis-tiered logging, a veto inside the window, a too-late veto refusal, and the aggregate escalation_count report) against a real decision log, which is the product-level evidence for this CLI mechanism.

Evidence: End-to-end CLI transcript: classify/auto/hard-stop/open/veto/status/report lifecycle over a real decision log

Source: End-to-end CLI transcript: classify/auto/hard-stop/open/veto/status/report lifecycle over a real decision log

$ bin/fm-decision-tier.sh classify merge
hard-stop

$ bin/fm-decision-tier.sh classify precedent-match
auto

$ bin/fm-decision-tier.sh classify some-new-category-nobody-registered   # fails closed
hard-stop

# --- auto tier: firstmate acts and logs it, no captain involvement ---
$ bin/fm-decision-tier.sh auto --now 1000 "$LOG" dec-auto-1 precedent-match "matched sibling ruling from dec-0"
(exit 0)

# --- a caller cannot mis-tier a decision: logging a merge as auto is refused ---
$ bin/fm-decision-tier.sh auto --now 1000 "$LOG" dec-bad merge "should be refused"
fm-decision-tier-lib: category merge classifies hard-stop, not auto; refusing
(exit 1)

# --- hard-stop tier: merges always escalate, unconditionally ---
$ bin/fm-decision-tier.sh hard-stop --now 1000 "$LOG" dec-hard-1 merge "attempted merge, escalated per hard rule"
(exit 0)

# --- default-with-veto tier: a stated recommendation + default + window ---
$ bin/fm-decision-tier.sh open --now 1000 "$LOG" dec-veto-1 two-option-tradeoff 300 "prefer option A" "apply option A"
(exit 0)

$ bin/fm-decision-tier.sh status --now 1100 "$LOG" dec-veto-1   # inside the 300s window
pending

$ bin/fm-decision-tier.sh status --now 1301 "$LOG" dec-veto-1   # window elapsed, no captain veto -> expires into action
expired

# --- a second default-veto decision, this time the captain vetoes in time ---
$ bin/fm-decision-tier.sh open --now 1000 "$LOG" dec-veto-2 process-default 300 "prefer routine default" "apply routine default"
(exit 0)
$ bin/fm-decision-tier.sh veto --now 1100 "$LOG" dec-veto-2 "captain overrides the default"
(exit 0)
$ bin/fm-decision-tier.sh status --now 5000 "$LOG" dec-veto-2   # stays vetoed long after the window would have elapsed
vetoed

$ bin/fm-decision-tier.sh veto --now 5000 "$LOG" dec-veto-1 "too late objection"   # window already expired, veto refused
fm-decision-tier-lib: cannot veto id dec-veto-1: status is expired, not pending
(exit 1)

# --- a session's escalation count is a reported, measurable quantity ---
$ bin/fm-decision-tier.sh report --now 5000 "$LOG"
total=4
auto=1
default_pending=0
default_expired=1
default_vetoed=1
hard_stop=1
escalation_count=1

# --- raw decision log on disk (the persisted state this all derives from) ---
$ cat "$LOG"
1000	dec-auto-1	precedent-match	auto	acted				matched sibling ruling from dec-0
1000	dec-hard-1	merge	hard-stop	escalated				attempted merge, escalated per hard rule
1000	dec-veto-1	two-option-tradeoff	default-veto	opened	300	prefer option A	apply option A	
1000	dec-veto-2	process-default	default-veto	opened	300	prefer routine default	apply routine default	
1100	dec-veto-2	process-default	default-veto	vetoed				captain overrides the default

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • bash tests/fm-decision-tier-lib.test.sh
  • bash tests/fm-decision-tier.test.sh
  • mutation test: flipped fm_decision_tier_classify's fail-closed default arm from hard-stop to auto, reran both suites (both went red on the fail-closed assertions), then restored the file and reran both suites (green, git diff clean)
  • manual CLI transcript: bin/fm-decision-tier.sh classify/auto/hard-stop/open/status/veto/report against a fresh temp log, covering the full tier lifecycle and mis-tier refusals
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Your Name added 2 commits August 23, 2026 19:53
Fleet engineering plan workstream 1 ("widen the judgment channel"): a
category->tier classifier (auto / default-veto / hard-stop, fail-closed
on anything unrecognized), a timed default-with-veto record that
expires into action, and a report that turns a session's escalation
count into a measured number instead of a felt impression.

Standalone building block only - nothing in firstmate's live
escalation path calls into it yet; wiring it in is the captain's call.
@greptile-apps

greptile-apps Bot commented Aug 23, 2026

Copy link
Copy Markdown

Confidence Score: 3/5

The PR is not yet safe to merge because lock recovery can permanently block all decision writers and can delete an unrelated empty directory.

The breaker mutex has no abandonment recovery, so termination during stale-lock handling leaves future writers retrying forever; recovery also still removes an empty directory without proving it belongs to this locking implementation.

Files Needing Attention: bin/fm-decision-tier-lib.sh

Reviews (8): Last reviewed commit: "no-mistakes: apply CI fixes" | Re-trigger Greptile

Comment thread bin/fm-decision-tier-lib.sh
Comment thread bin/fm-decision-tier-lib.sh
Comment thread bin/fm-decision-tier-lib.sh Outdated
Comment thread bin/fm-decision-tier-lib.sh
@greptile-apps

greptile-apps Bot commented Aug 23, 2026

Copy link
Copy Markdown

Want your agent to iterate on Greptile's feedback? Try greploops.

Comment thread bin/fm-decision-tier-lib.sh
Comment thread bin/fm-decision-tier-lib.sh Outdated
Comment thread bin/fm-decision-tier-lib.sh Outdated
Comment thread bin/fm-decision-tier-lib.sh Outdated
Comment thread bin/fm-decision-tier-lib.sh Outdated
Comment thread bin/fm-decision-tier-lib.sh Outdated
Comment thread bin/fm-decision-tier-lib.sh Outdated
Comment thread bin/fm-decision-tier-lib.sh Outdated
fm_decision_tier_lock_break() { # <lockdir>
local lockdir=$1
local mutex="$lockdir.break"
mkdir "$mutex" 2>/dev/null || return 0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Breaker mutex permanently blocks writers

When a process terminates after creating <log>.lock.break but before removing it, every later breaker returns because the mutex directory already exists. The stale primary lock therefore remains, causing all subsequent decision writers to retry indefinitely without recording their decisions.

Comment on lines +228 to +229
rmdir "$lockdir" 2>/dev/null
elif [ -e "$lockdir" ]; then

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Recovery deletes unrelated empty directories

When an unrelated empty directory exists at <log>.lock, a decision writer treats it as a legacy lock and removes it without establishing ownership. Starting an auto, hard-stop, or default-veto write therefore deletes the caller's directory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant