Repository navigation
Factory goal gate: owner-approved promotion over ledger-resolved verdicts, digest-checked artifacts, an isolated and sealed candidate run (closes the #154 verification's S1-S9) - #164
Conversation
…icts, digest-checked artifacts, an isolated and sealed candidate run, recomputed policy on verify, reviewer independence by provider family, scrubbed journals (the #154 verification's S1-S9) The factory may propose a spec; it may promote none. `spec promote`, `canonize` and `retire` now take a human actor (`kind:id`, resolved to a principal) and an approval that verifies: `factory approve` signs, with the owner's seed through the certificate module's Ed25519 and DSSE, a payload naming the spec (id, version, digest as on disk), the operation and the verdict ids; `spec promote` verifies it under `--owner-public`, requires the approver to be the actor, resolves every `--verdict` on the ledger to a `review` that accepted the very variant on disk (the `validate` event's digest must match), refuses an approver who proposed or reviewed it, stores the approval under `approvals/` and cites its `approval_recorded` hash from the `spec_changed`. `transition` accepts only the `Approved` record `verify_approval` returns. The variant loop no longer hints `--actor model:<reviewer>`. Ledger: every artifact load checks the file against the digest the ledger recorded (`load_run`, `load_space`; `set_baseline` goes through `load_run`), `verify` covers Space graphs, journals, their blobs, attestations and approvals, `problems()` says which, and `factory check` verifies the ledger before the candidate runs and again after it exits. Appends go through the kernel journal's writer (`syberlabs.journal.Journal`: cross-process lock, fsync, torn-tail repair); the SQLite projection rebuilds when the log it was built from moved (a `meta` row) and keeps the `features` table. Harness: the candidate runs under `python -I` in a host-owned scratch directory outside the worktree and the ledger and is never told the ledger's path; the parent moves the journal in afterwards. Every record carries a seal (HMAC chained from the previous record under a per-run key the parent generated); `verify_journal` refuses a planted line, an edited line, a torn tail or an empty file, and such a journal is not delivered. The report comes back over a private pipe bound by the run's nonce (the hidden checks' pattern); a line on stdout is output, and the status in the report must agree with the `end` record the parent reads itself. Timeouts stop the process tree (`checks._stop`, now a declared `factory -> git` edge). Subprocess argv and cwd, socket addresses and the tests tail are scrubbed of credential shapes (`_patterns.SECRET`, bearer tokens, PEM keys, credential `KEY=value`, the value after a credential flag) before they are journaled or attested. Attestation: `verify` recomputes the policy from the evidence and the gate and refuses a statement whose written policy says more; `record_attestation` logs `signed: true` only when the signature verifies under the key it is given, else `envelope: true, signed: false`. `independence(proposer, reviewer)` (`factory/independence.py`) judges by the provider family the command declared (`model` or `signal.harness.model`) and refuses an unknown family; `judge` uses it; `critic` is unchanged. The viewer builds its DOM with textContent and carries effects by digest, not payloads. `bootstrap` carries the contract's own tests timeout, `full_suite` means the suite ran to completion, and a hanging suite is stopped with its tree. Tests: 45 in tests/test_factory.py (35 before). New: the gate (model actor, human without approval, dict for an approval, wrong operation, wrong verdicts, wrong key, approver not the actor, approver who proposed), verdict resolution, principals, independence by family, edited golden (verify, load_run, set_baseline, projection, the check refuses before the candidate runs), projection staleness, three forging candidates refused and an honest one passing, the seal itself, scrubbing, a candidate forging its report through the Kit, a hanging suite stopped at the contract's timeout, the self-graded statement refused by verify, the end-to-end promote through approve. Fixture proposers declare their model; `fxapp/variants.py` holds the forgers. docs/FACTORY.md says 45 tests (it said 18), documents the gate and the trust boundary; spec/GAPS.md 65 records what remains (the approval is a signature, not yet a kernel `approved` event; the recorder shares the candidate's memory; the behavior-change label is per contract; no cross-process lock before this). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3c3a3ee49f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| """A proposer or reviewer as the ledger records it: the caller's name, and the model and provider family | ||
| the command declared in its answer (``model``, or ``signal.harness.model``). The family is what | ||
| independence is judged on; a command that declares no model has none and cannot be judged independent.""" | ||
| recorded = {"kind": kind, "id": name, "model": model_of(answer) if isinstance(answer, dict) else None} |
There was a problem hiding this comment.
Derive reviewer family from trusted provider metadata
When proposer and reviewer commands return their model metadata, party() accepts the response’s model or signal.harness.model as ground truth. The same underlying model can therefore claim different provider families in the two responses, causing judge() to pass a review that is not independent and allowing that verdict to be used for promotion. Bind the family to trusted host-side adapter metadata instead of response JSON.
AGENTS.md reference: AGENTS.md:L198-L199
Useful? React with 👍 / 👎.
Independent verification of #164 at head
|
| Repro | Result |
|---|---|
spec promote --actor model:reviewer |
exit 2 spec_actor |
--actor human:mateo, no approval |
exit 2 spec_approval |
--actor operator |
exit 2 actor_invalid |
human, valid approval, --verdict review-1 (a string) |
exit 2 spec_verdict: not a review on the ledger |
| human, valid approval, the loop's verdict, file edited after review | exit 2 spec_verdict (digest is not the reviewed variant) |
approval signed under another key, --owner-public the owner's |
exit 2 approval_unverified |
approval for human:tester, --actor human:claude |
exit 2 approval_actor |
| approver id equals the reviewer's id | exit 2 approval_not_independent |
| promote approval replayed for canonize; canonize approval replayed for retire | exit 2 approval_mismatch (operation) |
| retire approval, spec edited after signing | exit 2 approval_mismatch (spec_digest) |
| honest promote, then canonize with a fresh approval | promoted, canonical; spec_changed.approval.event_hash == the approval_recorded hash; verify() True |
edited golden, then factory check (CLI and through the Kit) |
exit 2 ledger_corrupt before the candidate runs; nothing attested |
forge_report, forge_journal, forge_end (local and through the Kit) |
error: no end record / journal_unsealed: line 5 carries no seal; journal not delivered; honest run ok |
credential in a subprocess argv (--token X, GITHUB_TOKEN=, -H 'Authorization: Bearer ...') |
<redacted> in the journal; viewer carries neither the values nor "payload" nor innerHTML; _run_tests argv and tail scrubbed, tail capped |
signed statement with diff_empty flipped, and one with policy forged to promote: true |
verify exit 1 policy mismatch; record_attestation under the wrong key logs signed: false, envelope: true |
| torn ledger tail | truncated on the next locked read, append proceeds, verify() True |
hanging test suite with a grandchild, --tests-timeout 2 |
stopped in well under 40 s, timed_out: true, full_suite: false, exit_code: null, refused |
Findings, most severe first
M1 (medium, goal gate): the owner's public key is whatever the caller passes; nothing pins it. --owner-public is a file path on spec promote's argv (syberlabs/factory/cli.py:129, read at cli.py:867), and spec.gate verifies under exactly that key (spec.py:140-151). Reproduced: I generated my own Ed25519 pair, ran factory approve ... --actor human:mateo --seed-env MINE, then spec promote ... --owner-public mine.pub: candidate -> promoted, ledger verifies, the approval_recorded event carries my keyid. So the gate binds promotion to whoever runs the host command, not to the owner's key: a model with shell access to the host CLI promotes exactly as before, with one extra step. For v0 this is acceptable only because the operator, a person, runs factory spec by hand; GAPS 65 (1) says the approval is a signature rather than a kernel event, but does not say the key is unpinned. Fix: record the owner's public key once in the ledger (an event at bootstrap, or the key the certificate module already trusts) and refuse an --owner-public that differs; or drop the flag and read the pinned key.
M2 (medium, secrets): the candidate's process inherits the host's environment, the signing seed with it. harness.record starts the child with no env= (harness.py:660-662; -I drops only PYTHON*), and _run_tests does the same for the suite (cli.py:657). When factory check --seed-env NAME runs, NAME's value is in that environment, so the recorded program or the candidate's test suite can read the seed that signs the attestation; with M1, it could also sign an owner approval. Through the Kit the check's environment is scrubbed (checks._environment), so this bites only when NAME is passed through SYBERLABS_CHECK_ENV, which is the only way a Kit-run check can sign at all. The harness docstring says the child inherits the environment "so a program's own secrets stay its own"; the host's seed is not the program's. Fix: build the child's and the suite's environment without the seed variable (the parent has already read it), or pass an allow-list.
L1 (low, trust boundary): a killed run's journal is delivered unverified. deliver = child_journal.exists() is set before the killed branch and never cleared there (harness.py:683-685), so the file is moved into runs/ at harness.py:712-714 without verify_journal. Reproduced with a sleeping entrypoint and timeout=2: an empty runs/<id>.jsonl landed in the ledger. load_run refuses it (run_unrecorded) and _record/_check return 1, so it is a stray file, not evidence; but verify() does not name it, and the body's "a journal that fails its seal is not delivered" does not hold for this path. Fix: deliver = False when killed.
L2 (low): Approved is a marker, not a capability. class Approved(dict) (spec.py:65-66) can be constructed by any code, and transition accepts it (spec.py:198-200). Reproduced in process: transition(doc, "promote", human, approval=Approved({...}), verdict_ids=[...]) moved a candidate to promoted with no signature. The CLI only ever produces one through gate(), so this does not weaken the command-line gate; the docstring's "Only verify_approval makes one" is overstated. A module-private construction token, or transition re-verifying the digest and key, would make the claim true.
L3 (low): the approved digest is over the parsed spec without history, not the file's bytes. spec_digest (spec.py:183-185) and before_digest (cli.py:850) digest the loaded document minus history. Consistent on both sides, so it is not a bypass, and I confirmed a re-indented file still promotes; but an edit to history after signing is invisible to the approval, and docs/FACTORY.md says "its digest as it is on disk", which reads as bytes. Say "its content".
L4 (low, hygiene): GAPS entry number 65 collides with main. #162 merged to main at 8a5e21f with its own entry 65 (guard-lane-2); #154 and this branch are based on older main. On the rebase this entry becomes 66, and the PR body's "GAPS 65" references go with it. Nothing else in the diff cites the number (docs/FACTORY.md and the harness docstring cite spec/GAPS.md without one), and tests.test_gaps would fail on two 65s, so the collision cannot land silently.
L5 (low, hygiene): the new factory -> git edge imports a private name. checks.py is in syberlabs-git in the declaration, so the edge is the right layer and test_repository_layering_has_no_drift passes; but both call sites import checks._stop (harness.py:618, cli.py:646). Give it a public name (checks.stop_tree) before more callers depend on an underscore.
Read for bypasses, held
- The approval is bound to
operation,spec_id,version,spec_digest, sortedverdict_idsand the approver (spec.py:69-79, checked atspec.py:101-111); a promote approval cannot be replayed for canonize or a later version (reproduced).transitionrechecks operation, spec id, version, approver and verdict ids (spec.py:201-207). resolve_verdictsrequires areviewevent withdecision == acceptand thevalidateevent that wrote that variant with aspec_digestequal to the file's (spec.py:114-137). Verdict ids are deterministic digests, so a re-run review resolves to the same id.- The ledger digests are over bytes (
ledger.py:70-71);verify()/problems()cover spaces, journals, their blobs, attestations and approvals (ledger.py:110-147);_checkverifies before and after (cli.py:486-502);project()skips an edited journal (ledger.py:337-338). - The per-run key and nonce reach the child only in the stdin config, which
_mainconsumes anddels (harness.py:457,501); the key stays onrecorder._keyand the nonce in_main's frame, which is exactly the residual GAPS 65 (2) states. The fixtures reach the recorder the same way (sys.modules["_factory_harness"]._CURRENT[-1]), so the residual is demonstrated, not hypothetical. The ledger path is not in the child's argv or environment; its cwd is the worktree, and the contract there names the absolute home, as GAPS 65 (3) says. - Independence reads the model the command declared (
spec.party,cli.py:769,794); a command can lie, as GAPS 65 (5) says.independence()refuses an unknown family and the same family under two names (reproduced:a=... gpt-5vsb=reviewrefusedboth served by openai).critic.pyunchanged. - Ledger appends go through
syberlabs.journal.Journal(lock, flush+fsync, torn-tail truncation under the lock,journal.py:144-205);_stalecompares the projection'smetahead and count with the log (ledger.py:367-378). bootstrapcopies the contract'stimeout_seconds(cli.py:616-619);full_suiteisexit_code is not None(cli.py:675).
Not-done list
Accurate as far as it goes: serves, mission_ref, done_when; mandatory human_owner; promote -> pass; tickets.py; codemap.py; tlog anchoring; the kernel approval seam. Two C1 items from the consolidation packet are deferred but appear only under "Known limitations", not in the deferred list: the behavior-change label moving from the check's argv to the ticket (GAPS 65 (4)) and signing the ledger's chain head with witness/tlog (D1; GAPS 65 (3)). Name them in the list so the next packet picks them up.
On GAPS 65 (1)
A signed owner approval standing in for a kernel approved event is acceptable for v0 provided the key is pinned (M1) and the entry says the gate currently trusts the caller's key. Without the pin, the entry understates what is open: the model-promotes-a-goal finding is closed for a model that cannot run the host CLI, and open for one that can.
Read-only verification; nothing merged or pushed, the PR body was not edited, #163 was not touched.
…the ledger (owner_key_set; bootstrap once, rotation signed by the pinned key; owner_key_unset and owner_key_mismatch refuse), run the candidate and its test suite under the scrubbed check environment without the host's seed, deliver no journal for a killed run, make Approved constructible by verify_approval only, name checks.stop_tree, and record the content-digest binding and the deferred C1 items M1: the approval verifies under the key the ledger pins, read from the events, never a key the caller names. `factory owner --public KEY` (or the first `--owner-public`) pins it once; `factory owner --public NEW --seed-env OLD` rotates under the pinned key's signature, stored under approvals/ and checked by verify. M2: harness.record and the tests runner build the child's environment from checks.scrubbed_environment and strip SYBERLABS_CERTIFICATE_SEED and the check's --seed-env variable. L1: deliver=False on the kill branch. L2: Approved takes a module-private token; transition checks it. L3: FACTORY.md says content digest (parsed, history excluded) and why the bytes are not bound. L5: checks.stop_tree and checks.scrubbed_environment are the public names. GAPS 65 states the trust model; the entry keeps its number on this branch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Follow-up
|
| Finding | Change | Test |
|---|---|---|
| M1 owner key unpinned | The key is pinned on the ledger as an owner_key_set event and spec.gate reads it from the events, never from argv. factory owner --public KEY (or the first --owner-public) pins it once (bootstrap, Class C, by hand); factory owner --public NEW --seed-env OLD rotates under the pinned key's signature, stored under approvals/ and checked by verify. No pin → owner_key_unset; any other key → owner_key_mismatch; --owner-public naming a different key on a pinned ledger → owner_key_mismatch; rotation without the pinned key's seed → owner_key_rotation. |
CliTests::test_the_owner_key_is_pinned_and_rotates_only_under_the_old_key (the verifier's own-keypair repro, both with and without --owner-public; rotation refused and then accepted; old key refused after rotation; tampered rotation named by verify), SpecTests::test_verdicts_resolve_against_the_ledger (owner_key_mismatch, owner_key_unset) |
| M2 child inherits the host env | harness.record and _run_tests build the child's environment from checks.scrubbed_environment (the Kit's routine: temp HOME/TMP, PATH, locale, caches, SYBERLABS_CANDIDATE/SYBERLABS_BASE passed through so a floor runner still sees its base) and strip SYBERLABS_CERTIFICATE_SEED plus the check's --seed-env name. |
HarnessTests::test_the_child_and_the_tests_runner_run_without_the_hosts_seed: a child reading len(os.environ.get(NAME, '')) sees 0 for both names, and so does the suite |
| L1 killed run's journal delivered | deliver = False on the kill branch. |
HarnessTests::test_a_killed_run_delivers_no_journal: runs/ stays empty, status killed |
L2 Approved is a marker |
Constructor demands a module-private token (approval_forged otherwise) and transition checks the token, so neither a dict nor an instance made around the constructor passes. Docstring fixed. |
in SpecTests::test_verdicts_resolve_against_the_ledger |
| L3 "as on disk" | FACTORY.md and the docstrings now say content digest (parsed, history excluded). Decision: content, not bytes: binding the bytes would refuse the re-indented file the verifier showed as acceptable for nothing gained; the ledger's spec_changed events are the history of record, and the doc says so. |
— (wording) |
| L4 GAPS number | Kept at 65 on this branch (tests here expect one 65). Renumber note: on the rebase to main this entry becomes 66 (main has #162's 65), and the body's "GAPS 65" references go with it. |
tests.test_gaps green |
L5 private checks._stop |
checks.stop_tree and checks.scrubbed_environment added as public one-line aliases; both call sites use them. |
layering test unchanged and green |
| Not-done list | FACTORY.md's "Next work package" now carries the explicit deferred list, with the two C1 items added: behavior-change label argv → ticket, and chain-head signing via witness/tlog. GAPS 65 (1) states the trust model precisely (bound to the pinned key, not the caller; bootstrap is the one unsigned act). |
— |
Verification here (Windows 11, CPython 3.12): tests.test_factory 48/48 (was 45); test_critic test_journal test_package test_gaps test_checks 61/61; conformance.run 23 matched; benchmarks/run_scenarios.py 11 replayed, exit 0. The M1 repro through the CLI: approve --seed-env MINE then spec retire --approval mine.json --owner-public mine.pub → exit 2 owner_key_mismatch; without the flag → exit 2 owner_key_mismatch; the pinned owner's approval → candidate -> retired, ledger verifies.
CI (syberlabs/factory-goal-gate-ci force-pushed to c78268a): all 18 runs for the SHA green (push and pull_request: ci on 3.11/3.12/3.13, admit, factory-gates, g2-g5, GitHits policy). Full suite: https://github.com/SyberLabs/SyberWork/actions/runs/41424384700 ; PR ci: https://github.com/SyberLabs/SyberWork/actions/runs/41424387078
Not changed: the two deferred C1 items above are recorded, not implemented; set_owner_key reads then appends under two lock acquisitions, the same shape as set_baseline.
Delta verification of
|
| Step | Result |
|---|---|
factory owner on an empty ledger |
exit 1 (unset) |
spec retire --approval owner.json, no pin |
exit 2 owner_key_unset |
factory owner --public owner.pub |
pinned (bootstrap) at seq 2; again: already pinned |
my key's approval, --owner-public mine.pub |
exit 2 owner_key_mismatch (cli.py:917) |
| my key's approval, no flag | exit 2 owner_key_mismatch (spec.py:220) |
owner --public mine.pub without seed / with my seed |
exit 2 owner_key_rotation both |
owner --public mine.pub --seed-env OLD |
rotated from <owner>; owner_key_set carries previous and rotation_digest |
| old key's approval after rotation | exit 2 owner_key_mismatch |
| new key's approval | candidate -> retired; verify() True |
| rotation file edited | problems(): approvals/<digest> does not match its digest |
Also held: gate() reads the key from events only (spec.py:215); an envelope whose keyid names another key is refused before any crypto, and one whose keyid lies is refused by verified_payload (dsse.py, keyid is a hint, the signature is checked). --owner-public on a pinned ledger with the same key is a no-op, with a different key refuses, and rotation is only factory owner (cli.py:908-917). The bootstrap event goes through append and is in the hash chain like every event. A failed first promote leaving the key pinned (cli.py:914 runs before gate) is the documented Class C race: whoever reaches an empty ledger first owns it; that is what the doc says, and it is right for a by-hand bootstrap.
M2
harness.record (harness.py:645-648, 674) and _run_tests (cli.py:671-675) build the child's env from checks.scrubbed_environment and pop SYBERLABS_CERTIFICATE_SEED plus strip_env; _check passes (args.seed_env,) (cli.py:505). My probe: child and suite see length 0 for both names, PATH present, SYBERLABS_CANDIDATE/SYBERLABS_BASE pass through as the host has them. scrubbed_environment is checks._environment and stop_tree is checks._stop (checks.py:122-123), the same routine run_check's caller uses at checks.py:197. L5 closed at both call sites.
L1-L4
- L1:
deliver = Falseon the kill branch (harness.py:699); the new test showsruns/empty. Held. - L2:
Approved.__init__demands_ISSUED(spec.py:85-88),transitionchecks_is_issued(spec.py:278); a dict and a__new__-made instance are both refused (spec_approval), a bareApproved({...})raisesapproval_forged. Held. Note:specmod._ISSUEDis a module attribute, so in-process code that imports it still constructs one; "nothing outside this module can construct" is Python convention, not enforcement. Fine for the CLI gate, which is the claim that matters; the docstring could say "by convention". - L3: content digest,
historyexcluded, now stated inFACTORY.mdand the docstrings; decision recorded. Held. - L4: one
65.inspec/GAPS.md(line 129),tests.test_gapsgreen; the renumber-to-66 note is in the follow-up comment, not in the file, which is the right place since the number is right on this branch.
New finding
M3 (medium, goal gate): verify() does not check the owner-key lineage, so one appended owner_key_set re-roots the gate and the ledger still verifies. pinned_owner_key takes the newest owner_key_set (spec.py:100-103) with no lineage check, and problems() looks at an owner_key_set only when it carries a rotation_digest, and then only checks that the file exists and matches its digest (ledger.py:143-150); it never calls verify_rotation, never checks previous against the key pinned before, and never refuses a second bare bootstrap. set_owner_key enforces all of that at write time (ledger.py:309-332), but append is public (ledger.py:159) and the chain is unkeyed (GAPS 65 (3)). Reproduced, each after an honest pin and rotation, each leaving verify() True with problems() == []:
home.append("owner_key_set", {"keyid": ATTACKER, "previous": None, "rotation_digest": None})->owner_key()is the attacker's;gate()accepts the attacker's approval; through the CLI,spec retire --approval third.json->candidate -> retired.- A rotation event whose envelope is signed by the new key, not the pinned one, with the file written at its digest -> accepted, undetected.
- A rotation whose
previousis a key rotated out two events earlier -> undetected. keyid: "not-a-key"as the last event ->owner_key()is None (owner_key_unsetfor everyone),verify()True.
Precondition: a write to events.jsonl, which is the same precondition as forging any event today. But the pin changes what that write buys: before this commit forging the key needed the host's argv; now it needs one line in a file a candidate can learn the path of from its own worktree's contract (GAPS 65 (3)), and _check re-verifies the ledger after the recording (cli.py:512-513) but not after _run_tests (cli.py:527), so the test suite's window is open until record_attestation (cli.py:548). The follow-up's own words, "every later one is a rotation signed by the key it replaces" (ledger.py:21-22) and "set_owner_key is the only writer" (test comment, tests/test_factory.py:958), are true of the writer and not of the verifier. Fix, small: in problems(), walk the owner_key_set events in order; the first must have previous: None; each later one must have previous == the keyid before it, a rotation_digest, and an envelope that verify_rotations under previous (and a _HEX_KEY keyid throughout); and have pinned_owner_key (or gate) use that walk rather than the last event. Until then, GAPS 65 (1) should say the pin is as strong as the ledger's write boundary, which (3) says is unanchored. The two-lock read-then-append in set_owner_key (noted by the follow-up) produces exactly shape 1 under a concurrent bootstrap, so the same check covers it.
#163
origin/syberlabs/autonomy-thin-slice is at f9eaf2a, which sits on c78268a (merge-base is c78268a). tests.test_factory_autonomy on that head: OK, exit 0 (25 test methods, no failures or errors; the loop's ledger trace runs through loop_stopped). Its promotion_record reads the key pinned before the approval event, which is the right historical reading; M3 above applies to it the same way, since it trusts the same unverified lineage.
Read-only verification; nothing merged or pushed, the PR body was not edited, #163 was not touched.
… owner key through the ledger's verified lineage, not its newest owner_key_set; verify the ledger before the gate and again after the candidate's test suite; make set_owner_key one lock acquisition M3: `pinned_owner_key` took the newest `owner_key_set`, `problems()` checked a rotation file for existence and digest only, and `append` is public, so one appended line re-rooted the gate with `verify()` True. Now `spec.owner_key_lineage(events, load_rotation)` walks every `owner_key_set` in order: the first is the one bare bootstrap (`previous` None, no `rotation_digest`); every later one carries `previous` equal to the key pinned before it and a `rotation_digest` whose stored envelope `verify_rotation`s under that previous key; every keyid is 64-hex. Any violation is a named `owner_key_lineage` problem, `verify()` is False, and the lineage pins nothing: `pinned_owner_key`, `Ledger.owner_key`, `spec.gate`, `factory owner` and `set_owner_key` refuse with `owner_key_lineage` rather than falling back to the last or an earlier key, so a second bare bootstrap is a violation, not a re-pin. `Ledger.load_rotation` supplies the envelopes; `gate` takes it as `load_rotation`, and the key still comes from the events alone. `spec promote|canonize|retire` verify the ledger (`problems()`) before the gate reads it and refuse `ledger_corrupt`; `factory check` verifies it once more after `_run_tests` and before `record_attestation`, against the head its own run record left, and refuses `ledger_corrupt` if it moved or has problems. `set_owner_key` reads the pinned key and appends under one acquisition of the journal lock (`_append`). Tests: the verifier's four shapes (bare appended attacker pin; rotation signed by the new key; stale `previous`; malformed keyid last) each leave the chain intact and each give `verify()` False, an `owner_key_lineage` problem, an `owner_key_lineage` refusal from the gate for the forged and the honest approval alike, a refused rotation, and `spec retire` exit 2; the honest bootstrap -> rotation -> retire path still passes; a test suite that appends to the ledger refuses the check at `ledger_corrupt` with nothing attested. The M1 test's edited-rotation tail now expects the lineage break. `tests/test_factory.py` is 50 tests; `docs/FACTORY.md` states the lineage rule; GAPS 65 (1) says the pin is as strong as the verified lineage and the boundary that remains is (3), a rewrite of the whole chain, for the deferred tlog anchoring. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Follow-up
|
…iled to tickets on the ledger, each feature built by a Build Thread the runner drives, inside an envelope; nothing merges featuremap.py: feature-map.v1 over #154's spec object (sectioned Markdown -> spec; D2), mapcheck (schema, spec digest, coverage, acyclic DAG, declared scope conflicts, act class, size, hidden-check names, approval digest), the task graph (critical path, total work, work/critical ratio, Dilworth width, conflict graph), the risk plan (seven code-computed inputs, standardized scores, v_i = V * softmax(z_i / tau) above a floor that never moves) and one feature -> one ticket in ticket_from_spec's shape (D5). policy.py: FeaturePolicy with the five typed questions over closed option sets, OfflinePolicy (critical path first, cheapest tier that fits, retry then stronger arm then owner, park when spent, stop on the envelope) and ShadowPolicy (records what an injected second policy would answer; never acts on it). loop.py: refuses an unapproved or stale map and an unpromoted spec, writes one contract per feature, drives each attempt with `syberlabs run`'s Runner (D17), reserves and settles an envelope ledger shaped like the eval spend ledger (D13), and records every proposal, step and outcome as ledger events whose digests exclude thread ids, commits and times. autonomy_cli.py: `syberlabs factory autonomy check|approve|envelope|run|why`, hooked from one line at the end of cli.py. 17 tests on a three-feature fixture over factory_app with a stand-in proposer command: tickets on the ledger, F1 -> F3 -> F2 order, shadow disagreement recorded, envelope and kill switch stop the loop, await_merge leaves a dependent waiting, a failing or out-of-scope candidate is refused by the kernel and parked, same fixture gives the same event digests, no merge or network call in the modules. docs/FACTORY_AUTONOMY.md says what the slice does and does not do, the glossary, and the next packets (C5, C6, C9, C10). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…motion record through C1's seam (the promote event, the approval it cites, the stored envelope, re-verified under --owner-public), retry_escalate_arm offers only the stronger arms so the recorded decision is the one that runs, one envelope ledger file per envelope digest so two maps or two grants in one home never share spend or a stop, stacking restores the owner's HEAD and refuses a dirty tree, the envelope ledger refuses negative or non-finite amounts, a kindless line, a second reserve and an orphan settle, a Rejected inside run() still writes the stop line and loop_stopped, the kill-switch path leaves the digest, scope paths compare case-folded without ./, risk_plan validates its weights, and the docs list the known limits and the deviations from C4 (#163 verification findings 1-10, 13) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d re-verifies the promotion's approval under the owner key the ledger pinned before the approval event (owner_key_unset and owner_key_mismatch refuse), autonomy run takes no --owner-public, the fixture bootstraps the key with set_owner_key and promotes through gate() as the CLI does, and the docs pin the key with factory owner Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Delta verification of
|
| Shape | Result |
|---|---|
second bootstrap with previous = first key, no rotation_digest |
refused, "a second pin is a rotation and names a rotation_digest" (spec.py:128) |
envelope verifies under the pinned key but its payload names another next |
refused by verify_rotation's next check (spec.py:172-173) |
rotation_digest cites the honest first->second envelope for an attacker keyid |
refused, payload previous/next do not match this handover |
approvals/<digest>.json missing |
refused, "missing, altered or unavailable" (spec.py:132); fail closed |
Ledger.load_rotation handed a name the stored envelope does not digest to |
None (ledger.py:323); ../x, "", None also None |
extra fields on an owner_key_set |
ignored by the walk; the bare-pin shape is still refused |
| uppercase-hex keyid | refused, _HEX_KEY is lowercase (spec.py:67) |
| any other writer to the log | writer._write appears once, inside _append (ledger.py:178); append and set_owner_key both go through it |
_check and _spec
cli.py:533-535: after _run_tests, head must equal the check's own run_recorded hash, length before + 1, problems() empty, else ledger_corrupt. test_a_ledger_that_changes_under_the_test_suite_refuses_the_check passes here: the suite's note lands, verdict.reason == candidate_check_failed:factory, no attested event. cli.py:914-917: problems() runs before the gate for promote/canonize/retire, inside the GATED branch; gate is then given home.load_rotation (cli.py:930).
New finding
M4 (medium-low, goal gate): the lineage walk accepts the same rotation envelope twice, so an honest A -> B handover replays after the owner has rotated B away. A rotation payload is {protocol, previous, next, at} (spec.py:155), bound to nothing on the chain, and the walk (spec.py:113-138) checks each owner_key_set against the key pinned right before it only; it never asks whether a rotation_digest was already used. Reproduced: bootstrap A; set_owner_key(B, seed=A) (envelope S1); set_owner_key(A, seed=B) (honest: the owner takes the ledger back from B, as one does when B leaks); then append("owner_key_set", {"keyid": B, "previous": A, "rotation_digest": S1}). owner_key_lineage() is (B, []), verify() True, problems() == [], and spec retire --approval b.json (signed by B) is candidate -> retired, exit 0. The attacker needs a ledger write and B's seed, which is exactly the key the rotation was meant to retire. GAPS 65 (1) says the remaining boundary is "an attacker who can rewrite the whole chain consistently"; this one is an append.
Fix, small: in owner_key_lineage, refuse a rotation_digest that already appeared earlier in the walk (closes exact replay); better, put the ledger head hash (or the seq of the owner_key_set being replaced) into rotation_payload when set_owner_key signs it and have the walk check it against the chain, which also binds the handover to its position and covers an honestly signed envelope that was stored but never appended. Until then, FACTORY.md "The lineage rule" (docs/FACTORY.md:116-118) and GAPS 65 (1) should say the lineage is as strong as the set of keys that were ever pinned, not only the current one.
Notes, not findings
- The follow-up's Factory autonomy thin slice (AM1): an owner-approved feature map compiled to tickets, each feature built by a Build Thread the runner drives, inside an envelope; nothing merges #163 note is right:
loop.promotion_recordatf9eaf2acallspinned_owner_keywithout a loader and will raiseowner_key_lineageon any rotated ledger once rebased. Not touched here. set_owner_keywrites the rotation file (ledger.py:364) inside the lock but before_append; a crash between the two leaves an orphan underapprovals/that nothing cites, which is harmless to the walk.
Read-only verification; nothing merged or pushed, the PR body was not edited.
…: promotion_record reads the pinned key through the ledger's verified lineage with its stored rotations, a lineage that does not verify refuses owner_key_lineage, the fixture gates with the ledger's loader, and the docs say map compilation honours the lineage Rebased onto syberlabs/factory-goal-gate at 9ff7573 (M3 of the #164 delta verification). The two branches touch disjoint files except the end-of-file hook in factory/cli.py; no conflicts. The adaptation #164's summary asked for: `loop.promotion_record` now calls `specmod.pinned_owner_key(prefix, ledger.load_rotation)` so a rotated ledger's envelopes can be checked, and `owner_key_lineage` propagates as the refusal instead of collapsing into `owner_key_unset`. The historical-prefix read (events with seq below the approval event's) still picks the key pinned at the time of the approval, so a later honest rotation does not unmake an earlier promotion. Because that prefix cannot see a pin appended after the approval, the whole ledger's lineage is verified first (`specmod.pinned_owner_key(events, ledger.load_rotation)`): a bare second `owner_key_set` appended with the hash chain intact leaves the ledger pinning nothing, and `Loop.compile` refuses `owner_key_lineage` rather than reading the earlier key. Tests (26, was 25): `EndToEnd::test_compile_reads_the_owner_key_through_the_ledgers_lineage` promotes a spec after an honest bootstrap -> rotation (approval signed by the second key) and compiles under the second key, shows the same lineage is unreadable without the loader, then appends the M3 repro (a bare `owner_key_set`) after an honest promotion on a second ledger: `compile` refuses `owner_key_lineage`, appends nothing, writes no contract. The fixture's `promoted_spec` passes `load_rotation=home.load_rotation` to `gate`, as the CLI does. Docs: FACTORY_AUTONOMY.md states that map compilation honours the ledger's owner-key lineage. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…a replayed envelope (M4) The rotation envelope (now owner-key/v1; v0 is refused) is signed over the seq of the owner_key_set it replaces and the ledger head at signing, the lineage walk checks both against the chain, and a rotation_digest cited twice is `rotation replayed`, so an honest A -> B handover cannot be appended again after the owner rotated B away, and an envelope signed for one position fails at any other. New CliTests case covers the replay, a moved envelope, a wrong replaces_seq and a v0 envelope end to end (verify, problems, gate, CLI); the M3 fixture signs over its position. FACTORY.md lineage rule and GAPS 65 (1)/(3) now say the remaining boundary is a consistent rewrite of the chain, not an append. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
M4 closed in
Touched only |
Final stack verification, A: the M4 delta (
|
Factory autonomy thin slice (AM1): an owner-approved feature map compiled to tickets, each feature built by a Build Thread the runner drives, inside an envelope; nothing merges
Admission
Class: C (a gate definition: who may promote a spec)
Agent-platform: claude
Reviewed-by: none recorded (the #154 verifier is asked to re-verify; consolidation packet C2)
Admitted-by: pending (owner key)
Evidence: tested
Stacked on #154 (
claude/factory-spacetime-v0at6dbd2f4). Not run through a Build Thread: the thread kit needsmain, and this branch's base is unmerged. Plain git, full tests locally, the full suite on the scratch branchsyberlabs/factory-goal-gate-ci(same commit). The owner merges this into #154 or lands both.Objective
Consolidation packet C1
factory-goal-gate: make the goal-hierarchy rule code (the factory proposes, the owner promotes, with a verifying approval), and close the high and medium findings of #154's independent verification (S1 artifact integrity, S2 the candidate records itself, S3 promote verdicts, S4 reviewer independence, S5 verify prints the signer's policy, S6 secret exposure) plus the low ones (S7 viewer, S8 durability, S9 timeouts).What changed
spec promote|canonize|retirerequire a human--actor(kind:id, resolved to a principal; free text refused) and an approval:factory approve SPEC --operation OP --verdict V --actor human:ID --seed-env NAME --out FILEsigns (certificate module Ed25519, DSSE) a payload naming the spec's id, version and digest as on disk, the operation and the verdict ids;spec promote --approval FILE --owner-public KEYverifies it, requires approver == actor, resolves each--verdictto a ledgerreviewthat accepted the variant whosevalidatedigest is this file's, refuses an approver who proposed or reviewed it, stores the approval underapprovals/, logsapproval_recorded, and thespec_changedcites that event's hash.transitiontakes only theApprovedrecordverify_approvalreturns. The variants loop's--actor model:<reviewer>hint is gone. No kernel case exists to hang anapprovedevent on yet, so the signature stands in; recorded in GAPS 65.load_runandload_spacerefuse a file whose digest is not the recorded one (artifact_tampered), an unlogged journal (run_unrecorded);set_baselinegoes throughload_run;verify()/problems()cover Space graphs, journals, their blobs, attestations and approvals;project()skips an edited journal;factory checkverifies the ledger before the candidate runs and again after it exits and refuses if anything moved.harness.recordruns the child underpython -Iin a host-owned scratch directory (outside the worktree and the ledger), never tells it the ledger's path, and moves the journal in afterwards. Every record is sealed (HMAC chained under a per-run key the parent holds);verify_journalrefuses a planted or edited line, a torn tail or an empty file, and such a journal is not delivered. The report travels over a private pipe bound by the run's nonce (the hidden checks'run_shardspattern); stdout is output; the report's status must match theendrecord the parent reads itself. Residual: the recorder shares the candidate's memory (GAPS 65).syberlabs/factory/independence.py:family()frommodel,signal.harness.modelorprovider.model;independence()true only when both families are recorded and differ.judgeuses it; the stand-in proposers declare their models.criticis unchanged.factory verifyprintsattest.recompute(predicate)and exits 1 on a mismatch with the written policy;record_attestation(document, public)logssigned: trueonly when the signature verifies, elseenvelope: true, signed: false.harness.scrub,scrub_argv,scrub_text: the kernel's shared credential pattern (_patterns.SECRET), bearer tokens, PEM keys, credentialKEY=value, the value after a credential flag; applied to the subprocess observer's argv and cwd, the socket address,_run_tests's tail (capped) and the viewer's io argv. The viewer carries effects by digest, no payloads.textContent/createElement; noinnerHTML.syberlabs.journal.Journal(lock, fsync, torn-tail repair) under<home>/.journal/; the projection rebuilds when itsmetarow no longer matches the log and keeps thefeaturestable.bootstrapcarries the contract's tests timeout (--tests-timeout), the factory check's timeout is 600 + that;full_suiteis "ran to completion";_run_testsandharness.recordstop the process tree on timeout withchecks._stop(declaredfactory -> gitedge).docs/FACTORY.md: the gate, the trust boundary, independence, limits, 45 tests (it said 18).spec/GAPS.md65.Files:
syberlabs/factory/{harness,ledger,spec,attest,cli}.py,syberlabs/factory/independence.py(new),syberlabs/static/factory.html,docs/architecture.declared.json,docs/FACTORY.md,spec/GAPS.md,tests/test_factory.py,tests/fixtures/factory_app/{proposers.py,fxapp/variants.py}.What did not change
githits*.py,query_guard.py,metrics.py,scripts/run_shards.py,session.py,settle.py, the prompt builder: untouched.critic.py,journal.py,checks.py: untouched (reused).Evidence
The #154 verifier's repros, re-run on this branch (Windows, CPython 3.12.6):
baselineverify()True,load_runreturns the edited content,checkdiffs against itverify()False (runs/<id> does not match the digest recorded at seq N),load_run->artifact_tampered,set_baselinerefused,factory check-> exit 2ledger_corruptbefore the candidate runs{"run_id":..., "status":"ok"}and prevents theendpromotereadsstatus: okerror: the journal has no end record; journal not delivered; outcome not taken;check-> "no journal", nothing attestedend) into its journal around the recorderjournal_unsealed: line N carries no seal; not deliveredspec promote --verdict X --actor model:reviewerspec_actorspec promote --verdict X --actor human:mateo(no approval)spec_approval--actor operator(free text){"kind": "operator", "id": "operator"}actor_invalid--verdict review-1(a string)spec_verdict: not a review on the ledgerspec_verdict(digest is not the reviewed variant)spec_changed.approval.event_hash== theapproval_recordedhashCI:
ci.ymlon the scratch branchsyberlabs/factory-goal-gate-ci(and on this branch's push). Result reported in the packet report.Known limitations
approvedevent on a kernel goal case; the ledger event is the citation, the journal hash is not (GAPS 65 (1))._factory_harness._CURRENT[-1]and its key throughsys.modules. The seal and the nonce raise the cost of a forgery; this is not a sandbox, and the recommendation on the open owner question stands: evidence with a person in the path (GAPS 65 (2)).verifydetects an inconsistent rewrite, not a consistent one across runs. Anchoring the chain head intlog/witnessis the fix (GAPS 65 (3)).independencereads the model the command declared, asmodel_usagedoes.--behavior-changeon the check's argv), not per ticket.serves,mission_ref,done_whenon specs; mandatoryhuman_owneron tickets;promote->passrename;tickets.py;codemap.py.🤖 Generated with Claude Code