diff --git a/docs/FACTORY_AUTONOMY.md b/docs/FACTORY_AUTONOMY.md new file mode 100644 index 0000000..5e0a8d3 --- /dev/null +++ b/docs/FACTORY_AUTONOMY.md @@ -0,0 +1,197 @@ +# The autonomy thin slice (AM1): an approved feature map, built to "ready for owner merge" + +Packet C4 of the consolidation plan (`reports/2026-11-01-syberfactory-consolidation.md`, section 5), built on the +factory of `docs/FACTORY.md`: the same ledger, the same spec object, the same tickets, the same Build Thread. It +realises the first slice of the autonomous typed mode (`2026-10-28-syberfactory-autonomous-typed-mode.md`) as +three modules under `syberlabs/factory/` and one CLI group, at $0, offline, with a stand-in command where a +model would be. + +Claim ceiling for this page: **implemented** and **tested** (`tests/test_factory_autonomy.py`, 25 tests, run on +Windows with Python 3.12 and in CI). Nothing here is measured on a real spec, deployed, or validated by anyone +outside the repository. This slice sits on the goal-gate packet (C1, `syberlabs/factory-goal-gate`): the spec it +builds under is one the owner promoted through `factory approve` and `factory spec promote`, and the loop checks +that on the ledger, not in the spec file. + +## What it does + +```text +spec (promoted) ─► FeatureMap (typed DAG) ─► mapcheck ─► owner approval ─► tickets ─► one Build Thread per attempt ─► ready for owner merge + featuremap.py loop.compile loop.run (Runner from `syberlabs run`) + FeaturePolicy asks: next_feature, proposer_arm, budget_tier, after_failure, worth_continuing + envelope ledger: reserve before, settle after, stop on the line +``` + +| Module | What it holds | +| --- | --- | +| `featuremap.py` | `feature-map.v1`: features with id, objective, `req_ids` (the spec requirements they trace to), scope paths, shown criteria, hidden-check names (names only), `depends_on`, act class, size, risk inputs. `spec_from_markdown` turns a sectioned Markdown spec (`## R1: text`, `- given, when, then` bullets) into #154's spec object (owner decision 11, D2). `mapcheck` is the deterministic floor: schema, spec digest, coverage, acyclic DAG, scope conflicts (an intersection must be declared with an order that agrees with the DAG), act class, size ceilings, hidden-check names, approval digest. `taskgraph`: critical path, total work, work/critical ratio, Dilworth width with a maximum antichain, conflict graph. `risk_plan`: seven risk inputs computed by code, `r_i` as a weighted sum, standardized `z_i`, `v_i = V * softmax(z_i / tau)` as integer units above a floor that is listed beside every allocation and never moves. `compile_ticket`: one feature to one ticket in `ticket_from_spec`'s shape (D5). `degenerate_map`: the map `ticket_from_spec` implies, one feature per requirement. | +| `policy.py` | `Question` (closed options built by code; the state a policy reads is ids and numbers only), `Answer`, `validate` (an answer outside the options is `policy_invalid`), `OfflinePolicy` (critical path first, cheapest arm and tier, retry then stronger arm then owner, park when the cap is spent, stop on the envelope), `ShadowPolicy` (acts on one policy, records what an injected second one would have answered and whether it agrees; an invalid or failing shadow is recorded and changes nothing), `recorded_answers` (a shadow over a fixture). | +| `loop.py` | `Loop.compile`: refuses a map without a current owner approval (`map_unapproved`), a spec the ledger did not promote (`spec_unpromoted`, see `promotion_record`), or a map with findings; writes one contract per feature; logs `map_frozen` (with `promoted_by`) and `ticket_opened{source: map}`. `Loop.run`: kill switch, eligibility, three policy questions recorded as proposals, envelope reservation, one unattended Build Thread, one proposer call through `CommandProvider`, then `syberlabs run`'s `Runner` drives the thread (D17), the envelope settles from the journal, the outcome is recorded; a `Rejected` while running still writes the envelope's `stop` and the ledger's `loop_stopped{reason: }` before it propagates. `promotion_record`: the goal gate's record on the ledger for exactly this spec (the `spec_changed` promote whose `spec_digest` is the file's, the `approval_recorded` it cites, the envelope under `approvals/`), re-verified under the owner key the ledger pinned at that point (`owner_key_set`, read through the ledger's verified owner-key lineage with its stored rotations, `Ledger.load_rotation`); a lineage that does not verify (a bare second pin, an unsigned or stale rotation, a malformed key) pins nothing and refuses `owner_key_lineage`; `owner_key_unset` and `owner_key_mismatch` refuse. `EnvelopeLedger`: the eval spend ledger's shape generalized to attempts (D13), one file per envelope digest under `envelopes/`, strict on every line. `why`: ticket -> feature -> map (approval) -> spec (promotion), from the ledger. `event_digests`: one digest per autonomy event without its volatile `refs`. | +| `autonomy_cli.py` | `syberlabs factory autonomy check | approve | envelope | run | why`, registered from one hook line at the end of `factory/cli.py`. | + +### A run on the fixture + +`tests/fixtures/autonomy/` holds a three-feature fixture over the factory's own `factory_app`: `spec.md` (three +sections), `map.json` (F1 -> F2 a dependency chain, F3 independent and larger), `hidden_checks.json` (the host's +catalogue, name to argv) and `standin_proposer.py` (the process boundary a model adapter sits behind: it reads the +request, appends a delivery marker to the one file in scope, and answers one candidate; `--break` and `--outside` +make it fail the visible check or edit outside its scope). + +The end-to-end test's ledger, in order: + +```text +map_frozen ticket_opened ×3 +policy_proposal(next_feature=F1) policy_proposal(proposer_arm) policy_proposal(budget_tier) +envelope_reserved attempt_started thread_step(check acceptable) thread_step(accept succeeded) envelope_settled feature_ready +policy_proposal(next_feature=F3) ... feature_ready +policy_proposal(next_feature=F2) ... feature_ready +loop_stopped(nothing_eligible) +``` + +F1 goes first (bottom level 2: a dependent waits on it), then F3 (size M, bottom level 2), then F2. Each feature +ends as an accepted candidate on `syberlabs/`, with the thread id, commit and target ref in the +`feature_ready` event's `refs`. `main` never moves, and the owner's checkout is as it was: HEAD on the same branch +at the same commit, no scratch branch left. Under `integration: stack` F2's thread starts from F1's accepted +commit; under `await_merge` F2 is recorded `feature_waiting` and the loop stops with F1 and F3 ready, because only +the owner's merge can release it. The same fixture run twice gives the same `event_digests` list: no timestamp, +thread id, commit, elapsed time, approval digest or kill-switch path enters a digest. + +After a failure, `retry_same` offers the next attempt every arm again; `retry_escalate_arm` (offered only when the +envelope holds an arm ranked above the one that failed) offers only the stronger arms, so the arm on +`attempt_started` is the arm the recorded decision named. The test runs `cheap, cheap, strong` then parks. + +### From a terminal + +```sh +python -m syberlabs.factory --home .syberlabs/factory owner --public owner.pub # pin the owner key on the ledger (once) +python -m syberlabs.factory --home .syberlabs/factory spec validate spec.json # the spec, as in FACTORY.md +python -m syberlabs.factory --home .syberlabs/factory approve spec.json --verdict VERDICT --actor human:mateo \ + --seed-env SYBERLABS_FACTORY_OWNER_SEED --out approval.json # the goal gate (C1): the owner signs +python -m syberlabs.factory --home .syberlabs/factory spec promote spec.json --verdict VERDICT --actor human:mateo \ + --approval approval.json # verified under the pinned key, recorded, cited +python -m syberlabs.factory autonomy check --map map.json --spec spec.json # findings, graph, risk plan; exit 1 on findings +python -m syberlabs.factory autonomy approve --map map.json --by mateo # the owner's act; any edit leaves it stale +python -m syberlabs.factory autonomy envelope --map map.json --calls 6 --attempts 2 --out envelope.json +python -m syberlabs.factory --home .syberlabs/factory autonomy run --map map.json --spec spec.json --envelope envelope.json \ + --owner mateo --proposer "/abs/python standin.py ." --check "tests=/abs/python -m unittest -q" \ + --hidden "hidden.f1.marker=/abs/python -c ..." # exit 0 when a feature is ready +python -m syberlabs.factory --home .syberlabs/factory autonomy why TICKET +``` + +`run` reads the spec's promotion from the same `--home` the promote wrote to and re-verifies its approval under the +owner key that ledger pins; no key is passed on the command line. Map compilation honours the ledger's owner-key +lineage (FACTORY.md): the key is the one the verified chain of `owner_key_set` events pinned before the approval, +each rotation checked against its stored envelope, and a ledger whose lineage does not verify compiles nothing +(`owner_key_lineage`), whatever its newest `owner_key_set` says. An envelope is one grant: its +ledger lives at `envelopes/.jsonl`, a second map or a new envelope in the same home starts +clean, and a run over an envelope whose ledger already carries a `stop` is refused (`envelope_stopped`). On +Windows, `--check` and `--hidden` commands with nested quotes need a script file (`split_command` is non-POSIX +there; pre-existing in #154). + +## What it does not do + +- **Publish or merge.** No action in the loop pushes, opens a pull request or merges; `loop.ACTIONS` is the closed + list `start_thread, propose, check, accept_to_thread_branch`, and a test asserts the three modules hold no merge + or network call. The terminal state is `ready_for_owner_merge`. The draft PR is C9. +- **Call Jev, or any model.** `ShadowPolicy` accepts an injected callable and records its answers; nothing imports + `jev_api` or opens a socket. Jev stays in shadow until E7 (owner decision 7). The adapter that reuses + `jev_api`'s request and validation code is a later packet; `recorded_answers` is its fixture-side stand-in. +- **Author the map.** The map is hand-written or produced elsewhere; `mapcheck` refuses what it cannot accept. + The variant loop as map author is C4's follow-up in `factory/variants.py` (D14), not this slice. +- **Grade with a replay.** The hidden checks run in the candidate's worktree as contract checks the host holds + (never in the repository), which is the Build Thread's own evidence. The independent replay grade (a fresh clone, + a fresh hidden copy, the SyberEval gate) is the `replay` arm, left for C9/C10 with the eval runner. +- **Run with concurrency.** `concurrency` is 1. The task graph's width and conflict graph are computed and logged + so the question of whether `c > 1` pays can be answered from the record; the scope lock is a check on declared + overlap order, not a scheduler. +- **Price anything.** Every proposer call here is a stand-in at $0. The envelope's dollar tiers are read and + reserved; a priced arm would settle from `search_finished.spend` exactly as the eval runner does. +- **Add an event kind.** `ledger.KINDS` belongs to `ledger.py` and to the goal-gate packet editing it. The slice + writes `note` events whose `body.note` names the autonomy event (`loop.EVENTS`) and `ticket_opened` with + `source: map`. When the vocabulary is extended, `loop.record` is the one place to change. + +## Glossary (per the consolidation report) + +- **Spec**: #154's spec object (`spec.py`). An L1 objective. Only an owner promotes it (C1 makes that code). +- **FeatureMap**: the typed task DAG of this page. L2 work. Frozen by the owner's approval digest. +- **Feature**: one node of the map; compiles to one ticket; built by one or more Build Threads (attempts). +- **Ticket**: `ticket_from_spec`'s shape with `map_ref`, scope as footprint, shown criteria, exit criteria naming + evidence kinds and hidden checks, `rubric_digest` over the hidden-check names, a mandatory `human_owner`. +- **Codemap**: #154's `features.py`, hashed vectors per symbol and component. An index the map author may query + and the `moved` sensor uses. It is never the feature map (D3; the rename to `codemap.py` is C1's). +- **Envelope**: the owner's grant of autonomy: dollars by tier, calls, seconds, attempts per feature, act classes, + arms, the closed action list, the kill switch. Its ledger reserves before and settles after every attempt. +- **Floor**: host checks, hidden checks, critic, safety gate. Every feature, every risk score. Extra verification + is only ever conjoined. +- **Promote**: an owner act on a spec. Not an attestation's verdict, not an accepted candidate (D4). +- **Shadow**: a policy whose answers are recorded beside the acting policy's and never acted on. + +## Next packets + +- **C5 `factory-check-real-checkout`**: `bootstrap` on a fresh SyberWork clone with `examples/factory_self_check.py` + as the golden; the published contract shape committed as an example; the four FACTORY.md candidates against the + real contract as a receipt. +- **C6 `independent-review`**: one `independence()` over the host-recorded served model's provider family (D8); + `review.independent` recorded but not blocking; `auto_merge_eligible` computed and journaled per feature; the + variant `judge` and the critic share it. The feature loop then records the composition of section 4 in shadow. +- **C9 `factory-publish` and the feature certificate**: the draft PR per ready feature with Feature, Map, Ticket, + Envelope and Grade fields from the record; a grep test that no merge call exists; `feature-delivered/v1` as a + third predicate referencing the child certificate and attestation digests (D6). Reads `settle` only. +- **C10 E8 `risk-allocation`**: planted defects in 2- to 4-feature maps; arms are allocation policies over one + budget V (uniform, softmax at tau = 1, argmax, water-filling); the metric is defects caught per unit and escapes + by risk decile. `risk_plan` here records the propensities that evaluation needs. + +## Limits of this slice + +- Stacking (`integration: stack`) starts a dependent from its last dependency's accepted commit. `Kit.start` starts + from HEAD, so the loop moves HEAD to that commit on a scratch branch `syberfactory//` for the start + only, then puts the owner's branch (or detached commit) back and deletes the scratch branch, whether or not the + start succeeded. A working tree with uncommitted tracked changes is refused first (`repository_dirty`), so nothing + is carried into a thread base or lost. A feature with two dependencies on separate branches is not integrated + here; the owner's merge order is. +- A hidden check runs as a contract check in the candidate's worktree, so it is as exposed to candidate code as any + check (register N4). The fixture's stand-in writes no code that runs; a model's would, and then the host decision + (C7) applies. +- The envelope ledger has no cross-process lock. C1 put the factory ledger on the journal's lock; the envelope file + is still appended without one (two loops on one envelope in one home is not a supported run). +- The risk weights and tau are priors. Nothing has calibrated them; `risk_plan` logs what calibration would need. + `risk_plan(weights=)` is validated (the known inputs, each in [0, 1], summing to 1) but not reachable from the CLI. +- **The map approval is a record, not a credential.** `featuremap.approve` writes `{digest, actor}` and + `approved()` checks the digest against the map, not the actor against a signature: swapping `approval.actor.id` + leaves the map approved. The spec's promotion is now bound to the owner's signed approval through C1's seam + (`promotion_record`, re-verified under the owner key the ledger's verified lineage pins); the map's approval has no such seam yet. Binding it is a + signed `approval_payload` over the map digest, which is C1's shape applied to a second artifact; proposed for the + packet that gives the factory a kernel case to hang approvals on. +- **The floor is listed, not enforced, by this slice.** `FLOOR` names host checks, hidden checks, critic and safety + gate, and every `buys` list carries all four. Only the first two are contract-required here (`required` in the + compiled contract is the visible checks plus the hidden checks). No critic runs and no safety gate is computed + until C6 (`independent-review`) and C9 bring them; "every feature, never loosened" is true of the two that exist. + +## Deviations from C4 as written + +The consolidation report's C4 row (section 5) asked for more than this slice ships. Each deviation is deliberate, +and none of them is hidden in "Deliberately not in this slice" above: + +- **The envelope ledger is re-implemented, not imported from #150's spend ledger (D13 "build it once").** The eval + spend ledger lives in `syberlabs/eval/`; `docs/architecture.declared.json` declares no `factory -> eval` edge, and + the layering test (`test_repository_layering_has_no_drift`) refuses an undeclared import. `EnvelopeLedger` copies + the shape (one JSON object per line, reserve before, settle after, strict read, non-finite refused) rather than the + code. Proposed for C11: declare `syberlabs-factory -> syberlabs-eval` (or lift the spend ledger into a module both + can import) and delete this copy. +- **`RecordedJevPolicy` does not reuse `jev_api`'s validator (D11).** Same layering: `jev_api` is outside the + factory's declared edges. `policy.validate` is the closed-option check; `recorded_answers` is the fixture-side + shadow. The adapter that reuses `jev_api`'s request and validation code is the packet that declares that edge. +- **Decisions are factory-ledger `note` events, not kernel-journal `observed{policy_decision}`, and `propose` is + untagged.** The kernel's `Thread.propose` takes no origin field today (an `origin` is recorded only for migrated + candidates, through `migrate`), and the kernel's event vocabulary is not this packet's to extend. Every policy + answer is a `policy_proposal` note on the factory ledger before it is acted on. Proposed for C6: an + `observed{policy_decision}` on the case and `propose(origin="model")` once the kernel accepts an origin on a + proposal. +- **`factory autonomy why` takes a ticket id only**, not a PR or thread id: there is no PR until C9, and the thread + id is in the `feature_ready` event the ticket walk already returns. +- **The fixture is `tests/fixtures/autonomy/`, not `factory/fixtures/spec-slugify-pricing/`.** It reuses the + factory's own `factory_app`, so the Space graph, the ledger and the hidden checks are the ones the rest of the + factory tests already exercise. A separate product fixture adds a second app to maintain for no extra coverage in + this slice. +- **`OfflinePolicy` is critical-path-first, not risk-per-unit-of-work-first.** `next_feature` orders by bottom + level, then risk, then map order. Risk per unit of work needs a cost model per feature that `risk_plan` does not + yet have (its units are verification units, not build cost); deferred until C10's E8 gives risk a measured scale. diff --git a/syberlabs/factory/autonomy_cli.py b/syberlabs/factory/autonomy_cli.py new file mode 100644 index 0000000..0d544fe --- /dev/null +++ b/syberlabs/factory/autonomy_cli.py @@ -0,0 +1,167 @@ +"""``syberlabs factory autonomy ...``: the thin slice from a terminal. + + syberlabs factory autonomy check --map map.json --spec spec.json # mapcheck, task graph, risk plan; exit 1 on findings + syberlabs factory autonomy approve --map map.json --by mateo # the owner's approval over the map's digest (G1) + syberlabs factory autonomy envelope --map map.json --calls 6 --out env.json + syberlabs factory autonomy run --map map.json --spec spec.json --envelope env.json --owner mateo \\ + --proposer "python standin.py ." --check "tests=python -m unittest -q" --hidden "hidden.f1.marker=python -c ..." + syberlabs factory autonomy why TICKET # ticket -> feature -> map -> spec, from the ledger + +Registered through ``extend`` from one hook line at the end of ``cli.py``, so that file takes no other edit for +this packet. ``--home`` is the factory ledger, ``--root`` the repository the features change. +""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + +from syberlabs.errors import Rejected +from syberlabs.factory import featuremap, loop +from syberlabs.factory import policy as policymod +from syberlabs.providers import split_command + +COMMAND = "autonomy" + + +def extend(populate, run): + """Wrap ``cli.populate`` and ``cli.run`` so the factory parser gains ``autonomy`` and dispatches it here.""" + def populate_with_autonomy(factory): + populate(factory) + steps = next(a for a in factory._actions if isinstance(a, argparse._SubParsersAction)) + add_commands(steps) + + def run_with_autonomy(args): + if getattr(args, "factory_command", None) != COMMAND: + return run(args) + try: + return _run(args) + except Rejected as exc: + print(f"error: {exc.code}: {exc.detail}", file=sys.stderr) + return 2 + + return populate_with_autonomy, run_with_autonomy + + +def add_commands(steps) -> None: + auto = steps.add_parser(COMMAND, help="the autonomous typed mode's thin slice: approved feature map -> tickets -> " + "Build Threads -> ready for owner merge, inside an envelope (docs/FACTORY_AUTONOMY.md)") + sub = auto.add_subparsers(dest="autonomy_command", required=True) + check = sub.add_parser("check", help="mapcheck findings, the task graph and the risk plan") + check.add_argument("--map", required=True) + check.add_argument("--spec", required=True, help="the promoted spec JSON (syberlabs factory spec ...)") + check.add_argument("--tau", type=float, default=1.0) + check.add_argument("--units", type=int, default=None, help="extra verification units to allocate (default: one per feature)") + check.add_argument("--json", action="store_true") + approve = sub.add_parser("approve", help="record the owner's approval over the map's current digest (a human act)") + approve.add_argument("--map", required=True) + approve.add_argument("--by", required=True, help="the approving person's id") + env = sub.add_parser("envelope", help="write an envelope for a map: dollars, calls, seconds, attempts") + env.add_argument("--map", required=True) + env.add_argument("--dollars", type=float, default=0.0) + env.add_argument("--calls", type=int, required=True) + env.add_argument("--seconds", type=float, default=3600.0) + env.add_argument("--attempts", type=int, default=2) + env.add_argument("--kill-switch", help="absolute path; when the file exists the loop stops before its next decision") + env.add_argument("--out", required=True) + runp = sub.add_parser("run", help="compile the approved map to tickets and drive each feature through a Build Thread") + runp.add_argument("--map", required=True) + runp.add_argument("--spec", required=True, help="the promoted spec JSON; its promotion is read from --home and re-verified under " + "the owner key the ledger pins (factory owner)") + runp.add_argument("--envelope", required=True) + runp.add_argument("--owner", required=True, help="the accountable human on every ticket") + runp.add_argument("--kit-home", default=".syberlabs", help="the Build Thread home under --root (default .syberlabs)") + runp.add_argument("--proposer", required=True, help="the proposer command (a stand-in or a model adapter), as one string") + runp.add_argument("--check", action="append", default=[], metavar="NAME=CMD", help="a visible check (repeatable)") + runp.add_argument("--hidden", action="append", default=[], metavar="NAME=CMD", + help="a hidden check the host holds under the name the map uses (repeatable; never in a worktree)") + runp.add_argument("--baseline", help="the golden run id to name on tickets") + runp.add_argument("--max-steps", type=int, default=50) + runp.add_argument("--shadow", help="recorded answers JSON for a shadow policy; recorded beside each decision, never acted on") + why = sub.add_parser("why", help="walk a ticket back to its feature, map, approval, spec and its promotion") + why.add_argument("ticket") + + +def _parse_checks(items: list[str], what: str) -> dict: + out = {} + for item in items: + name, found, command = item.partition("=") + if not found or not name or not command.strip(): + raise Rejected("check_argument", f"{what} must be NAME=COMMAND, not {item!r}") + out[name] = split_command(command) + return out + + +def _run(args) -> int: + from syberlabs.factory import ledger as ledgermod + command = args.autonomy_command + if command == "check": + document = loop.load_json(args.map, "map") + spec = loop.load_json(args.spec, "spec") + findings = featuremap.mapcheck(document, spec) + if findings: + for finding in findings: + print(f"finding {finding}") + print(f"{len(findings)} finding(s); the map is not compiled") + return 1 + graph = featuremap.taskgraph(document) + plan = featuremap.risk_plan(document, graph, budget=args.units, tau=args.tau) + if args.json: + print(json.dumps({"graph": graph, "plan": plan, "approved": featuremap.approved(document)}, indent=2, sort_keys=True)) + return 0 + print(f"map {document['id']} {featuremap.map_digest(document)[:12]}: {graph['features']} features, {graph['edges']} edges; " + f"approved {featuremap.approved(document)}") + print(f"critical path {' -> '.join(graph['critical_path'])} (length {graph['critical_path_length']}); total work " + f"{graph['total_work']}; work/critical {graph['work_critical_ratio']}; width {graph['width']} {graph['max_antichain']}") + for conflict in graph["conflicts"]: + print(f"conflict {conflict['a']} {conflict['b']} on {conflict['paths']}" + (" (ordered by the DAG)" if conflict["ordered_by_dag"] else "")) + for fid in plan["r"]: + print(f"risk {fid:5} r {plan['r'][fid]:.3f} z {plan['z'][fid]:+.2f} share {plan['share'][fid]:.3f} " + f"units {plan['units'][fid]} buys {', '.join(plan['buys'][fid][len(featuremap.FLOOR):]) or '-'}") + print(f"floor {', '.join(plan['floor'])} (every feature, never loosened); tau {plan['tau']} on standardized scores") + return 0 + if command == "approve": + document = loop.load_json(args.map, "map") + approval = featuremap.approve(document, {"kind": "human", "id": args.by}) + Path(args.map).write_text(json.dumps(document, indent=2, sort_keys=True) + "\n", encoding="utf-8") + print(f"approved {document['id']} at {approval['digest'][:12]} by {args.by}; any edit of the map leaves this stale") + return 0 + if command == "envelope": + document = loop.load_json(args.map, "map") + envelope = loop.new_envelope(featuremap.map_digest(document), dollars_total=args.dollars, calls_total=args.calls, + seconds_total=args.seconds, attempts_per_feature=args.attempts, kill_switch=args.kill_switch) + Path(args.out).write_text(json.dumps(envelope, indent=2, sort_keys=True) + "\n", encoding="utf-8") + print(f"wrote {args.out}: envelope {loop.envelope_digest(envelope)[:12]} for map {document['id']}") + return 0 + if command == "run": + document = loop.load_json(args.map, "map") + spec = loop.load_json(args.spec, "spec") + envelope = loop.load_json(args.envelope, "envelope") + policy = policymod.OfflinePolicy() + if args.shadow: + policy = policymod.ShadowPolicy(policy, policymod.recorded_answers(loop.load_json(args.shadow, "shadow")), shadow_name="recorded") + checks = {name: {"argv": argv, "timeout_seconds": 300} for name, argv in _parse_checks(args.check, "--check").items()} + if not checks: + raise Rejected("check_argument", "name at least one visible check with --check NAME=COMMAND") + root = Path(args.root).resolve() + kit_home = Path(args.kit_home) + driver = loop.Loop(repo_root=root, kit_home=kit_home if kit_home.is_absolute() else root / kit_home, + ledger=ledgermod.Ledger(args.home), map=document, spec=spec, envelope=envelope, policy=policy, + proposer_argv=split_command(args.proposer), checks=checks, hidden_catalogue=_parse_checks(args.hidden, "--hidden"), + human_owner=args.owner, baseline_run=args.baseline, log=print) + tickets = driver.compile() + for ticket in tickets: + print(f"ticket {ticket['ticket_id']} {ticket['map_ref']['feature_id']} {ticket['title']}") + summary = driver.run(max_steps=args.max_steps) + for fid, row in summary["features"].items(): + print(f"feature {fid:5} {row['state']:22} attempts {row['attempts']}" + (f" {row['last_reason']}" if row["last_reason"] else "")) + totals = summary["envelope"] + print(f"stopped: {summary['stopped']}; envelope ${totals['dollars']:.4f}, {totals['calls']} calls, {totals['seconds']}s; " + f"ready for owner merge: {', '.join(summary['ready_for_owner_merge']) or 'none'}. Nothing was merged or pushed.") + return 0 if summary["ready_for_owner_merge"] else 1 + if command == "why": + print(json.dumps(loop.why(ledgermod.Ledger(args.home), args.ticket), indent=2, sort_keys=True)) + return 0 + raise Rejected("autonomy_command", f"unknown command {command!r}") diff --git a/syberlabs/factory/cli.py b/syberlabs/factory/cli.py index f1cc515..94d4f75 100644 --- a/syberlabs/factory/cli.py +++ b/syberlabs/factory/cli.py @@ -1027,3 +1027,9 @@ def main(argv: list[str] | None = None) -> int: formatter_class=argparse.RawDescriptionHelpFormatter) populate(parser) return run(parser.parse_args(argv)) + + +# The autonomy thin slice (docs/FACTORY_AUTONOMY.md) registers its subcommands from its own module; this hook is the +# only edit this file takes for it. ``main`` and ``syberlabs/cli.py`` look ``populate`` and ``run`` up at call time. +from syberlabs.factory import autonomy_cli as _autonomy_cli # noqa: E402 +populate, run = _autonomy_cli.extend(populate, run) diff --git a/syberlabs/factory/featuremap.py b/syberlabs/factory/featuremap.py new file mode 100644 index 0000000..7b0f358 --- /dev/null +++ b/syberlabs/factory/featuremap.py @@ -0,0 +1,654 @@ +"""FeatureMap: the typed task DAG between a promoted spec and its tickets (``syberfactory/feature-map.v1``). + +A spec (``spec.py``, the one spec object, consolidation D2) is the objective. A feature map is the work: a DAG +of features, each with an objective a Build Thread can start from, a scope (the paths it may edit), the +acceptance criteria the builder is shown, the names of the hidden checks the host holds (names only; the +checks themselves never enter a worktree), the features it depends on, and the inputs of its risk score. +Every feature traces to at least one requirement of the spec (a requirement is a section, so coverage is +checkable), and every requirement is covered or listed as ``uncovered`` with a reason. + +This module is code, not a model. ``mapcheck`` is the deterministic floor the plan-to-one P3 gate needs: +schema, spec digest, coverage, DAG (acyclic), scope conflicts (an intersection of two scopes must be +declared with an order that agrees with the DAG), act class, size ceilings, hidden-check names, and the +owner-approval digest. ``taskgraph`` gives the four standard measures (critical path, total work, their +ratio, Dilworth width with a maximum antichain) and the conflict graph. ``risk_plan`` standardizes the risk +scores and allocates extra verification as ``v_i = V * softmax(z_i / tau)`` above a hard floor that never +moves: host checks, hidden checks, critic and gate apply to every feature whatever its score. + +Relation to the rest of the factory: + +- D2: the map is derived from #154's spec object. ``spec_ref`` pins ``spec_id``, ``version`` and + ``spec_digest``; feature ``req_ids`` are the spec's ``req_id``s. ``degenerate_map`` is the map + ``ticket_from_spec`` implies: one feature per requirement, no dependencies. +- D3: this is the FeatureMap. ``features.py`` (hashed vectors) is the codemap, an index; it is not imported here. +- D5: ``compile_ticket`` turns one feature into one ticket in ``ticket_from_spec``'s shape: scope is the + footprint, shown criteria are ``acceptance_criteria``, evidence kinds and hidden-check names are the exit + criteria, and ``rubric_digest`` is the digest of the hidden-check names. A ticket's ``map_ref`` carries the + map digest and feature id, so ``why`` can walk ticket -> map -> spec. + +Nothing here has an opinion: a map that fails a check is refused with the findings, never repaired. +""" + +from __future__ import annotations + +import math +import re + +from syberlabs.canonical import digest +from syberlabs.errors import Rejected +from syberlabs.factory import PROTOCOL +from syberlabs.factory import spec as specmod + +SCHEMA = "syberfactory/feature-map.v1" +ACT_CLASSES = ("R", "C", "P", "X") # syberlabs/build.py ACT_CLASSES; copied so the factory keeps its declared edges +SIZES = ("S", "M", "L") +DURATION = {"S": 1, "M": 2, "L": 4} # abstract work units per size tier +ACT_RANK = {"R": 0, "C": 1, "P": 2, "X": 3} +INTEGRATIONS = ("await_merge", "stack") +MIN_FEATURES, MAX_FEATURES = 1, 40 +MAX_CRITERIA = 8 +MAX_FILES_CEILING, MAX_DIFF_BYTES_CEILING = 12, 40_000 +FEATURE_FIELDS = frozenset({"id", "title", "objective", "req_ids", "depends_on", "scope", "overlaps", "act_class", + "criteria", "hidden_checks", "size", "risk_inputs"}) +MAP_FIELDS = frozenset({"schema", "id", "spec_ref", "base", "author", "features", "uncovered", "integration", "approval"}) +_FID = re.compile(r"^F[0-9]{1,3}$") +_CHECK = re.compile(r"^[a-z][a-z0-9_.-]{0,63}$") # a contract check name (build._NAME) +_SECTION = re.compile(r"^##\s+(?P[A-Za-z][A-Za-z0-9_-]*)\s*[:.]\s*(?P.+?)\s*$") + +# Risk: weights sum to 1 so r is in [0, 1]. From the theory report, section 4; unvalidated priors. +WEIGHTS = {"scope": 0.15, "act_class": 0.15, "new_deps": 0.10, "needs_lookup": 0.10, "disagreement": 0.20, + "centrality": 0.20, "uncovered": 0.10} +ACT_RISK = {"R": 0.0, "C": 0.5, "P": 0.75, "X": 1.0} +FLOOR = ("host_checks", "hidden_checks", "critic", "safety_gate") # every feature, every score; never loosened +LADDER = ("independent_replay_rerun", "extra_seeds_x2", "adversarial_review", "owner_blind_review") +TAU_MIN = 0.05 # keeps every propensity positive (the overlap condition for calibration) + + +# -- spec side: sectioned Markdown -> #154's spec object (owner decision 11, D2) ---------------------------------- + +def spec_from_markdown(text: str, spec_id: str, capability: str, footprint_hint: list[str], *, + author: dict | None = None, evidence_kinds: list[str] | None = None) -> dict: + """A sectioned Markdown spec (``## R1: requirement text`` then ``- given ... / when ... / then ...`` lines) + becomes a draft #154 spec with one requirement per section. Scenario bullets of the form + ``- given X, when Y, then Z`` become scenarios; other bullets are kept as free text scenarios.""" + requirements, current = [], None + for line in text.splitlines(): + found = _SECTION.match(line) + if found: + current = {"req_id": found.group("id"), "text": found.group("text"), "scenarios": [], + "evidence_kinds": list(evidence_kinds or ["tests"])} + requirements.append(current) + continue + stripped = line.strip() + if current is not None and stripped.startswith("- "): + current["scenarios"].append(_scenario(stripped[2:])) + if not requirements: + raise Rejected("spec_invalid", "no '## : ' section found") + return specmod.new_spec(spec_id, capability, requirements, footprint_hint, author=author) + + +def _scenario(text: str) -> dict: + parts = re.split(r",\s*(?=when\b|then\b)", text, flags=re.IGNORECASE) + if len(parts) == 3 and parts[0].lower().startswith("given"): + return {"given": parts[0][5:].strip(), "when": parts[1][4:].strip(), "then": parts[2][4:].strip()} + return {"text": text} + + +# -- the map ------------------------------------------------------------------------------------------------------- + +def new_map(map_id: str, spec: dict, features: list[dict], *, base: str | None, author: dict, + uncovered: list[dict] | None = None, integration: str = "await_merge") -> dict: + """A map over ``spec`` as it is now. Validation is ``mapcheck``; this only assembles.""" + return {"schema": SCHEMA, "id": map_id, + "spec_ref": {"spec_id": spec["spec_id"], "version": spec["version"], "spec_digest": specmod.spec_digest(spec)}, + "base": base, "author": dict(author), "features": [dict(f) for f in features], + "uncovered": list(uncovered or []), "integration": integration, "approval": None} + + +def feature(fid: str, title: str, objective: str, *, req_ids: list[str], paths: list[str], depends_on: list[str] = (), + criteria: list[dict] | None = None, hidden_checks: list[str] = (), act_class: str = "R", size: str = "S", + overlaps: list[dict] = (), exclude: list[str] = (), max_files: int = 6, max_diff_bytes: int = 20_000, + new_dependencies: int = 0, needs_lookup: bool = False, disagreement: float = 0.0) -> dict: + return {"id": fid, "title": title, "objective": objective, "req_ids": list(req_ids), "depends_on": list(depends_on), + "scope": {"paths": list(paths), "exclude": list(exclude), "max_files": max_files, "max_diff_bytes": max_diff_bytes}, + "overlaps": [dict(o) for o in overlaps], "act_class": act_class, + "criteria": [dict(c) for c in (criteria or [])], "hidden_checks": list(hidden_checks), "size": size, + "risk_inputs": {"new_dependencies": new_dependencies, "needs_lookup": needs_lookup, "disagreement": disagreement}} + + +def degenerate_map(spec: dict, *, base: str | None, author: dict) -> dict: + """The map ``ticket_from_spec`` implies (D5): one feature per requirement, no dependencies, the spec's + resolved footprint files as every feature's scope, scenarios as the shown criteria. Two requirements over + one footprint therefore overlap; the degenerate map declares each overlap in requirement order.""" + files = (spec.get("footprint") or {}).get("files") or [] + if not files: + raise Rejected("footprint_unresolved", "resolve the spec's footprint against a Space graph first") + features = [] + for number, requirement in enumerate(spec["requirements"], 1): + fid = f"F{number}" + criteria = [{"id": f"{fid}.c{i}", "text": _criterion_text(s), "kind": "test"} + for i, s in enumerate(requirement["scenarios"], 1)] or [{"id": f"{fid}.c1", "text": requirement["text"], "kind": "judgment"}] + overlaps = [{"with": f"F{other}", "paths": sorted(files), "order": f"F{min(number, other)}_first"} + for other in range(1, len(spec["requirements"]) + 1) if other != number] + features.append(feature(fid, requirement["text"][:80], requirement["text"], req_ids=[requirement["req_id"]], + paths=sorted(files), criteria=criteria, overlaps=overlaps, + max_files=min(MAX_FILES_CEILING, max(1, len(files))))) + return new_map(f"{spec['spec_id']}.degenerate", spec, features, base=base, author=author) + + +def _criterion_text(scenario: dict) -> str: + if "text" in scenario: + return scenario["text"] + return f"given {scenario.get('given', '')}, when {scenario.get('when', '')}, then {scenario.get('then', '')}" + + +def map_digest(document: dict) -> str: + """The map without its approval, so approval is over exactly this digest.""" + return digest({k: v for k, v in document.items() if k != "approval"}) + + +def approve(document: dict, actor: dict) -> dict: + """The owner's approval (G1): a human actor over the map's digest. Any later edit of the map leaves the + approval stale, and ``approved`` says so. This records; it does not decide that the map is good.""" + if not isinstance(actor, dict) or actor.get("kind") != "human" or not actor.get("id"): + raise Rejected("approval_actor", "a map is approved by a human actor {kind: human, id: ...}, never a model") + document["approval"] = {"digest": map_digest(document), "actor": {"kind": "human", "id": str(actor["id"])}} + return document["approval"] + + +def approved(document: dict) -> bool: + approval = document.get("approval") + return (isinstance(approval, dict) and approval.get("digest") == map_digest(document) + and isinstance(approval.get("actor"), dict) and approval["actor"].get("kind") == "human" + and bool(approval["actor"].get("id"))) + + +# -- mapcheck ------------------------------------------------------------------------------------------------------ + +def normalize_path(path: str) -> str: + """A scope entry as compared: backslashes as slashes, leading ``./`` segments and the trailing slash + removed, case folded (the kernel's scope check runs on case-insensitive filesystems too).""" + text = path.strip().replace("\\", "/") + while text.startswith("./"): + text = text[2:] + return text.rstrip("/").casefold() + + +def path_intersects(a: str, b: str) -> bool: + """Prefix intersection of two scope entries at directory boundaries, after :func:`normalize_path`. + ``"."`` or empty is the whole tree.""" + ad, bd = normalize_path(a), normalize_path(b) + if ad in ("", ".") or bd in ("", "."): + return True + return ad == bd or bd.startswith(ad + "/") or ad.startswith(bd + "/") + + +def scope_intersection(f: dict, g: dict) -> list[str]: + out = set() + for p in f["scope"]["paths"]: + for q in g["scope"]["paths"]: + if path_intersects(p, q): + out.add(p if len(p.rstrip("/")) >= len(q.rstrip("/")) else q) + return sorted(out) + + +def mapcheck(document: dict, spec: dict) -> list[str]: + """Every finding, as strings. Empty means the map may be compiled. Order: shape, then graph, then coverage, + then conflicts, then approval. A shape finding stops the deeper checks, which would only cascade.""" + findings: list[str] = [] + note = findings.append + if not isinstance(document, dict): + return ["map: not a JSON object"] + missing, extra = MAP_FIELDS - set(document), set(document) - MAP_FIELDS + if missing or extra: + return [f"map: missing {sorted(missing)} unknown {sorted(extra)}"] + if document["schema"] != SCHEMA: + note(f"map.schema must be {SCHEMA!r}") + if not isinstance(document["id"], str) or not document["id"].strip(): + note("map.id must be a non-empty string") + ref = document["spec_ref"] + if not (isinstance(ref, dict) and set(ref) == {"spec_id", "version", "spec_digest"}): + note("map.spec_ref must be {spec_id, version, spec_digest}") + else: + if ref["spec_id"] != spec["spec_id"] or ref["version"] != spec["version"]: + note(f"map.spec_ref names {ref['spec_id']} v{ref['version']}, the spec is {spec['spec_id']} v{spec['version']}") + if ref["spec_digest"] != specmod.spec_digest(spec): + note("map.spec_ref.spec_digest does not equal the spec's digest (the spec changed after the map was written)") + if document["base"] is not None and not (isinstance(document["base"], str) and document["base"]): + note("map.base must be a commit string or null") + author = document["author"] + if not (isinstance(author, dict) and author.get("kind") in ("human", "model", "command") and author.get("id")): + note("map.author must be {kind: human|model|command, id}") + if document["integration"] not in INTEGRATIONS: + note(f"map.integration must be one of {INTEGRATIONS}") + features = document["features"] + if not (isinstance(features, list) and MIN_FEATURES <= len(features) <= MAX_FEATURES): + note(f"features must be a list of {MIN_FEATURES}..{MAX_FEATURES}") + return findings + req_ids = [r["req_id"] for r in spec["requirements"]] + ids: list[str] = [] + check_names: dict[str, str] = {} + for index, item in enumerate(features): + where = f"feature[{index}]" + if not isinstance(item, dict): + note(f"{where}: not an object") + continue + missing, extra = FEATURE_FIELDS - set(item), set(item) - FEATURE_FIELDS + if missing or extra: + note(f"{where}: missing {sorted(missing)} unknown {sorted(extra)}") + continue + fid = item["id"] + if not (isinstance(fid, str) and _FID.match(fid)): + note(f"{where}: bad id {fid!r} (F1..F999)") + continue + if fid in ids: + note(f"{where}: duplicate id {fid}") + ids.append(fid) + where = fid + if not (isinstance(item["title"], str) and item["title"].strip() and isinstance(item["objective"], str) and item["objective"].strip()): + note(f"{where}: title and objective must be non-empty strings") + if not (isinstance(item["req_ids"], list) and item["req_ids"]): + note(f"{where}: req_ids must name at least one spec requirement (every feature traces to a section)") + else: + for rid in item["req_ids"]: + if rid not in req_ids: + note(f"{where}: req_ids names unknown requirement {rid!r}") + if not (isinstance(item["depends_on"], list) and all(isinstance(x, str) for x in item["depends_on"])): + note(f"{where}: depends_on must be a list of feature ids") + elif len(set(item["depends_on"])) != len(item["depends_on"]) or fid in item["depends_on"]: + note(f"{where}: depends_on repeats an id or names itself") + scope = item["scope"] + if not (isinstance(scope, dict) and set(scope) == {"paths", "exclude", "max_files", "max_diff_bytes"}): + note(f"{where}: scope must be exactly {{paths, exclude, max_files, max_diff_bytes}}") + else: + paths = scope["paths"] + if not (isinstance(paths, list) and paths and all(isinstance(p, str) and p.strip() for p in paths)): + note(f"{where}: scope.paths must be a non-empty list of strings") + elif any(p.startswith("/") or ".." in p.split("/") or p.strip() in (".", "./") for p in paths): + note(f"{where}: scope.paths must be repository-relative, without '..', and not the whole tree") + if not (isinstance(scope["exclude"], list) and all(isinstance(p, str) for p in scope["exclude"])): + note(f"{where}: scope.exclude must be a list of strings") + for key, ceiling in (("max_files", MAX_FILES_CEILING), ("max_diff_bytes", MAX_DIFF_BYTES_CEILING)): + value = scope[key] + if type(value) is not int or not 1 <= value <= ceiling: + note(f"{where}: scope.{key} must be an int 1..{ceiling} (size check)") + if not isinstance(item["overlaps"], list): + note(f"{where}: overlaps must be a list") + else: + for overlap in item["overlaps"]: + if not (isinstance(overlap, dict) and set(overlap) == {"with", "paths", "order"} and isinstance(overlap["with"], str) + and isinstance(overlap["paths"], list) and isinstance(overlap["order"], str)): + note(f"{where}: overlap entries must be exactly {{with, paths, order}}") + elif overlap["order"] not in (f"{fid}_first", f"{overlap['with']}_first"): + note(f"{where}: overlap with {overlap['with']} has order {overlap['order']!r}; it must be '{fid}_first' or '{overlap['with']}_first'") + if item["act_class"] not in ACT_CLASSES: + note(f"{where}: act_class must be one of {ACT_CLASSES}") + criteria = item["criteria"] + if not (isinstance(criteria, list) and 1 <= len(criteria) <= MAX_CRITERIA): + note(f"{where}: criteria must be a list of 1..{MAX_CRITERIA} (what the builder is shown)") + else: + seen = set() + for criterion in criteria: + if not (isinstance(criterion, dict) and set(criterion) == {"id", "text", "kind"}): + note(f"{where}: criterion fields must be exactly id, text, kind") + continue + if not (isinstance(criterion["id"], str) and re.match(rf"^{re.escape(fid)}\.c[0-9]+$", criterion["id"])) or criterion["id"] in seen: + note(f"{where}: bad or duplicate criterion id {criterion.get('id')!r}") + seen.add(criterion["id"]) + if not (isinstance(criterion["text"], str) and criterion["text"].strip()): + note(f"{where}: criterion {criterion['id']} has empty text") + if criterion["kind"] not in ("test", "judgment"): + note(f"{where}: criterion {criterion['id']} kind must be test or judgment") + hidden = item["hidden_checks"] + if not (isinstance(hidden, list) and all(isinstance(h, str) and _CHECK.match(h) for h in hidden)): + note(f"{where}: hidden_checks must be a list of check names (names only; the checks stay with the host)") + else: + for name in hidden: + if name in check_names and check_names[name] != fid: + note(f"{where}: hidden check {name!r} is also named by {check_names[name]}") + check_names.setdefault(name, fid) + if item["size"] not in SIZES: + note(f"{where}: size must be S, M or L") + risk = item["risk_inputs"] + if not (isinstance(risk, dict) and set(risk) == {"new_dependencies", "needs_lookup", "disagreement"} + and type(risk["new_dependencies"]) is int and risk["new_dependencies"] >= 0 + and isinstance(risk["needs_lookup"], bool) + and isinstance(risk["disagreement"], (int, float)) and not isinstance(risk["disagreement"], bool) + and math.isfinite(risk["disagreement"]) and 0 <= risk["disagreement"] <= 1): + note(f"{where}: risk_inputs must be {{new_dependencies: int >= 0, needs_lookup: bool, disagreement: 0..1}}") + if findings: + return findings + by_id = {f["id"]: f for f in features} + for item in features: + for dep in item["depends_on"]: + if dep not in by_id: + note(f"{item['id']}: depends_on names unknown feature {dep!r}") + for overlap in item["overlaps"]: + if overlap["with"] not in by_id or overlap["with"] == item["id"]: + note(f"{item['id']}: overlap names unknown feature {overlap['with']!r}") + if findings: + return findings + deps = {f["id"]: list(f["depends_on"]) for f in features} + cycle = find_cycle(deps) + if cycle: + note("depends_on has a cycle: " + " -> ".join(cycle)) + # coverage: every requirement is traced by a feature or listed as uncovered with a reason, never both + uncovered = document["uncovered"] + listed = set() + if not isinstance(uncovered, list): + note("uncovered must be a list") + else: + for entry in uncovered: + if not (isinstance(entry, dict) and set(entry) == {"req_id", "reason"} and entry["req_id"] in req_ids + and isinstance(entry["reason"], str) and entry["reason"].strip()): + note(f"uncovered entry {entry!r} must be {{req_id: , reason: }}") + else: + listed.add(entry["req_id"]) + covered = {rid: sorted(f["id"] for f in features if rid in f["req_ids"]) for rid in req_ids} + for rid in req_ids: + if not covered[rid] and rid not in listed: + note(f"requirement {rid} is covered by no feature and is not listed as uncovered") + if covered[rid] and rid in listed: + note(f"requirement {rid} is both covered ({covered[rid]}) and listed as uncovered") + # conflicts: an intersection must be declared, with one order that does not contradict the DAG + ancestors_ = ancestors(deps) if not cycle else {} + for i, a in enumerate(ids): + for b in ids[i + 1:]: + intersection = scope_intersection(by_id[a], by_id[b]) + declared = ([o for o in by_id[a]["overlaps"] if o["with"] == b] + [o for o in by_id[b]["overlaps"] if o["with"] == a]) + if intersection and not declared: + note(f"scope conflict: {a} and {b} both claim {intersection} and declare no overlap") + if declared and not intersection: + note(f"{a} and {b} declare an overlap but their scopes do not intersect") + orders = {o["order"] for o in declared} + if len(orders) > 1: + note(f"{a} and {b} declare contradictory overlap orders {sorted(orders)}") + elif orders and ancestors_: + first = next(iter(orders))[: -len("_first")] + second = b if first == a else a + if second in ancestors_.get(first, set()): + note(f"overlap order {first}_first contradicts depends_on ({first} depends on {second})") + approval = document["approval"] + if approval is not None and not approved(document): + note("approval is present but stale or not by a human: it does not cover this map's digest") + return findings + + +def find_cycle(deps: dict[str, list[str]]) -> list[str] | None: + white, grey, black = 0, 1, 2 + colour = {k: white for k in deps} + stack: list[str] = [] + + def visit(node): + colour[node] = grey + stack.append(node) + for other in deps.get(node, []): + if other not in colour: + continue + if colour[other] == grey: + return stack[stack.index(other):] + [other] + if colour[other] == white: + found = visit(other) + if found: + return found + stack.pop() + colour[node] = black + return None + + for node in deps: + if colour[node] == white: + found = visit(node) + if found: + return found + return None + + +def ancestors(deps: dict[str, list[str]]) -> dict[str, set[str]]: + """Transitive dependencies (what each feature waits for).""" + memo: dict[str, set[str]] = {} + + def go(node): + if node not in memo: + memo[node] = set() + for dep in deps.get(node, []): + memo[node] |= {dep} | go(dep) + return memo[node] + + for node in deps: + go(node) + return memo + + +# -- task graph ----------------------------------------------------------------------------------------------------- + +def taskgraph(document: dict) -> dict: + """Critical path (forward and backward pass over size durations), total work and the work/critical-path + ratio (the most concurrency that can pay), Dilworth width with one maximum antichain (bipartite matching + and Koenig's construction), topological levels, slack, and the conflict graph of scope intersections.""" + features = document["features"] + ids = [f["id"] for f in features] + order = {fid: i for i, fid in enumerate(ids)} + deps = {f["id"]: list(f["depends_on"]) for f in features} + if find_cycle(deps): + raise Rejected("map_cyclic", "the task graph has a cycle; run mapcheck first") + duration = {f["id"]: DURATION[f["size"]] for f in features} + anc = ancestors(deps) + children = {fid: [g for g in ids if fid in deps[g]] for fid in ids} + topo = _topological(ids, deps, children, order) + est, eft = {}, {} + for node in topo: + est[node] = max((eft[d] for d in deps[node]), default=0) + eft[node] = est[node] + duration[node] + length = max(eft.values()) if eft else 0 + bottom = {} + for node in reversed(topo): + bottom[node] = duration[node] + max((bottom[c] for c in children[node]), default=0) + slack = {node: length - (est[node] + bottom[node]) for node in ids} + path, current = [], None + heads = [n for n in ids if not deps[n] and slack[n] == 0] + current = min(heads, key=order.get) if heads else None + while current: + path.append(current) + nxt = [c for c in children[current] if slack[c] == 0 and est[c] == eft[current]] + current = min(nxt, key=order.get) if nxt else None + level = {} + for node in topo: + level[node] = 0 if not deps[node] else 1 + max(level[d] for d in deps[node]) + antichain = _max_antichain(ids, anc) + conflicts = [] + by_id = {f["id"]: f for f in features} + for i, a in enumerate(ids): + for b in ids[i + 1:]: + intersection = scope_intersection(by_id[a], by_id[b]) + if intersection: + conflicts.append({"a": a, "b": b, "paths": intersection, "ordered_by_dag": a in anc[b] or b in anc[a]}) + total = sum(duration.values()) + return {"features": len(ids), "edges": sum(len(v) for v in deps.values()), "durations": duration, + "topological_order": topo, "topo_level": level, "earliest_start": est, "bottom_level": bottom, "slack": slack, + "critical_path": path, "critical_path_length": length, "total_work": total, + "work_critical_ratio": round(total / length, 3) if length else 0.0, + "width": len(antichain), "max_antichain": antichain, "conflicts": conflicts, + "descendants": {fid: sum(1 for g in ids if fid in anc[g]) for fid in ids}} + + +def _topological(ids, deps, children, order) -> list[str]: + indegree = {n: len(deps[n]) for n in ids} + ready = sorted([n for n in ids if indegree[n] == 0], key=order.get) + out = [] + while ready: + node = ready.pop(0) + out.append(node) + for child in children[node]: + indegree[child] -= 1 + if indegree[child] == 0: + ready.append(child) + ready.sort(key=order.get) + return out + + +def _max_antichain(ids: list[str], anc: dict[str, set[str]]) -> list[str]: + """Dilworth: width = n - |maximum matching| over the strict order a < b; Koenig recovers an antichain.""" + adjacent = {a: [b for b in ids if a in anc[b]] for a in ids} + match_right: dict[str, str] = {} + + def augment(u, seen): + for v in adjacent[u]: + if v in seen: + continue + seen.add(v) + if v not in match_right or augment(match_right[v], seen): + match_right[v] = u + return True + return False + + for u in ids: + augment(u, set()) + match_left = {u: v for v, u in match_right.items()} + left_reach, right_reach = set(u for u in ids if u not in match_left), set() + stack = list(left_reach) + while stack: + u = stack.pop() + for v in adjacent[u]: + if v not in right_reach: + right_reach.add(v) + w = match_right.get(v) + if w is not None and w not in left_reach: + left_reach.add(w) + stack.append(w) + cover_left, cover_right = set(ids) - left_reach, right_reach + antichain = [x for x in ids if x not in cover_left and x not in cover_right] + if len(antichain) != len(ids) - len(match_right): + raise Rejected("taskgraph_internal", "Koenig construction mismatch") + return antichain + + +# -- risk and the verification plan --------------------------------------------------------------------------------- + +def risk_inputs(item: dict, graph: dict) -> dict: + """The seven standardized inputs in [0, 1], all computable by code from the map and the graph.""" + scope = item["scope"] + n = graph["features"] + criteria = item["criteria"] + mechanical = sum(1 for c in criteria if c["kind"] == "test") + return {"scope": round(0.5 * scope["max_files"] / MAX_FILES_CEILING + 0.5 * scope["max_diff_bytes"] / MAX_DIFF_BYTES_CEILING, 4), + "act_class": ACT_RISK[item["act_class"]], + "new_deps": min(1.0, item["risk_inputs"]["new_dependencies"] / 2), + "needs_lookup": 1.0 if item["risk_inputs"]["needs_lookup"] else 0.0, + "disagreement": float(item["risk_inputs"]["disagreement"]), + "centrality": round(graph["descendants"][item["id"]] / max(1, n - 1), 4), + "uncovered": round(1 - mechanical / len(criteria), 4) if criteria else 1.0} + + +def risk_score(inputs: dict, weights: dict = WEIGHTS) -> float: + return round(sum(weights[k] * inputs[k] for k in weights), 4) + + +def softmax(values: list[float], tau: float) -> list[float]: + top = max(values) + exps = [math.exp((v - top) / tau) for v in values] + total = sum(exps) + return [e / total for e in exps] + + +def integer_units(shares: list[float], budget: int) -> list[int]: + """Largest-remainder rounding of ``budget * shares`` to integers that sum to ``budget``.""" + raw = [budget * s for s in shares] + base = [math.floor(x) for x in raw] + remainder = budget - sum(base) + order = sorted(range(len(shares)), key=lambda i: (-(raw[i] - base[i]), i)) + for i in order[:remainder]: + base[i] += 1 + return base + + +def ladder(units: int) -> list[str]: + rungs = list(LADDER[: min(units, len(LADDER))]) + if units > len(LADDER): + rungs.append(f"extra_seeds_x{2 * (units - len(LADDER))}") + return rungs + + +def risk_plan(document: dict, graph: dict | None = None, *, budget: int | None = None, tau: float = 1.0, + weights: dict = WEIGHTS) -> dict: + """Standardized risk scores and the softmax allocation of ``budget`` extra verification units above the floor. + + ``z_i = (r_i - mean) / sd`` so tau is in standard deviations of this map's spread (a map whose raw scores + spread over 0.14 would otherwise be allocated uniformly at any stated tau); with one feature or zero + spread the allocation is uniform. ``share`` is the propensity each unit had of landing on a feature and is + recorded with the plan, so later calibration can reweight. The floor is listed beside every allocation and is + the same for every feature: it is what a low score can never buy its way out of.""" + graph = graph or taskgraph(document) + if tau < TAU_MIN: + raise Rejected("risk_tau", f"tau must be at least {TAU_MIN} so every feature keeps a positive propensity") + if not isinstance(weights, dict) or set(weights) != set(WEIGHTS): + raise Rejected("risk_weights", f"weights must name exactly the inputs {sorted(WEIGHTS)}") + if any(isinstance(w, bool) or not isinstance(w, (int, float)) or not math.isfinite(w) or not 0 <= w <= 1 for w in weights.values()): + raise Rejected("risk_weights", "each weight is a finite number in [0, 1]") + if abs(sum(weights.values()) - 1.0) > 1e-6: + raise Rejected("risk_weights", f"weights sum to {sum(weights.values()):.4f}, not 1, so r would leave [0, 1]") + features = document["features"] + ids = [f["id"] for f in features] + budget = len(ids) if budget is None else budget + if type(budget) is not int or budget < 0: + raise Rejected("risk_budget", "the verification budget is a non-negative integer number of units") + inputs = {f["id"]: risk_inputs(f, graph) for f in features} + scores = [risk_score(inputs[fid], weights) for fid in ids] + mean = sum(scores) / len(scores) + spread = math.sqrt(sum((s - mean) ** 2 for s in scores) / len(scores)) + z = [((s - mean) / spread) if spread > 0 else 0.0 for s in scores] + shares = softmax(z, tau) + units = integer_units(shares, budget) + return {"schema": "syberfactory/verification-plan.v1", "map_digest": map_digest(document), "weights": dict(weights), + "tau": tau, "budget": budget, "standardized": spread > 0, + "inputs": inputs, "r": dict(zip(ids, scores)), "z": {fid: round(v, 4) for fid, v in zip(ids, z)}, + "share": {fid: round(s, 4) for fid, s in zip(ids, shares)}, "units": dict(zip(ids, units)), + "floor": list(FLOOR), "buys": {fid: list(FLOOR) + ladder(u) for fid, u in zip(ids, units)}} + + +# -- compile: one feature -> one ticket (D5) ------------------------------------------------------------------------- + +def compile_ticket(document: dict, item: dict, spec: dict, *, human_owner: str, baseline_run: str | None = None, + space: dict | None = None) -> dict: + """The work order of one feature in ``ticket_from_spec``'s shape. ``human_owner`` is mandatory: agents are + delegated to, a person stays accountable. ``rubric_digest`` covers the hidden-check names the host holds; + the checks' content is never in the ticket, which a builder may read. With a Space graph the footprint's + components are the owners of the scope's files; without one no component is claimed.""" + if not isinstance(human_owner, str) or not human_owner.strip(): + raise Rejected("ticket_owner", "a ticket names the accountable human; none given") + requirements = {r["req_id"]: r for r in spec["requirements"]} + kinds: list[str] = [] + for rid in item["req_ids"]: + for kind in requirements[rid]["evidence_kinds"]: + if kind not in kinds: + kinds.append(kind) + components = sorted({record["component"] for path, record in (space or {}).get("files", {}).items() + if record.get("component") and any(path_intersects(p, path) for p in item["scope"]["paths"])}) + sha = map_digest(document) + return { + "protocol": PROTOCOL, "kind": "ticket", + "ticket_id": digest({"map": sha, "feature": item["id"]})[:16], + "title": item["title"][:120], + "spec_ref": {"spec_id": spec["spec_id"], "version": spec["version"], "req_ids": list(item["req_ids"])}, + "map_ref": {"map_id": document["id"], "map_digest": sha, "feature_id": item["id"]}, + "objective": item["objective"], + "acceptance_criteria": [dict(c) for c in item["criteria"]], + "footprint": {"components": components, "files": sorted(item["scope"]["paths"]), + "revision": document["base"], "space_digest": None}, + "scope": {k: (list(v) if isinstance(v, list) else v) for k, v in item["scope"].items()}, + "depends_on": list(item["depends_on"]), "act_class": item["act_class"], "size": item["size"], + "repro": {"baseline_run": baseline_run}, + "exit_criteria": [f"evidence:{kind}" for kind in kinds] + [f"hidden:{name}" for name in item["hidden_checks"]], + "rubric_digest": digest(sorted(item["hidden_checks"])) if item["hidden_checks"] else None, + "delegate": None, "human_owner": human_owner.strip(), + "state": "open", "candidates": [], "verdicts": [], "evidence_links": [], + } + + +def compile_tickets(document: dict, spec: dict, *, human_owner: str, baseline_run: str | None = None, + space: dict | None = None) -> list[dict]: + """Every feature, in map order. Refuses a map that fails ``mapcheck``.""" + findings = mapcheck(document, spec) + if findings: + raise Rejected("map_invalid", "; ".join(findings)) + return [compile_ticket(document, item, spec, human_owner=human_owner, baseline_run=baseline_run, space=space) + for item in document["features"]] diff --git a/syberlabs/factory/loop.py b/syberlabs/factory/loop.py new file mode 100644 index 0000000..b94379f --- /dev/null +++ b/syberlabs/factory/loop.py @@ -0,0 +1,725 @@ +"""The thin-slice driver (AM1): an approved feature map, compiled to tickets, built by Build Threads, stopped by an envelope. + + loop = Loop(repo_root=".", kit_home=".syberlabs", ledger=Ledger(".syberlabs/factory"), map=document, spec=spec, + envelope=envelope, policy=OfflinePolicy(), proposer_argv=[python, "standin.py", "."], + checks={"tests": {"argv": [python, "-m", "unittest", "-q"], "timeout_seconds": 300}}, + hidden_catalogue={"hidden.f1.marker": [python, "-c", "..."]}, human_owner="mateo") + loop.compile() # refuses a map without a current owner approval; tickets go on the ledger + summary = loop.run() # features end "ready for owner merge", parked, escalated or waiting; nothing merges + +What it does, in order, every step: reads the kill switch; computes the eligible features from the map and +the record (dependencies ready, declared overlap order respected, act class inside the envelope, attempts +left, room in the envelope); asks the policy ``next_feature``, ``proposer_arm`` and ``budget_tier`` (each +answer is recorded on the factory ledger as a proposal before anything happens); reserves the tier on the +envelope ledger; starts one unattended Build Thread on a contract compiled from the feature's ticket (its +scope is the feature's scope, its checks are the repository's visible checks plus the hidden checks the +host holds under the names the ticket lists); runs the proposer command once through ``CommandProvider``; +then drives the thread with ``syberlabs run``'s own ``Runner`` (consolidation D17: recover, check and +accept are the kernel's, never re-implemented here) until the pass is idle; settles the envelope from the +thread's journal; and records the outcome. A failed attempt asks ``worth_continuing`` and ``after_failure``. + +What it never does: merge, push, open a pull request, edit the map, the spec, the envelope or a contract +after compile, call a model or a network. The terminal state of a feature is ``ready_for_owner_merge``: an +accepted candidate on ``syberlabs/``, with the thread id and commit on the ledger for the owner. + +Events go on #154's ledger. Its event vocabulary is owned by ``ledger.py`` (and by the goal-gate packet +that is editing it), so this slice writes its events as ``note`` with ``body.note`` naming the autonomy +event and ``body.refs`` holding the values that differ between two runs of the same fixture (thread ids, +commits, seconds). ``event_digests`` digests every autonomy event without ``refs``, so two runs of one +fixture give one list of digests. Tickets use the existing ``ticket_opened`` kind with ``source: map``. + +The envelope ledger (D13) is the eval spend ledger's shape (one JSON object per line, appended before the +work and settled after it, read strictly) generalized to attempts: ``reserve`` before an attempt holds the +tier's dollars and one call, ``settle`` replaces the hold with what the journal recorded, ``stop`` closes +the run with a reason. Totals are the settled values plus unsettled holds, so a killed attempt stays counted. +One file per envelope (``envelopes/.jsonl`` under the factory home), and every line names +the envelope it belongs to, so two maps or two grants in one home never read each other's spend or stop. + +The spec is promoted when the ledger says so, not when the file says so: ``promotion_record`` finds the +``spec_changed`` that promoted exactly this spec (its digest), the ``approval_recorded`` it cites and the +owner's envelope under ``approvals/``, and re-verifies the signature under the owner key the ledger pinned +(``owner_key_set``, read through the ledger's verified lineage with its stored rotations; the goal gate's +``verify_approval``). ``compile`` refuses a spec the ledger did not promote (``spec_unpromoted``), a ledger +whose owner-key lineage does not verify (``owner_key_lineage``: a bare second pin, an unsigned or stale +rotation, a malformed key), a ledger without a pinned key (``owner_key_unset``) and an approval under +another key (``owner_key_mismatch``). +""" + +from __future__ import annotations + +import json +import math +import time +from pathlib import Path + +from syberlabs.build import ACT_CLASSES, PROMOTE, Kit, Runner +from syberlabs.canonical import canonical, digest +from syberlabs.errors import Rejected +from syberlabs.factory import PROTOCOL, featuremap, policy as policymod +from syberlabs.factory import spec as specmod +from syberlabs.providers import CommandProvider + +ENVELOPE_SCHEMA = "syberfactory/envelope.v1" +ENVELOPE_LINE_SCHEMA = "syberlabs.factory/envelope-ledger.v1" +ENVELOPE_DIR = "envelopes" +ENVELOPE_LINE_KINDS = ("reserve", "settle", "stop") +ACTIONS = ("start_thread", "propose", "check", "accept_to_thread_branch") # closed; no merge, push or publish +EVENTS = ("map_frozen", "policy_proposal", "envelope_reserved", "envelope_settled", "attempt_started", "thread_step", + "feature_ready", "feature_failed", "feature_parked", "feature_escalated", "feature_waiting", "loop_stopped") +STATES = ("open", "building", "ready_for_owner_merge", "parked", "escalated", "waiting_owner_merge") +MAX_PASSES = 8 # Runner passes per attempt: check, accept, idle; more only when evidence ages or recovery runs + + +# -- the envelope ---------------------------------------------------------------------------------------------------- + +def new_envelope(map_digest: str, *, dollars_total: float, calls_total: int, seconds_total: float, + attempts_per_feature: int = 2, tiers: dict | None = None, arms: list[dict] | None = None, + act_classes: tuple = ("R",), max_candidates: int = 3, kill_switch: str | None = None) -> dict: + """The owner's grant: what the loop may spend, which classes it may build, which arms it may run.""" + tiers = tiers or {"S": 0.0, "M": 0.0, "L": 0.0} + arms = arms or [{"id": "standin", "tier_rank": 0, "revision": "standin"}] + if set(tiers) != set(policymod.TIERS) or any(not isinstance(v, (int, float)) or v < 0 for v in tiers.values()): + raise Rejected("envelope_invalid", "tiers must price S, M and L with non-negative dollars") + if not arms or any(not isinstance(a, dict) or not a.get("id") for a in arms): + raise Rejected("envelope_invalid", "arms must be a non-empty list of {id, tier_rank, revision}") + if any(c not in ACT_CLASSES for c in act_classes): + raise Rejected("envelope_invalid", f"act_classes must be from {ACT_CLASSES}") + for name, value in (("dollars_total", dollars_total), ("seconds_total", seconds_total)): + if not isinstance(value, (int, float)) or isinstance(value, bool) or not math.isfinite(value) or value < 0: + raise Rejected("envelope_invalid", f"{name} must be a finite number of at least 0") + if type(calls_total) is not int or calls_total < 0 or type(attempts_per_feature) is not int or attempts_per_feature < 1: + raise Rejected("envelope_invalid", "calls_total is an int of at least 0 and attempts_per_feature an int of at least 1") + return {"schema": ENVELOPE_SCHEMA, "map_digest": map_digest, + "dollars": {"total": float(dollars_total), "tiers": {k: float(tiers[k]) for k in policymod.TIERS}}, + "calls": {"total": calls_total}, "seconds": {"total": float(seconds_total)}, + "attempts_per_feature": attempts_per_feature, "concurrency": 1, "act_classes": sorted(act_classes), + "arms": [dict(a) for a in arms], "actions": list(ACTIONS), "max_candidates": max_candidates, + "kill_switch": kill_switch} + + +def envelope_digest(envelope: dict) -> str: + """The grant's digest. The kill switch is a path on this host, not a term of the grant, so it stays out: + two runs of one fixture with and without a switch give one ``map_frozen`` digest.""" + return digest({k: v for k, v in envelope.items() if k != "kill_switch"}) + + +def envelope_path(home: str | Path, envelope: dict) -> Path: + return Path(home) / ENVELOPE_DIR / f"{envelope_digest(envelope)}.jsonl" + + +def _amount(value, name: str, where: str, *, integer: bool = False) -> float: + """A finite, non-negative number (an int when ``integer``); anything else is refused, never summed.""" + if isinstance(value, bool) or not isinstance(value, (int, float)) or not math.isfinite(value) or value < 0: + raise Rejected("envelope_unpriced", f"{where}: {name} must be a finite number of at least 0, not {value!r}") + if integer and type(value) is not int: + raise Rejected("envelope_unpriced", f"{where}: {name} must be an integer, not {value!r}") + return value + + +def _strict_loads(text: str): + def no_repeats(pairs): + out = {} + for key, value in pairs: + if key in out: + raise ValueError(f"repeated key {key!r}") + out[key] = value + return out + + def no_constant(name): + raise ValueError(f"non-finite number {name}") + + return json.loads(text, object_pairs_hook=no_repeats, parse_constant=no_constant) + + +class EnvelopeLedger: + """Append-only lines for one envelope under the factory home. Reserve, settle, stop; totals from the lines. + + Read strictly, as the eval spend ledger is: a torn line, a repeated key, a non-finite number, a line of + another schema or another envelope, a line without a kind, a negative or non-finite amount, a second + ``reserve`` for one attempt or a ``settle`` without its ``reserve`` each refuse the whole file rather than + change a total. The loop writes none of these; a hand-edited or corrupted file does.""" + + def __init__(self, path: str | Path, envelope: dict): + self.path, self.envelope = Path(path), envelope + self.digest = envelope_digest(envelope) + self.path.parent.mkdir(parents=True, exist_ok=True) + + def _check(self, line: dict, where: str) -> None: + if line.get("envelope_digest") != self.digest: + raise Rejected("envelope_unreadable", f"{where} belongs to envelope {str(line.get('envelope_digest'))[:12]}, not {self.digest[:12]}") + kind = line.get("kind") + if kind not in ENVELOPE_LINE_KINDS: + raise Rejected("envelope_unreadable", f"{where}: kind must be one of {ENVELOPE_LINE_KINDS}, not {kind!r}") + if kind == "stop": + if not isinstance(line.get("reason"), str) or not line["reason"]: + raise Rejected("envelope_unreadable", f"{where}: a stop names its reason") + return + if not isinstance(line.get("feature"), str) or not line["feature"] or type(line.get("attempt")) is not int or line["attempt"] < 1: + raise Rejected("envelope_unreadable", f"{where}: a {kind} names a feature and an attempt number of at least 1") + _amount(line.get("dollars"), "dollars", where) + _amount(line.get("calls"), "calls", where, integer=True) + _amount(line.get("seconds"), "seconds", where) + + def lines(self) -> list[dict]: + if not self.path.is_file(): + return [] + out = [] + for number, raw in enumerate(self.path.read_text(encoding="utf-8").splitlines(), 1): + if not raw.strip(): + continue + where = f"{self.path}: line {number}" + try: + value = _strict_loads(raw) + except ValueError as exc: + raise Rejected("envelope_unreadable", f"{where}: {exc}") from None + if not isinstance(value, dict) or value.get("schema") != ENVELOPE_LINE_SCHEMA: + raise Rejected("envelope_unreadable", f"{where} is not an envelope ledger line") + self._check(value, where) + out.append(value) + return out + + def _append(self, line: dict) -> dict: + line = {"schema": ENVELOPE_LINE_SCHEMA, "envelope_digest": self.digest, **line} + self._check(line, f"{self.path}: new {line.get('kind')} line") + with self.path.open("a", encoding="utf-8", newline="\n") as handle: + handle.write(canonical(line) + "\n") + handle.flush() + return line + + def _fold(self) -> tuple[dict, dict, str | None]: + reserved: dict[tuple, dict] = {} + settled: dict[tuple, dict] = {} + stopped = None + for number, line in enumerate(self.lines(), 1): + key = (line.get("feature"), line.get("attempt")) + if line["kind"] == "reserve": + if key in reserved: + raise Rejected("envelope_conflict", f"{self.path}: line {number} reserves {key[0]} attempt {key[1]} a second time") + reserved[key] = line + elif line["kind"] == "settle": + if key not in reserved: + raise Rejected("envelope_conflict", f"{self.path}: line {number} settles {key[0]} attempt {key[1]}, which was never reserved") + if key in settled: + raise Rejected("envelope_conflict", f"{self.path}: line {number} settles {key[0]} attempt {key[1]} a second time") + settled[key] = line + else: + stopped = line["reason"] + return reserved, settled, stopped + + def totals(self) -> dict: + reserved, settled, stopped = self._fold() + dollars = calls = seconds = 0.0 + for key, hold in reserved.items(): + actual = settled.get(key, hold) + dollars += float(actual["dollars"]) + calls += int(actual["calls"]) + seconds += float(actual["seconds"]) + return {"dollars": round(dollars, 6), "calls": int(calls), "seconds": round(seconds, 3), "attempts": len(reserved), + "unsettled": len(set(reserved) - set(settled)), "stopped": stopped, + "dollars_left": round(self.envelope["dollars"]["total"] - dollars, 6), + "calls_left": self.envelope["calls"]["total"] - int(calls), + "seconds_left": round(self.envelope["seconds"]["total"] - seconds, 3)} + + def room(self, tier: str, calls: int = 1) -> bool: + totals = self.totals() + return (totals["stopped"] is None and totals["dollars_left"] >= self.envelope["dollars"]["tiers"][tier] + and totals["calls_left"] >= calls and totals["seconds_left"] > 0) + + def reserve(self, feature: str, attempt: int, tier: str, arm: str) -> dict: + reserved, _, _ = self._fold() + if (feature, attempt) in reserved: + raise Rejected("envelope_conflict", f"{feature} attempt {attempt} is already reserved") + if not self.room(tier): + raise Rejected("envelope_exhausted", f"no room for tier {tier} on {feature}: {self.totals()}") + return self._append({"kind": "reserve", "feature": feature, "attempt": attempt, "tier": tier, "arm": arm, + "dollars": self.envelope["dollars"]["tiers"][tier], "calls": 1, "seconds": 0.0}) + + def settle(self, feature: str, attempt: int, *, dollars: float, calls: int, seconds: float, unknown_models: list) -> dict: + where = f"{feature} attempt {attempt}" + _amount(dollars, "dollars", where) + _amount(calls, "calls", where, integer=True) + _amount(seconds, "seconds", where) + reserved, settled, _ = self._fold() + if (feature, attempt) not in reserved: + raise Rejected("envelope_conflict", f"{where} was never reserved; nothing to settle") + if (feature, attempt) in settled: + raise Rejected("envelope_conflict", f"{where} is already settled") + return self._append({"kind": "settle", "feature": feature, "attempt": attempt, "dollars": float(dollars), "calls": calls, + "seconds": round(float(seconds), 3), "unknown_models": list(unknown_models)}) + + def stop(self, reason: str) -> dict: + return self._append({"kind": "stop", "reason": reason}) + + +# -- ledger events ----------------------------------------------------------------------------------------------------- + +def record(ledger, note: str, map_digest: str, body: dict, refs: dict | None = None) -> dict: + if note not in EVENTS: + raise Rejected("autonomy_event", f"unknown autonomy event {note!r}") + return ledger.append("note", {"note": note, "map_digest": map_digest, **body, "refs": dict(refs or {})}) + + +def autonomy_events(ledger, map_digest: str | None = None) -> list[dict]: + """The slice's events in ledger order: ``note`` events it wrote and ``ticket_opened`` with ``source: map``.""" + out = [] + for event in ledger.events(): + body = event["body"] + if event["kind"] == "note" and body.get("note") in EVENTS or event["kind"] == "ticket_opened" and body.get("source") == "map": + if map_digest is None or body.get("map_digest") == map_digest: + out.append(event) + return out + + +def event_digests(ledger, map_digest: str | None = None) -> list[str]: + """One digest per autonomy event over its kind and body without ``refs``: the same fixture gives the same list.""" + return [digest({"kind": e["kind"], "body": {k: v for k, v in e["body"].items() if k != "refs"}}) + for e in autonomy_events(ledger, map_digest)] + + +def promotion_record(ledger, spec: dict) -> dict: + """The ledger's record that the owner promoted exactly this spec, through the goal gate's own shape. + + The spec on disk must be the spec the ledger wrote after a ``promote`` (or a later ``canonize``): the + ``spec_changed`` whose ``spec_digest`` is this file's digest, with ``to`` promoted or canonical. The + promotion's ``approval`` must cite an ``approval_recorded`` event (by hash) for this spec and version by a + human approver, whose envelope is under ``approvals/`` at the recorded digest and was signed under the owner + key the ledger pinned at that point (``owner_key_set``, read with ``spec.pinned_owner_key`` through the + ledger's verified lineage, its stored rotations supplied by ``ledger.load_rotation``; a later rotation does + not unmake an earlier approval). The whole ledger's lineage must verify first: a pin appended after the + approval that is not a signed rotation leaves the ledger pinning nothing (``owner_key_lineage`` refuses, + never ``owner_key_unset`` and never an earlier key). The envelope is re-verified under that key + (``spec.verify_approval`` over the digest the owner signed), so the record is a credential, not a string. + The file's ``state`` string decides nothing here.""" + events = ledger.events() + changes = [e for e in events if e["kind"] == "spec_changed" and e["body"].get("spec_id") == spec.get("spec_id")] + current = specmod.spec_digest(spec) + written = next((e for e in reversed(changes) if e["body"].get("spec_digest") == current), None) + if written is None or written["body"].get("to") not in ("promoted", "canonical"): + raise Rejected("spec_unpromoted", f"the ledger holds no promotion of spec {spec.get('spec_id')!r} at digest {current[:12]}; " + "the file was edited after promotion, or was never promoted through `factory spec promote`") + version = written["body"].get("version") + promotion = next((e for e in reversed(changes[: changes.index(written) + 1]) if e["body"].get("operation") == "promote" + and e["body"].get("to") == "promoted" and e["body"].get("version") == version), None) + if promotion is None: + raise Rejected("spec_unpromoted", f"spec {spec['spec_id']} v{version} has no promote event before its current record") + cited = promotion["body"].get("approval") + actor = promotion["body"].get("actor") + if not isinstance(actor, dict) or actor.get("kind") != "human" or not isinstance(cited, dict) or not cited.get("event_hash"): + raise Rejected("spec_unpromoted", f"the promotion of {spec['spec_id']} at seq {promotion['seq']} names no human actor or no approval event") + recorded = next((e for e in events if e["kind"] == "approval_recorded" and e["hash"] == cited["event_hash"]), None) + if recorded is None: + raise Rejected("spec_unpromoted", f"the promotion cites approval event {str(cited['event_hash'])[:12]}, which is not on the ledger") + body = recorded["body"] + approver = body.get("approver") or {} + if (body.get("operation") != "promote" or body.get("spec_id") != spec["spec_id"] or body.get("version") != version + or approver.get("kind") != "human" or not approver.get("id") or body.get("approval_digest") != cited.get("approval_digest") + or approver != actor): + raise Rejected("spec_unpromoted", f"approval event {recorded['seq']} does not cover the promotion of {spec['spec_id']} v{version} by {actor.get('id')!r}") + path = Path(ledger.home) / "approvals" / f"{body['approval_digest']}.json" + try: + envelope = _strict_loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError) as exc: + raise Rejected("spec_unpromoted", f"the approval envelope {body['approval_digest'][:12]} is missing or unreadable: {type(exc).__name__}") from None + if digest(envelope) != body["approval_digest"]: + raise Rejected("spec_unpromoted", f"approvals/{body['approval_digest'][:12]} does not match its recorded digest") + specmod.pinned_owner_key(events, ledger.load_rotation) # the whole lineage verifies, or the ledger pins nothing + pinned = specmod.pinned_owner_key([e for e in events if e["seq"] < recorded["seq"]], ledger.load_rotation) + if pinned is None: + raise Rejected("owner_key_unset", f"the ledger pinned no owner key before approval event {recorded['seq']}; the approval rests on no trust root") + if body.get("keyid") != pinned: + raise Rejected("owner_key_mismatch", f"approval event {recorded['seq']} is under key {str(body.get('keyid'))[:10]}, not the owner key " + f"the ledger pinned then ({pinned[:10]})") + specmod.verify_approval(envelope, bytes.fromhex(pinned), spec, "promote", verdict_ids=list(body.get("verdict_ids") or []), + spec_digest_value=body.get("spec_digest")) + return {"seq": promotion["seq"], "approval_seq": recorded["seq"], "approver": dict(approver), "keyid": pinned, + "verdict_ids": list(body.get("verdict_ids") or []), "approval_digest": body["approval_digest"], + "signed_digest": body.get("spec_digest")} + + +def why(ledger, ticket_id: str) -> dict: + """The §1.3 walk as far as the ledger goes: ticket -> feature -> map (its approval) -> spec (its promotion).""" + opened = next((e for e in ledger.events() if e["kind"] == "ticket_opened" and e["body"].get("ticket_id") == ticket_id), None) + if opened is None: + raise Rejected("ticket_unknown", f"no ticket_opened event for {ticket_id}") + body = opened["body"] + frozen = next((e for e in ledger.events() if e["kind"] == "note" and e["body"].get("note") == "map_frozen" + and e["body"].get("map_digest") == body.get("map_digest")), None) + ready = next((e for e in ledger.events() if e["kind"] == "note" and e["body"].get("note") == "feature_ready" + and e["body"].get("ticket_id") == ticket_id), None) + frozen_body = (frozen or {}).get("body", {}) + return {"ticket": ticket_id, "feature": body.get("feature_id"), "requirements": body.get("req_ids"), + "map": {"digest": body.get("map_digest"), "frozen_seq": frozen["seq"] if frozen else None, + "approved_by": frozen_body.get("approved_by")}, + "spec": {"spec_id": body.get("spec_id"), "version": body.get("version"), "digest": frozen_body.get("spec_digest"), + "promoted_by": frozen_body.get("promoted_by"), "promotion": frozen_body.get("refs", {}).get("promotion")}, + "ready_for_owner_merge": {"seq": ready["seq"], **ready["body"]["refs"]} if ready else None} + + +# -- contracts ----------------------------------------------------------------------------------------------------------- + +def feature_contract(ticket: dict, *, checks: dict, hidden_catalogue: dict, max_candidates: int) -> dict: + """One ``repo-change``-shaped contract per ticket: scope from the feature, the visible checks plus the hidden + checks the host holds under the ticket's names, every check required. The id carries the ticket id so a + changed map never collides with a published contract (contracts are immutable once installed).""" + found = {name: {"argv": list(c["argv"]), "timeout_seconds": int(c.get("timeout_seconds", 300))} for name, c in checks.items()} + for exit_criterion in ticket["exit_criteria"]: + if not exit_criterion.startswith("hidden:"): + continue + name = exit_criterion[len("hidden:"):] + argv = hidden_catalogue.get(name) + if not argv: + raise Rejected("hidden_check_unknown", f"ticket {ticket['ticket_id']} names hidden check {name!r} the host does not hold") + if name in found: + raise Rejected("hidden_check_collision", f"hidden check {name!r} has the name of a visible check") + found[name] = {"argv": list(argv), "timeout_seconds": 120} + if not found: + raise Rejected("contract_unchecked", "a feature contract needs at least one check") + return {"id": f"syberfactory-{ticket['ticket_id']}", "version": 1, "title": ticket["title"], + "inputs": {"objective": "string", "act_class": "string"}, + "actions": {PROMOTE: {}}, "acceptance": [{"id": "accepted", "kind": "effect", "action": PROMOTE}], + "evolution": {"scope": dict(ticket["scope"]), "operators": ["patch"], + "budget": {"max_candidates": max_candidates, "max_evaluations": 2 * max_candidates, "max_seconds": 900}, + "evaluation": {"checks": found, "required": sorted(found), "max_age_seconds": 3600}, + "promotion": {"action": PROMOTE, "roles": ["developer"]}}} + + +# -- the loop ------------------------------------------------------------------------------------------------------------- + +class Loop: + def __init__(self, *, repo_root: str | Path, kit_home: str | Path, ledger, map: dict, spec: dict, envelope: dict, + policy, proposer_argv: list[str], checks: dict, hidden_catalogue: dict | None = None, human_owner: str, + baseline_run: str | None = None, actor: str = "factory@syberlabs.host", log=None, max_passes: int = MAX_PASSES): + self.repo_root, self.kit_home, self.ledger = Path(repo_root), Path(kit_home), ledger + self.map, self.spec, self.envelope, self.policy = map, spec, envelope, policy + self.proposer_argv, self.checks, self.hidden = list(proposer_argv), dict(checks), dict(hidden_catalogue or {}) + self.human_owner, self.baseline_run, self.actor = human_owner, baseline_run, actor + self.log = log or (lambda line: None) + self.max_passes = max_passes + self.map_digest = featuremap.map_digest(map) + if envelope.get("map_digest") != self.map_digest: + raise Rejected("envelope_map", "the envelope names a different map digest") + self.spend = EnvelopeLedger(envelope_path(self.ledger.home, envelope), envelope) + self.graph = None + self.plan = None + self.promotion = None + self.tickets: dict[str, dict] = {} + self.features = {f["id"]: f for f in map["features"]} + self.state = {fid: {"state": "open", "attempts": 0, "dollars": 0.0, "thread": None, "commit": None, "last_reason": None, + "last_arm": None, "arm_floor": 0} + for fid in self.features} + self.kit = None + self.order: list[str] = [] + + # -- compile ----------------------------------------------------------------------------------------------------------- + + def compile(self) -> list[dict]: + """Refuse a map without a current owner approval, a spec the ledger did not promote, or a map with + findings; write one contract per feature; log ``map_frozen`` and one ``ticket_opened`` per feature. + Idempotent for one map digest.""" + if not featuremap.approved(self.map): + raise Rejected("map_unapproved", "the loop runs only an owner-approved map whose approval covers its current digest") + if self.spec.get("state") not in ("promoted", "canonical"): + raise Rejected("spec_state", f"work opens under a promoted spec, not one in state {self.spec.get('state')!r}") + self.promotion = promotion_record(self.ledger, self.spec) + findings = featuremap.mapcheck(self.map, self.spec) + if findings: + raise Rejected("map_invalid", "; ".join(findings)) + self.graph = featuremap.taskgraph(self.map) + self.plan = featuremap.risk_plan(self.map, self.graph) + tickets = featuremap.compile_tickets(self.map, self.spec, human_owner=self.human_owner, baseline_run=self.baseline_run, + space=self.ledger.latest_space()) + contracts = self.kit_home / "contracts" + contracts.mkdir(parents=True, exist_ok=True) + for ticket in tickets: + contract = feature_contract(ticket, checks=self.checks, hidden_catalogue=self.hidden, + max_candidates=self.envelope["max_candidates"]) + path = contracts / f"{contract['id']}.v{contract['version']}.json" + if not path.exists(): + path.write_text(json.dumps(contract, indent=2, sort_keys=True) + "\n", encoding="utf-8") + self.tickets[ticket["map_ref"]["feature_id"]] = ticket + already = any(e["body"].get("note") == "map_frozen" and e["body"].get("map_digest") == self.map_digest + for e in self.ledger.events() if e["kind"] == "note") + if not already: + record(self.ledger, "map_frozen", self.map_digest, { + "map_id": self.map["id"], "spec_id": self.spec["spec_id"], "spec_version": self.spec["version"], + "spec_digest": self.map["spec_ref"]["spec_digest"], "approved_by": self.map["approval"]["actor"], + "promoted_by": self.promotion["approver"], + "envelope_digest": envelope_digest(self.envelope), "features": [f["id"] for f in self.map["features"]], + "critical_path": self.graph["critical_path"], "critical_path_length": self.graph["critical_path_length"], + "width": self.graph["width"], "work_critical_ratio": self.graph["work_critical_ratio"], + "risk": self.plan["r"], "verification_units": self.plan["units"], "tau": self.plan["tau"], "floor": self.plan["floor"]}, + refs={"promotion": {k: self.promotion[k] for k in ("seq", "approval_seq", "approval_digest", "keyid", "verdict_ids")}}) + for ticket in tickets: + self.ledger.append("ticket_opened", { + "ticket_id": ticket["ticket_id"], "spec_id": self.spec["spec_id"], "version": self.spec["version"], + "req_id": ticket["spec_ref"]["req_ids"][0], "req_ids": ticket["spec_ref"]["req_ids"], "source": "map", + "map_digest": self.map_digest, "feature_id": ticket["map_ref"]["feature_id"], + "footprint": ticket["footprint"]["components"], "files": ticket["footprint"]["files"], + "revision": self.map["base"], "rubric_digest": ticket["rubric_digest"], "human_owner": ticket["human_owner"]}) + return tickets + + # -- state and eligibility -------------------------------------------------------------------------------------------- + + def _ready(self, fid: str) -> bool: + return self.state[fid]["state"] == "ready_for_owner_merge" + + def eligible(self) -> list[str]: + out = [] + for fid, item in self.features.items(): + row = self.state[fid] + if row["state"] != "open": + continue + if item["act_class"] not in self.envelope["act_classes"]: + row["state"] = "escalated" + record(self.ledger, "feature_escalated", self.map_digest, {"feature": fid, "reason": f"act_class_{item['act_class']}_outside_envelope"}) + continue + if not all(self._ready(d) for d in item["depends_on"]): + continue + if item["depends_on"] and self.map["integration"] == "await_merge": + row["state"] = "waiting_owner_merge" + record(self.ledger, "feature_waiting", self.map_digest, {"feature": fid, "waits_for": list(item["depends_on"])}) + continue + if any(o["order"] == f"{o['with']}_first" and not self._ready(o["with"]) and self.state[o["with"]]["state"] == "open" + for o in item["overlaps"]): + continue # the declared order says the other feature's scope goes first + if row["attempts"] >= self.envelope["attempts_per_feature"]: + continue + out.append(fid) + return out + + def policy_state(self) -> dict: + """Ids and numbers only.""" + totals = self.spend.totals() + tiers = self.envelope["dollars"]["tiers"] + return {"features": {fid: {"bottom_level": self.graph["bottom_level"][fid], "slack": self.graph["slack"][fid], + "topo_level": self.graph["topo_level"][fid], "risk": self.plan["r"][fid], + "size_units": featuremap.DURATION[self.features[fid]["size"]], + "act_rank": featuremap.ACT_RANK[self.features[fid]["act_class"]], + "attempts": self.state[fid]["attempts"], + "attempts_left": self.envelope["attempts_per_feature"] - self.state[fid]["attempts"], + "dollars_used": self.state[fid]["dollars"], "map_order": self.order.index(fid), + "state": STATES.index(self.state[fid]["state"])} for fid in self.features}, + "envelope": {"dollars_left": totals["dollars_left"], "calls_left": totals["calls_left"], "seconds_left": totals["seconds_left"], + "smallest_tier_dollars": min(tiers.values()), "attempts_per_feature": self.envelope["attempts_per_feature"], + "arms": [{"id": a["id"], "tier_rank": a.get("tier_rank", 0)} for a in self.envelope["arms"]], + "tiers": dict(tiers)}} + + def ask(self, key: str, feature: str | None, options: list[str], numbers: dict | None = None) -> policymod.Answer: + question = policymod.Question(key, feature, tuple(options), dict(numbers or {})) + answer = policymod.ask(self.policy, self.policy_state(), question) + record(self.ledger, "policy_proposal", self.map_digest, {"question": key, "feature": feature, "options": list(options), + "answer": answer.record(), "policy": getattr(self.policy, "name", "policy")}) + self.log(f"policy {key} {feature or '-'} -> {answer.value} ({answer.source}, {answer.reason})" + + (f"; shadow {answer.shadow['policy']} would say {answer.shadow['value']}" if answer.shadow else "")) + return answer + + # -- run ---------------------------------------------------------------------------------------------------------------- + + def run(self, max_steps: int = 50) -> dict: + """Steps until nothing is eligible, the envelope is spent, the kill switch is set or ``max_steps`` pass. + An envelope whose ledger already carries a ``stop`` is refused before anything starts: a grant is spent + once. A ``Rejected`` raised while running (an invalid policy answer, a refused reservation, a thread or + checkout the kernel refused) still ends the record: the envelope gets its ``stop`` line and the ledger + its ``loop_stopped`` with the refusal's code as the reason, and then the refusal propagates.""" + if not self.tickets: + self.compile() + previous = self.spend.totals()["stopped"] + if previous: + raise Rejected("envelope_stopped", f"this envelope's ledger already stopped ({previous}); issue a new envelope for a new run") + self.order = [f["id"] for f in self.map["features"]] + kill = self.envelope.get("kill_switch") + stopped, failure = None, None + try: + self.kit = Kit.local(self.kit_home, self.repo_root, actor=self.actor) + for _ in range(max_steps): + if kill and Path(kill).exists(): + stopped = "kill_switch" + break + if self.spend.totals()["stopped"]: + stopped = self.spend.totals()["stopped"] + break + eligible = self.eligible() + if not eligible: + stopped = "nothing_eligible" + break + tiers = [t for t in policymod.TIERS if self.spend.room(t)] + if not tiers: + stopped = "envelope_exhausted" + break + chosen = self.ask("next_feature", None, eligible).value + arm = self.ask("proposer_arm", chosen, self._arms(chosen)).value + tier = self.ask("budget_tier", chosen, tiers).value + self._attempt(chosen, arm, tier) + else: + stopped = "max_steps" + except Rejected as exc: + stopped, failure = exc.code, exc + finally: + if self.kit is not None: + self.kit.close() + self.spend.stop(stopped) + record(self.ledger, "loop_stopped", self.map_digest, {"reason": stopped, "states": {fid: row["state"] for fid, row in self.state.items()}, + "calls": self.spend.totals()["calls"], "dollars": self.spend.totals()["dollars"]}, + refs={"detail": failure.detail} if failure else None) + if failure is not None: + raise failure + return self.summary(stopped) + + def _arm_rank(self, arm_id: str) -> int: + return int(next(a for a in self.envelope["arms"] if a["id"] == arm_id).get("tier_rank", 0)) + + def _arms(self, fid: str) -> list[str]: + """The arms this feature may run now: every arm of the envelope, or after ``retry_escalate_arm`` only the + arms ranked above the one that failed, so the recorded decision is the one that happens.""" + floor = self.state[fid]["arm_floor"] + offered = [a["id"] for a in self.envelope["arms"] if int(a.get("tier_rank", 0)) >= floor] + if not offered: + raise Rejected("arm_unavailable", f"{fid}: no arm of the envelope is ranked at or above {floor}") + return offered + + def summary(self, stopped: str | None = None) -> dict: + totals = self.spend.totals() + return {"stopped": stopped or totals["stopped"], "map_digest": self.map_digest, + "features": {fid: {"state": row["state"], "attempts": row["attempts"], "ticket": self.tickets.get(fid, {}).get("ticket_id"), + "thread": row["thread"], "commit": row["commit"], "last_reason": row["last_reason"]} + for fid, row in self.state.items()}, + "ready_for_owner_merge": [fid for fid, row in self.state.items() if row["state"] == "ready_for_owner_merge"], + "envelope": totals, "events": len(autonomy_events(self.ledger, self.map_digest))} + + def _attempt(self, fid: str, arm_id: str, tier: str) -> None: + item, ticket, row = self.features[fid], self.tickets[fid], self.state[fid] + row["attempts"] += 1 + attempt = row["attempts"] + self.spend.reserve(fid, attempt, tier, arm_id) + record(self.ledger, "envelope_reserved", self.map_digest, {"feature": fid, "attempt": attempt, "tier": tier, "arm": arm_id, + "dollars": self.envelope["dollars"]["tiers"][tier]}) + arm = next(a for a in self.envelope["arms"] if a["id"] == arm_id) + row["last_arm"] = arm_id + started = time.monotonic() + stacked = None + objective = f"{fid}: {ticket['objective']}\nCriteria: " + "; ".join(f"{c['id']} {c['text']}" for c in ticket["acceptance_criteria"]) + contract = f"syberfactory-{ticket['ticket_id']}.v1" + if item["depends_on"] and self.map["integration"] == "stack": + stacked = self.state[item["depends_on"][-1]]["commit"] # one branch: the last dependency's accepted commit + thread = self._start_stacked(fid, stacked, objective, contract, item) + else: + thread = self.kit.start(objective, contract=contract, paths=item["scope"]["paths"], act_class=item["act_class"], unattended=True) + row["state"], row["thread"] = "building", thread.id + record(self.ledger, "attempt_started", self.map_digest, {"feature": fid, "attempt": attempt, "ticket_id": ticket["ticket_id"], + "arm": arm_id, "tier": tier, "contract": contract, "stacked": stacked is not None}, + refs={"thread": thread.id, "base": thread.base, "stacked_on": stacked}) + provider = CommandProvider(self.proposer_argv, name=_name(arm_id), revision=str(arm.get("revision", "1")), + timeout=int(arm.get("timeout_seconds", 300))) + outcome, reason = "failed", None + try: + candidates = thread.propose(provider) + if not candidates: + reason = "no_candidate" + else: + outcome, reason = self._drive(thread, fid, attempt) + except Rejected as exc: + reason = exc.code + seconds = time.monotonic() - started + dollars, unknown = _spend(thread.history()) + row["dollars"] = round(row["dollars"] + dollars, 6) + self.spend.settle(fid, attempt, dollars=dollars, calls=provider.calls, seconds=seconds, unknown_models=unknown) + record(self.ledger, "envelope_settled", self.map_digest, {"feature": fid, "attempt": attempt, "dollars": dollars, + "calls": provider.calls, "unknown_models": unknown}, + refs={"seconds": round(seconds, 3)}) + if outcome == "accepted": + accepted = next(v for v in thread._views() if v["authoritative"]) + row["state"], row["commit"], row["last_reason"] = "ready_for_owner_merge", accepted["commit"], None + record(self.ledger, "feature_ready", self.map_digest, {"feature": fid, "attempt": attempt, "ticket_id": ticket["ticket_id"], + "exit_criteria": ticket["exit_criteria"], "next": "owner merges; nothing here merges"}, + refs={"thread": thread.id, "commit": accepted["commit"], "target_ref": thread.target_ref}) + self.log(f"{fid} ready for owner merge: {thread.target_ref} at {accepted['commit'][:10]}") + return + row["last_reason"] = reason + record(self.ledger, "feature_failed", self.map_digest, {"feature": fid, "attempt": attempt, "reason": reason}, refs={"thread": thread.id}) + self._after_failure(fid) + + def _start_stacked(self, fid: str, commit: str, objective: str, contract: str, item: dict): + """Start a thread from a dependency's accepted commit. ``Kit.start`` starts from HEAD, so HEAD is moved + to that commit on a scratch branch for the start and put back afterwards (the branch or detached commit + the owner had), whether or not the start succeeded; the scratch branch is deleted. A working tree with + uncommitted tracked changes is refused first (``repository_dirty``): a checkout would carry or lose them.""" + repo = self.kit.repo + if repo.git("status", "--porcelain", "--untracked-files=no").strip(): + raise Rejected("repository_dirty", f"{repo.root} has uncommitted changes; commit or stash them before a stacked feature starts") + branch = repo.current_branch() + head = repo.rev("HEAD") + scratch = f"syberfactory/{self.map['id']}/{fid}" + repo.git("checkout", "-q", "-B", scratch, commit) + try: + return self.kit.start(objective, contract=contract, paths=item["scope"]["paths"], act_class=item["act_class"], unattended=True) + finally: + if branch: + repo.git("checkout", "-q", branch[len("refs/heads/"):] if branch.startswith("refs/heads/") else branch) + else: + repo.git("checkout", "-q", "--detach", head) + repo.git("branch", "-q", "-D", scratch, ok=(0, 1)) + + def _drive(self, thread, fid: str, attempt: int) -> tuple[str, str | None]: + """``syberlabs run``'s passes over this one thread until nothing is left: the kernel checks, accepts and + recovers; this records what each pass did.""" + runner = Runner(self.kit, thread=thread.id, log=self.log) + for number in range(1, self.max_passes + 1): + steps = runner.once() + for step in steps: + record(self.ledger, "thread_step", self.map_digest, {"feature": fid, "attempt": attempt, "pass": number, "step": step.step, + "outcome": step.outcome, "reason": step.reason}, refs={"thread": thread.id}) + if any(s.step == "accept" and s.outcome == "succeeded" for s in steps): + return "accepted", None + refused = [s for s in steps if s.refused] + if refused: + return "failed", refused[0].reason or refused[0].outcome + if all(s.step in ("idle", "skip") for s in steps): + idle = steps[-1] if steps else None + return "failed", (idle.reason if idle else "no_step") + return "failed", "passes_exhausted" + + def _after_failure(self, fid: str) -> None: + row, item = self.state[fid], self.features[fid] + left = self.envelope["attempts_per_feature"] - row["attempts"] + options = ["continue", "stop"] if left > 0 else ["stop"] + worth = self.ask("worth_continuing", fid, options).value + choices = [] + last_rank = self._arm_rank(row["last_arm"]) if row["last_arm"] else 0 + stronger = [a["id"] for a in self.envelope["arms"] if int(a.get("tier_rank", 0)) > last_rank] + if worth == "continue" and left > 0: + choices.append("retry_same") + if stronger: + choices.append("retry_escalate_arm") + choices += ["escalate_owner", "park"] + decision = self.ask("after_failure", fid, choices).value + if decision in ("retry_same", "retry_escalate_arm"): + if decision == "retry_escalate_arm": + row["arm_floor"] = last_rank + 1 # the next attempt's proposer_arm offers only the stronger arms + row["state"] = "open" # eligible again; the next step may pick it or another feature + return + row["state"] = "escalated" if decision == "escalate_owner" else "parked" + record(self.ledger, "feature_escalated" if decision == "escalate_owner" else "feature_parked", self.map_digest, + {"feature": fid, "reason": row["last_reason"], "attempts": row["attempts"]}) + + +def _name(arm_id: str) -> str: + cleaned = "".join(c if c.isalnum() or c in "_.-" else "-" for c in arm_id.lower()) + return (cleaned if cleaned[:1].isalpha() else "arm-" + cleaned)[:64] + + +def _spend(history: list[dict]) -> tuple[float, list[str]]: + """Host-observed dollars from the thread's finished searches, and the models the host could not price.""" + dollars, unknown = 0.0, [] + for event in history: + if event["kind"] == "search_finished" and isinstance(event["body"].get("spend"), dict): + spend = event["body"]["spend"] + dollars += float(spend.get("dollars") or 0.0) + unknown += [m for m in spend.get("unknown_models", []) if m not in unknown] + return round(dollars, 6), unknown + + +def load_json(path: str | Path, what: str) -> dict: + try: + return _strict_loads(Path(path).read_text(encoding="utf-8")) + except (OSError, ValueError) as exc: + raise Rejected(f"{what}_unreadable", f"{path}: {type(exc).__name__}: {exc}") from None diff --git a/syberlabs/factory/policy.py b/syberlabs/factory/policy.py new file mode 100644 index 0000000..853b17b --- /dev/null +++ b/syberlabs/factory/policy.py @@ -0,0 +1,188 @@ +"""FeaturePolicy: the five typed questions a loop over a feature map asks, with closed answer sets. + +A policy decides nothing. It proposes: ``decide(state, question)`` returns an ``Answer`` whose value is one of +the options code built before the question was asked, and the loop records the answer on the factory ledger +as a proposal before it acts on it. An answer outside the options is refused (``policy_invalid``) and never +acted on. The state a policy reads is ids and numbers (owner decision 1): feature ids, graph positions, risk +scores, attempts, dollars and calls. No titles, criteria, diffs, check output or paths. + +The questions (autonomous design, section 4.2), each with the option set code builds: + +- ``next_feature`` (choice): the eligible features (dependencies satisfied, no conflicting scope running, act + class allowed, room in the envelope, attempts left). With no eligible feature the loop stops; no question. +- ``proposer_arm`` (choice): the envelope's arms; after ``retry_escalate_arm`` only the arms ranked above the one + that failed, so the cheapest answer is then the cheapest stronger arm. +- ``budget_tier`` (choice): the tiers whose ceiling fits the remaining dollars (``S``, ``M``, ``L``). +- ``after_failure`` (choice): ``retry_same``, ``retry_escalate_arm``, ``escalate_owner``, ``park``; an option is + removed when the caps forbid it (no attempts left removes the retries). +- ``worth_continuing`` (choice over ``continue`` and ``stop``): may only stop early, never extend a cap. + +``OfflinePolicy`` is the default (owner decision 7): critical path first (highest bottom level, then higher +risk, then map order) for ``next_feature``, the cheapest arm and tier that fit, retry then escalate the arm +then park after a failure, stop when the envelope has no room. ``ShadowPolicy`` wraps an acting policy and a +second one that only answers: what it would have said is recorded beside the acting answer and never acted +on (Jev stays in shadow until E7). The shadow is an injected callable; nothing here imports or calls a +network client. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Any, Callable, Protocol + +from syberlabs.errors import Rejected + +QUESTIONS = ("next_feature", "proposer_arm", "budget_tier", "after_failure", "worth_continuing") +TIERS = ("S", "M", "L") +AFTER_FAILURE = ("retry_same", "retry_escalate_arm", "escalate_owner", "park") +WORTH = ("continue", "stop") +SOURCES = ("offline", "shadow", "rule") + + +@dataclass(frozen=True) +class Question: + key: str + feature: str | None + options: tuple[str, ...] + numbers: dict = field(default_factory=dict) # ids and numbers only; what a remote policy would be sent + + def __post_init__(self): + if self.key not in QUESTIONS: + raise Rejected("policy_question", f"unknown question {self.key!r}; one of {QUESTIONS}") + if not self.options or len(set(self.options)) != len(self.options): + raise Rejected("policy_question", f"{self.key}: options must be a non-empty list without repeats") + closed = {"budget_tier": TIERS, "after_failure": AFTER_FAILURE, "worth_continuing": WORTH}.get(self.key) + if closed is not None and any(o not in closed for o in self.options): + raise Rejected("policy_question", f"{self.key}: options must come from {closed}") + + +@dataclass +class Answer: + key: str + value: str + source: str + reason: str + probabilities: dict | None = None + shadow: dict | None = None + + def record(self) -> dict: + return {"key": self.key, "value": self.value, "source": self.source, "reason": self.reason, + "probabilities": self.probabilities, "shadow": self.shadow} + + +class FeaturePolicy(Protocol): + name: str + + def decide(self, state: dict, question: Question) -> Answer: ... + + +def validate(question: Question, answer: Any, source: str) -> Answer: + """An answer is an ``Answer`` or a plain value; either way its value must be one of the options sent.""" + if isinstance(answer, Answer): + value, reason, probabilities = answer.value, answer.reason, answer.probabilities + elif isinstance(answer, dict): + value, reason, probabilities = answer.get("value"), answer.get("reason", "cleared"), answer.get("probabilities") + else: + value, reason, probabilities = answer, "cleared", None + if value not in question.options: + raise Rejected("policy_invalid", f"{source} answered {value!r} to {question.key}; the options were {list(question.options)}") + if probabilities is not None: + if not isinstance(probabilities, dict) or set(probabilities) != set(question.options): + raise Rejected("policy_invalid", f"{source}: probabilities must cover exactly the options sent") + for option, p in probabilities.items(): + if isinstance(p, bool) or not isinstance(p, (int, float)) or not 0 <= p <= 1 or p != p: + raise Rejected("policy_invalid", f"{source}: probability for {option!r} is not a number in [0, 1]") + if abs(sum(probabilities.values()) - 1) > 0.02: + raise Rejected("policy_invalid", f"{source}: probabilities sum to {sum(probabilities.values()):.3f}, not 1") + return Answer(question.key, value, source, str(reason), probabilities) + + +class OfflinePolicy: + """Deterministic rules over the state's numbers. The default.""" + name = "offline" + + def decide(self, state: dict, question: Question) -> Answer: + features = state["features"] + if question.key == "next_feature": + pick = max(question.options, key=lambda fid: (features[fid]["bottom_level"], features[fid]["risk"], -features[fid]["map_order"])) + return Answer(question.key, pick, self.name, "rule:critical_path_first") + if question.key == "proposer_arm": + arms = {arm["id"]: arm for arm in state["envelope"]["arms"]} + pick = min(question.options, key=lambda a: (arms[a].get("tier_rank", 0), question.options.index(a))) + return Answer(question.key, pick, self.name, "rule:cheapest_arm") + if question.key == "budget_tier": + attempts = features[question.feature]["attempts"] + wanted = TIERS[min(attempts, len(TIERS) - 1)] + pick = wanted if wanted in question.options else question.options[0] + return Answer(question.key, pick, self.name, "rule:cheapest_tier_that_fits") + if question.key == "after_failure": + # First failure: the same arm again. Later failures with attempts left: a stronger arm when the envelope + # has one, else the owner. No retry offered (the cap is spent): park; the ledger holds the reason. + attempts = features[question.feature]["attempts"] + retry = [o for o in question.options if o.startswith("retry_")] + if attempts <= 1 and "retry_same" in retry: + return Answer(question.key, "retry_same", self.name, "rule:first_failure_retry") + if "retry_escalate_arm" in retry: + return Answer(question.key, "retry_escalate_arm", self.name, "rule:escalate_arm") + if retry and "escalate_owner" in question.options: + return Answer(question.key, "escalate_owner", self.name, "rule:no_stronger_arm") + pick = "park" if "park" in question.options else question.options[-1] + return Answer(question.key, pick, self.name, "rule:attempts_spent") + if question.key == "worth_continuing": + envelope = state["envelope"] + room = envelope["dollars_left"] >= envelope["smallest_tier_dollars"] and envelope["calls_left"] > 0 + left = features[question.feature]["attempts_left"] > 0 + pick = "continue" if (room and left and "continue" in question.options) else "stop" + return Answer(question.key, pick, self.name, "rule:room_and_attempts" if pick == "continue" else "rule:stop_on_envelope") + raise Rejected("policy_question", f"offline policy has no rule for {question.key}") + + +class ShadowPolicy: + """Act on ``acting``; ask ``shadow`` the same question and record what it would have answered. + + ``shadow(state, question)`` is any callable, injected by the caller: a recorded-answer selector, a test + stub, or later an adapter that reuses ``jev_api``'s request and validation code. Its answer is validated + against the same options; an invalid or failing shadow is recorded as such (``shadow.reason``) and changes + nothing. The acting answer is returned unchanged, with ``shadow`` filled in.""" + name = "shadow" + + def __init__(self, acting: FeaturePolicy, shadow: Callable[[dict, Question], Any], *, shadow_name: str = "shadow"): + self.acting, self.shadow, self.shadow_name = acting, shadow, shadow_name + self.disagreements = 0 + + def decide(self, state: dict, question: Question) -> Answer: + answer = validate(question, self.acting.decide(state, question), self.acting.name) + try: + would = validate(question, self.shadow(state, question), self.shadow_name) + record = {"policy": self.shadow_name, "value": would.value, "probabilities": would.probabilities, + "agrees": would.value == answer.value, "reason": "recorded"} + except Rejected as exc: + record = {"policy": self.shadow_name, "value": None, "probabilities": None, "agrees": None, "reason": exc.code} + except Exception as exc: # a shadow that breaks never breaks the loop + record = {"policy": self.shadow_name, "value": None, "probabilities": None, "agrees": None, + "reason": f"shadow_error:{type(exc).__name__}"} + if record["agrees"] is False: + self.disagreements += 1 + answer.shadow = record + return answer + + +def recorded_answers(answers: dict[str, Any]) -> Callable[[dict, Question], Any]: + """A shadow over a fixture: ``answers`` maps ``""`` or ``":"`` to a value or + ``{"value", "probabilities"}``. A question with no recorded answer raises, which the shadow wrapper + records as ``shadow_unanswered``.""" + def shadow(state: dict, question: Question): + for key in (f"{question.key}:{question.feature}", question.key): + if key in answers: + return answers[key] + raise Rejected("shadow_unanswered", f"no recorded answer for {question.key}") + return shadow + + +def ask(policy: FeaturePolicy, state: dict, question: Question) -> Answer: + """One validated proposal. The caller records it before acting.""" + answer = policy.decide(state, question) + source = answer.source if isinstance(answer, Answer) else getattr(policy, "name", "policy") + checked = validate(question, answer, source) + checked.shadow = answer.shadow if isinstance(answer, Answer) else None + return checked diff --git a/tests/fixtures/autonomy/hidden_checks.json b/tests/fixtures/autonomy/hidden_checks.json new file mode 100644 index 0000000..b109720 --- /dev/null +++ b/tests/fixtures/autonomy/hidden_checks.json @@ -0,0 +1,6 @@ +{ + "_comment": "The host's hidden-check catalogue: name -> argv, run in the candidate's worktree. '{python}' is the interpreter. The map names these checks; the worktree never holds them.", + "hidden.f1.marker": ["{python}", "-c", "import sys, pathlib; sys.exit(0 if pathlib.Path('fxapp/rules.py').read_text().rstrip().endswith('# F1 delivered') else 1)"], + "hidden.f2.marker": ["{python}", "-c", "import sys, pathlib; t = pathlib.Path('fxapp/ingest.py').read_text(); r = pathlib.Path('fxapp/rules.py').read_text(); sys.exit(0 if t.rstrip().endswith('# F2 delivered') and '# F1 delivered' in r else 1)"], + "hidden.f3.marker": ["{python}", "-c", "import sys, pathlib; sys.exit(0 if pathlib.Path('fxapp/variants.py').read_text().rstrip().endswith('# F3 delivered') else 1)"] +} diff --git a/tests/fixtures/autonomy/map.json b/tests/fixtures/autonomy/map.json new file mode 100644 index 0000000..39acc75 --- /dev/null +++ b/tests/fixtures/autonomy/map.json @@ -0,0 +1,40 @@ +{ + "schema": "syberfactory/feature-map.v1", + "id": "order-markers.m1", + "spec_ref": {"spec_id": "order-markers", "version": 1, "spec_digest": ""}, + "base": null, + "author": {"kind": "human", "id": "fixture"}, + "features": [ + { + "id": "F1", "title": "Rules reviewed", "objective": "Append the F1 delivery marker to fxapp/rules.py", + "req_ids": ["R1"], "depends_on": [], + "scope": {"paths": ["fxapp/rules.py"], "exclude": [], "max_files": 1, "max_diff_bytes": 2000}, + "overlaps": [], "act_class": "R", + "criteria": [{"id": "F1.c1", "text": "fxapp/rules.py ends with the line '# F1 delivered'", "kind": "test"}], + "hidden_checks": ["hidden.f1.marker"], "size": "S", + "risk_inputs": {"new_dependencies": 0, "needs_lookup": false, "disagreement": 0.1} + }, + { + "id": "F2", "title": "Ingest reviewed after rules", "objective": "Append the F2 delivery marker to fxapp/ingest.py", + "req_ids": ["R2"], "depends_on": ["F1"], + "scope": {"paths": ["fxapp/ingest.py"], "exclude": [], "max_files": 1, "max_diff_bytes": 2000}, + "overlaps": [], "act_class": "R", + "criteria": [{"id": "F2.c1", "text": "fxapp/ingest.py ends with the line '# F2 delivered'", "kind": "test"}], + "hidden_checks": ["hidden.f2.marker"], "size": "S", + "risk_inputs": {"new_dependencies": 0, "needs_lookup": false, "disagreement": 0.0} + }, + { + "id": "F3", "title": "Variants reviewed", "objective": "Append the F3 delivery marker to fxapp/variants.py", + "req_ids": ["R3"], "depends_on": [], + "scope": {"paths": ["fxapp/variants.py"], "exclude": [], "max_files": 1, "max_diff_bytes": 2000}, + "overlaps": [], "act_class": "R", + "criteria": [{"id": "F3.c1", "text": "fxapp/variants.py ends with the line '# F3 delivered'", "kind": "test"}, + {"id": "F3.c2", "text": "the entrypoints keep their names", "kind": "judgment"}], + "hidden_checks": ["hidden.f3.marker"], "size": "M", + "risk_inputs": {"new_dependencies": 0, "needs_lookup": false, "disagreement": 0.0} + } + ], + "uncovered": [], + "integration": "stack", + "approval": null +} diff --git a/tests/fixtures/autonomy/spec.md b/tests/fixtures/autonomy/spec.md new file mode 100644 index 0000000..547b6ae --- /dev/null +++ b/tests/fixtures/autonomy/spec.md @@ -0,0 +1,16 @@ +# Order markers (fixture spec for the autonomy thin slice) + +Sectioned Markdown with ids (owner decision 11, 2026-10-28). Each `##` section is one requirement of #154's spec +object; the bullets are its scenarios. The three features of `map.json` trace to these sections. + +## R1: The rules module records that its decision thresholds were reviewed. + +- given fxapp/rules.py, when the feature is delivered, then the module ends with the delivery marker for F1 + +## R2: The ingest module records that its normalization was reviewed after the rules. + +- given fxapp/ingest.py, when the feature is delivered after F1, then the module ends with the delivery marker for F2 + +## R3: The variants module records that its entrypoints were reviewed. + +- given fxapp/variants.py, when the feature is delivered, then the module ends with the delivery marker for F3 diff --git a/tests/fixtures/autonomy/standin_proposer.py b/tests/fixtures/autonomy/standin_proposer.py new file mode 100644 index 0000000..c252be3 --- /dev/null +++ b/tests/fixtures/autonomy/standin_proposer.py @@ -0,0 +1,34 @@ +"""A stand-in proposer for the autonomy thin slice: the process boundary a model adapter would sit behind. + +Usage: ``standin_proposer.py REPO_ROOT [--break | --outside]``. It reads the search request on stdin (the +thread's objective, its scope and the file list), takes the first scope path, reads that file from REPO_ROOT's +working tree, and answers one candidate that appends the feature's delivery marker (``# F1 delivered``) to it. +``--break`` appends a line that is not Python instead, so the visible check fails; ``--outside`` edits a path +outside the scope, which the host refuses as out of scope. No model, no network, no cost. +""" + +import json +import sys +from pathlib import Path + + +def main(argv: list[str]) -> int: + root = Path(argv[0]) + mode = argv[1] if len(argv) > 1 else "" + request = json.load(sys.stdin) + feature = request["objective"].split(":", 1)[0].strip() + path = request["scope"][0] + text = (root / path).read_text(encoding="utf-8") + if mode == "--break": + changes = {path: text.rstrip("\n") + "\nthis line is not python\n"} + elif mode == "--outside": + changes = {"fxapp/main.py": "# edited outside the feature's scope\n"} + else: + changes = {path: text.rstrip("\n") + f"\n\n# {feature} delivered\n"} + json.dump({"candidates": [{"changes": changes, "message": f"{feature}: delivery marker", "signal": None}], + "recommended": [0]}, sys.stdout) + return 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv[1:])) diff --git a/tests/test_factory_autonomy.py b/tests/test_factory_autonomy.py new file mode 100644 index 0000000..da71cc2 --- /dev/null +++ b/tests/test_factory_autonomy.py @@ -0,0 +1,771 @@ +"""The autonomy thin slice (AM1) on the fixture app: spec -> approved feature map -> tickets on the ledger -> Build +Threads driven by the runner with a stand-in proposer -> features ready for the owner's merge, inside an envelope. +Offline, no model, no network, no merge. The spec is promoted through the goal gate's real path: review and +validate events on the ledger, the owner's signed approval verified and recorded, the promotion citing it.""" + +import json +import math +import os +import shutil +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +from syberlabs.canonical import canonical +from syberlabs.ed25519 import generate +from syberlabs.errors import Rejected +from syberlabs.factory import featuremap, ledger, loop, space +from syberlabs.factory import policy as policymod +from syberlabs.factory import spec as specmod + +ROOT = Path(__file__).resolve().parents[1] +APP = ROOT / "tests" / "fixtures" / "factory_app" +AUTONOMY = ROOT / "tests" / "fixtures" / "autonomy" +DECLARED = json.loads((APP / "architecture.json").read_text(encoding="utf-8")) +OWNER = {"kind": "human", "id": "tester"} +OWNER_SEED, OWNER_PUBLIC = generate() +VERDICT = "review-1" +GIT_ENV = {"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@example.test", "GIT_COMMITTER_NAME": "t", + "GIT_COMMITTER_EMAIL": "t@example.test", "GIT_AUTHOR_DATE": "2026-01-01T00:00:00Z", + "GIT_COMMITTER_DATE": "2026-01-01T00:00:00Z"} + + +def candidate_spec(project: Path) -> dict: + """The fixture spec as #154's object: from sectioned Markdown, footprint resolved against Space, validated.""" + document = featuremap.spec_from_markdown((AUTONOMY / "spec.md").read_text(encoding="utf-8"), "order-markers", + "delivery markers on the order path", ["rules", "ingest", "main"], author=OWNER) + graph = space.index(project, revision="base") + space.map_components(graph, DECLARED) + specmod.resolve_footprint(document, graph) + specmod.transition(document, "validate", OWNER, space=graph) + return document + + +def promoted_spec(project: Path, home: ledger.Ledger, *, seed: bytes = OWNER_SEED, public: bytes = OWNER_PUBLIC) -> dict: + """The candidate promoted the way `factory spec promote` does it (C1's goal gate): the ledger pins the owner + key (bootstrap, once); a model's variant, an independent reviewer's accepting verdict and the validate that + wrote the candidate are on the ledger; the owner signs an approval over the spec's digest, `gate` resolves the + verdict and verifies the approval under the pinned key (read through the lineage with the ledger's stored + rotations), `transition` promotes, the approval is recorded and + the `spec_changed` cites it.""" + document = candidate_spec(project) + if home.owner_key() is None: + home.set_owner_key(public) + before = specmod.spec_digest(document) + reviewer = {"kind": "model", "id": "gpt", "model": "gpt-5", "family": "openai"} + home.append("spec_changed", {"spec_id": document["spec_id"], "version": document["version"], "operation": "mutate", + "variant_id": "var1", "actor": {"kind": "model", "id": "claude", "family": "anthropic"}}) + home.append("spec_changed", {"spec_id": document["spec_id"], "version": document["version"], "operation": "review", + "verdict_id": VERDICT, "variant_id": "var1", "decision": "accept", "actor": reviewer}) + home.append("spec_changed", {"spec_id": document["spec_id"], "version": document["version"], "operation": "validate", + "accepted_variant": "var1", "verdict_id": VERDICT, "spec_digest": before}) + envelope = specmod.sign_approval(specmod.approval_payload(document, "promote", OWNER, verdict_ids=[VERDICT]), seed) + approval, resolved = specmod.gate(document, "promote", OWNER, home.events(), envelope, verdict_ids=[VERDICT], spec_digest_value=before, + load_rotation=home.load_rotation) + record = specmod.transition(document, "promote", OWNER, verdict_ids=[VERDICT], approval=approval, spec_digest=before, + resolved_verdicts=resolved) + event = home.record_approval(envelope, approval) + record["approval"] = {**record["approval"], "event_hash": event["hash"], "approval_digest": event["body"]["approval_digest"]} + home.append("spec_changed", {"spec_id": document["spec_id"], "version": document["version"], **record, + "spec_digest": specmod.spec_digest(document), "footprint": document["footprint"]["components"]}) + return document + + +def fixture_map(document: dict, base: str, **overrides) -> dict: + mapping = json.loads((AUTONOMY / "map.json").read_text(encoding="utf-8")) + mapping["spec_ref"]["spec_digest"] = specmod.spec_digest(document) + mapping["base"] = base + mapping.update(overrides) + return mapping + + +def hidden_catalogue() -> dict: + raw = json.loads((AUTONOMY / "hidden_checks.json").read_text(encoding="utf-8")) + return {name: [arg.replace("{python}", sys.executable) for arg in argv] for name, argv in raw.items() if not name.startswith("_")} + + +def git(project: Path, *args: str) -> str: + env = {**os.environ, **GIT_ENV} + return subprocess.run(["git", "-C", str(project), *args], check=True, capture_output=True, text=True, env=env).stdout.strip() + + +class Scratch(unittest.TestCase): + def setUp(self): + self.scratch = Path(tempfile.mkdtemp(prefix="autonomy-")) + self.addCleanup(shutil.rmtree, self.scratch, True) + + def project(self, name="project") -> tuple[Path, str]: + """A copy of the fixture app as a Git repository with a deterministic base commit.""" + project = self.scratch / name + shutil.copytree(APP, project) + git(project, "init", "-q", "-b", "main") + git(project, "add", ".") + git(project, "-c", "commit.gpgsign=false", "commit", "-qm", "base") + return project, git(project, "rev-parse", "HEAD") + + def home(self, name="ledger") -> ledger.Ledger: + return ledger.Ledger(self.scratch / name) + + def make_loop(self, project: Path, mapping: dict, document: dict, home: ledger.Ledger, *, calls_total=10, attempts=2, mode="", + policy=None, kill_switch=None, arms=None) -> loop.Loop: + envelope = loop.new_envelope(featuremap.map_digest(mapping), dollars_total=0.0, calls_total=calls_total, seconds_total=600, + attempts_per_feature=attempts, kill_switch=kill_switch, arms=arms) + argv = [sys.executable, str(AUTONOMY / "standin_proposer.py"), str(project)] + ([mode] if mode else []) + checks = {"tests": {"argv": [sys.executable, "-c", "import fxapp.main, fxapp.variants"], "timeout_seconds": 60}} + return loop.Loop(repo_root=project, kit_home=project / ".syberlabs", ledger=home, map=mapping, spec=document, envelope=envelope, + policy=policy or policymod.OfflinePolicy(), proposer_argv=argv, checks=checks, hidden_catalogue=hidden_catalogue(), + human_owner="tester") + + def ready_loop(self, name="project", ledger_name="ledger", **overrides) -> tuple[loop.Loop, Path, str, dict]: + """A project, a promoted spec on a fresh ledger and an approved fixture map: the loop ready to compile.""" + project, base = self.project(name) + home = self.home(ledger_name) + document = promoted_spec(project, home) + mapping = fixture_map(document, base, **overrides.pop("map", {})) + featuremap.approve(mapping, OWNER) + return self.make_loop(project, mapping, document, home, **overrides), project, base, mapping + + +def notes(events, note): + return [e for e in events if e["kind"] == "note" and e["body"]["note"] == note] + + +class SpecAndMap(Scratch): + def test_sectioned_markdown_becomes_the_spec_object(self): + document = featuremap.spec_from_markdown((AUTONOMY / "spec.md").read_text(encoding="utf-8"), "order-markers", "cap", ["rules"]) + self.assertEqual([r["req_id"] for r in document["requirements"]], ["R1", "R2", "R3"]) + self.assertEqual(document["requirements"][0]["scenarios"][0]["given"], "fxapp/rules.py") + self.assertEqual(document["requirements"][0]["scenarios"][0]["then"], "the module ends with the delivery marker for F1") + self.assertEqual(document["state"], "draft") + with self.assertRaises(Rejected): + featuremap.spec_from_markdown("no sections here", "x", "y", ["rules"]) + + def test_fixture_map_validates_and_the_task_graph_measures(self): + project, base = self.project() + document = promoted_spec(project, self.home()) + mapping = fixture_map(document, base) + self.assertEqual(featuremap.mapcheck(mapping, document), []) + graph = featuremap.taskgraph(mapping) + self.assertEqual(graph["critical_path"], ["F1", "F2"]) + self.assertEqual(graph["critical_path_length"], 2) + self.assertEqual(graph["total_work"], 4, "S + S + M") + self.assertEqual(graph["work_critical_ratio"], 2.0) + self.assertEqual(graph["width"], 2) + self.assertIn(sorted(graph["max_antichain"]), (["F1", "F3"], ["F2", "F3"]), "either pair is a maximum antichain") + self.assertEqual(graph["bottom_level"], {"F1": 2, "F2": 1, "F3": 2}) + self.assertEqual(graph["conflicts"], []) + featuremap.approve(mapping, OWNER) + self.assertTrue(featuremap.approved(mapping)) + self.assertEqual(featuremap.mapcheck(mapping, document), []) + mapping["features"][0]["title"] = "edited after approval" + self.assertFalse(featuremap.approved(mapping)) + self.assertIn("approval is present but stale", featuremap.mapcheck(mapping, document)[0]) + with self.assertRaises(Rejected) as refused: + featuremap.approve(mapping, {"kind": "model", "id": "claude"}) + self.assertEqual(refused.exception.code, "approval_actor") + + def test_mapcheck_refuses_a_cycle_flags_a_scope_conflict_and_checks_coverage(self): + project, base = self.project() + document = promoted_spec(project, self.home()) + cyclic = fixture_map(document, base) + cyclic["features"][0]["depends_on"] = ["F2"] + findings = featuremap.mapcheck(cyclic, document) + self.assertTrue(any("cycle" in f for f in findings), findings) + with self.assertRaises(Rejected) as refused: + featuremap.taskgraph(cyclic) + self.assertEqual(refused.exception.code, "map_cyclic") + conflict = fixture_map(document, base) + conflict["features"][2]["scope"]["paths"] = ["fxapp/"] + findings = featuremap.mapcheck(conflict, document) + self.assertTrue(any(f.startswith("scope conflict: F1 and F3") for f in findings), findings) + self.assertTrue(any(f.startswith("scope conflict: F2 and F3") for f in findings), findings) + declared = fixture_map(document, base) + declared["features"][2]["scope"]["paths"] = ["fxapp/"] + declared["features"][2]["overlaps"] = [{"with": "F1", "paths": ["fxapp/rules.py"], "order": "F1_first"}, + {"with": "F2", "paths": ["fxapp/ingest.py"], "order": "F2_first"}] + self.assertEqual(featuremap.mapcheck(declared, document), [], "a declared overlap with an order is not a finding") + declared["features"][2]["depends_on"] = ["F1"] + declared["features"][2]["overlaps"][0]["order"] = "F3_first" + self.assertTrue(any("contradicts depends_on" in f for f in featuremap.mapcheck(declared, document))) + uncovered = fixture_map(document, base) + uncovered["features"][2]["req_ids"] = ["R1"] + findings = featuremap.mapcheck(uncovered, document) + self.assertIn("requirement R3 is covered by no feature and is not listed as uncovered", findings) + uncovered["uncovered"] = [{"req_id": "R3", "reason": "deferred to the next map"}] + self.assertEqual(featuremap.mapcheck(uncovered, document), []) + unknown = fixture_map(document, base) + unknown["features"][0]["req_ids"] = ["R9"] + self.assertIn("F1: req_ids names unknown requirement 'R9'", featuremap.mapcheck(unknown, document)) + stale = fixture_map(document, base) + stale["spec_ref"]["spec_digest"] = "0" * 64 + self.assertTrue(any("spec changed after the map" in f for f in featuremap.mapcheck(stale, document))) + shared = fixture_map(document, base) + shared["features"][1]["hidden_checks"] = ["hidden.f1.marker"] + self.assertTrue(any("also named by F1" in f for f in featuremap.mapcheck(shared, document))) + # Scope entries are compared normalised: case, leading "./" segments, backslashes and a trailing slash. + cased = fixture_map(document, base) + cased["features"][2]["scope"]["paths"] = ["./FXAPP/Rules.py"] + self.assertTrue(any(f.startswith("scope conflict: F1 and F3") for f in featuremap.mapcheck(cased, document)), + "FXAPP/Rules.py is fxapp/rules.py on a case-insensitive filesystem") + self.assertTrue(featuremap.path_intersects("./fxapp\\rules.py", "FXAPP/")) + self.assertFalse(featuremap.path_intersects(".github/x", "github/x"), "a leading dot is part of a name, not a './' prefix") + self.assertFalse(featuremap.path_intersects("fxapp/rules.py", "fxapp/rules")) + + def test_risk_plan_keeps_the_floor_and_allocates_softmax_units(self): + project, base = self.project() + document = promoted_spec(project, self.home()) + mapping = fixture_map(document, base) + plan = featuremap.risk_plan(mapping, budget=4, tau=1.0) + self.assertEqual(sum(plan["units"].values()), 4) + self.assertTrue(plan["standardized"]) + self.assertAlmostEqual(sum(plan["share"].values()), 1.0, places=3) + for fid, buys in plan["buys"].items(): + self.assertEqual(buys[: len(featuremap.FLOOR)], list(featuremap.FLOOR), f"{fid} keeps the floor") + self.assertGreater(plan["r"]["F1"], plan["r"]["F2"], "F1 has a dependent and disagreement; F2 has neither") + self.assertEqual(plan["inputs"]["F3"]["uncovered"], 0.5, "one of F3's two criteria has no mechanical check") + sharp = featuremap.risk_plan(mapping, budget=4, tau=0.1) + self.assertGreaterEqual(max(sharp["share"].values()), max(plan["share"].values()), "a lower tau concentrates") + with self.assertRaises(Rejected): + featuremap.risk_plan(mapping, tau=0.01) + # Weights are validated: exactly the known inputs, each in [0, 1], summing to 1, so r stays in [0, 1]. + for bad in ({**featuremap.WEIGHTS, "scope": 0.5}, {k: v for k, v in featuremap.WEIGHTS.items() if k != "scope"}, + {**featuremap.WEIGHTS, "extra": 0.0}, {**featuremap.WEIGHTS, "scope": -0.15, "act_class": 0.45}, + {**featuremap.WEIGHTS, "scope": float("nan")}): + with self.assertRaises(Rejected, msg=str(bad)) as refused: + featuremap.risk_plan(mapping, weights=bad) + self.assertEqual(refused.exception.code, "risk_weights") + flat = fixture_map(document, base) + for item in flat["features"]: + item["depends_on"], item["size"], item["criteria"] = [], "S", [{"id": f"{item['id']}.c1", "text": "x", "kind": "test"}] + item["risk_inputs"]["disagreement"] = 0.0 + plan = featuremap.risk_plan(flat, budget=3) + self.assertFalse(plan["standardized"]) + self.assertEqual(plan["units"], {"F1": 1, "F2": 1, "F3": 1}, "zero spread allocates uniformly") + + def test_a_feature_compiles_to_one_ticket_in_the_spec_ticket_shape(self): + project, base = self.project() + document = promoted_spec(project, self.home()) + mapping = fixture_map(document, base) + featuremap.approve(mapping, OWNER) + graph = space.index(project, revision="base") + space.map_components(graph, DECLARED) + tickets = featuremap.compile_tickets(mapping, document, human_owner="tester", baseline_run=None, space=graph) + self.assertEqual(len(tickets), 3) + ticket = tickets[1] + reference = specmod.ticket_from_spec(document, "R2", human_owner="tester") + self.assertEqual(set(reference) - set(ticket), set(), "every key of ticket_from_spec's shape is present") + self.assertEqual(ticket["ticket_id"], featuremap.digest({"map": featuremap.map_digest(mapping), "feature": "F2"})[:16]) + self.assertEqual(ticket["footprint"]["files"], ["fxapp/ingest.py"]) + self.assertEqual(ticket["footprint"]["components"], ["ingest"], "the Space graph's owner of the scope's files") + bare = featuremap.compile_ticket(mapping, mapping["features"][1], document, human_owner="tester") + self.assertEqual(bare["footprint"]["components"], [], "no Space graph, no component claim") + self.assertEqual(ticket["exit_criteria"], ["evidence:tests", "hidden:hidden.f2.marker"]) + self.assertEqual(ticket["rubric_digest"], featuremap.digest(["hidden.f2.marker"])) + self.assertEqual(ticket["map_ref"], {"map_id": "order-markers.m1", "map_digest": featuremap.map_digest(mapping), "feature_id": "F2"}) + self.assertEqual(ticket["acceptance_criteria"], mapping["features"][1]["criteria"]) + self.assertEqual(ticket["depends_on"], ["F1"]) + with self.assertRaises(Rejected) as refused: + featuremap.compile_ticket(mapping, mapping["features"][0], document, human_owner="") + self.assertEqual(refused.exception.code, "ticket_owner") + degenerate = featuremap.degenerate_map(document, base=base, author=OWNER) + self.assertEqual(featuremap.mapcheck(degenerate, document), [], "one feature per requirement, overlaps declared") + self.assertEqual([f["req_ids"] for f in degenerate["features"]], [["R1"], ["R2"], ["R3"]]) + + +class Policies(unittest.TestCase): + STATE = {"features": {"F1": {"bottom_level": 2, "risk": 0.2, "map_order": 0, "attempts": 0, "attempts_left": 2}, + "F3": {"bottom_level": 2, "risk": 0.1, "map_order": 2, "attempts": 0, "attempts_left": 2}}, + "envelope": {"dollars_left": 0.0, "calls_left": 1, "seconds_left": 10, "smallest_tier_dollars": 0.0, + "arms": [{"id": "cheap", "tier_rank": 0}, {"id": "strong", "tier_rank": 1}], "tiers": {"S": 0, "M": 0, "L": 0}}} + + def test_offline_rules(self): + offline = policymod.OfflinePolicy() + answer = policymod.ask(offline, self.STATE, policymod.Question("next_feature", None, ("F1", "F3"))) + self.assertEqual((answer.value, answer.reason), ("F1", "rule:critical_path_first")) + self.assertEqual(policymod.ask(offline, self.STATE, policymod.Question("proposer_arm", "F1", ("strong", "cheap"))).value, "cheap") + self.assertEqual(policymod.ask(offline, self.STATE, policymod.Question("budget_tier", "F1", ("S", "M"))).value, "S") + self.assertEqual(policymod.ask(offline, self.STATE, policymod.Question("worth_continuing", "F1", ("continue", "stop"))).value, "continue") + state = json.loads(json.dumps(self.STATE)) + state["envelope"]["calls_left"] = 0 + self.assertEqual(policymod.ask(offline, state, policymod.Question("worth_continuing", "F1", ("continue", "stop"))).value, "stop") + state["features"]["F1"]["attempts"] = 1 + self.assertEqual(policymod.ask(offline, state, policymod.Question("after_failure", "F1", policymod.AFTER_FAILURE)).value, "retry_same") + state["features"]["F1"]["attempts"] = 2 + self.assertEqual(policymod.ask(offline, state, policymod.Question("after_failure", "F1", policymod.AFTER_FAILURE)).value, "retry_escalate_arm") + self.assertEqual(policymod.ask(offline, state, policymod.Question("after_failure", "F1", ("retry_same", "escalate_owner", "park"))).value, + "escalate_owner", "attempts left but no stronger arm: the owner decides") + self.assertEqual(policymod.ask(offline, state, policymod.Question("after_failure", "F1", ("escalate_owner", "park"))).value, "park", + "no retry offered: the cap is spent") + + def test_closed_option_sets_are_enforced(self): + with self.assertRaises(Rejected): + policymod.Question("budget_tier", "F1", ("XL",)) + with self.assertRaises(Rejected): + policymod.Question("replan", "F1", ("a",)) + question = policymod.Question("next_feature", None, ("F1", "F3")) + with self.assertRaises(Rejected) as refused: + policymod.validate(question, "F2", "test") + self.assertEqual(refused.exception.code, "policy_invalid") + with self.assertRaises(Rejected): + policymod.validate(question, {"value": "F1", "probabilities": {"F1": 0.9, "F3": 0.3}}, "test") + good = policymod.validate(question, {"value": "F1", "probabilities": {"F1": 0.7, "F3": 0.3}}, "test") + self.assertEqual(good.probabilities["F3"], 0.3) + + def test_shadow_records_disagreement_and_never_acts(self): + recorded = policymod.recorded_answers({"next_feature": {"value": "F3", "probabilities": {"F1": 0.4, "F3": 0.6}}, + "budget_tier:F1": "M", "after_failure": "F9"}) + shadow = policymod.ShadowPolicy(policymod.OfflinePolicy(), recorded, shadow_name="recorded-jev") + answer = policymod.ask(shadow, self.STATE, policymod.Question("next_feature", None, ("F1", "F3"))) + self.assertEqual(answer.value, "F1", "the acting policy's answer stands") + self.assertEqual(answer.shadow, {"policy": "recorded-jev", "value": "F3", "probabilities": {"F1": 0.4, "F3": 0.6}, + "agrees": False, "reason": "recorded"}) + self.assertEqual(shadow.disagreements, 1) + answer = policymod.ask(shadow, self.STATE, policymod.Question("budget_tier", "F1", ("S", "M"))) + self.assertEqual((answer.value, answer.shadow["value"], answer.shadow["agrees"]), ("S", "M", False)) + answer = policymod.ask(shadow, self.STATE, policymod.Question("proposer_arm", "F1", ("cheap",))) + self.assertEqual(answer.shadow["reason"], "shadow_unanswered") + self.assertIsNone(answer.shadow["agrees"]) + state = json.loads(json.dumps(self.STATE)) + state["features"]["F1"]["attempts"] = 1 + answer = policymod.ask(shadow, state, policymod.Question("after_failure", "F1", ("retry_same", "park"))) + self.assertEqual((answer.value, answer.shadow["reason"]), ("retry_same", "policy_invalid"), "an answer outside the options is recorded, not acted on") + self.assertEqual(shadow.disagreements, 2) + + +class Envelope(unittest.TestCase): + def setUp(self): + self.scratch = Path(tempfile.mkdtemp(prefix="envelope-")) + self.addCleanup(shutil.rmtree, self.scratch, True) + self.envelope = loop.new_envelope("m" * 64, dollars_total=1.0, calls_total=3, seconds_total=10.0) + self.spend = loop.EnvelopeLedger(loop.envelope_path(self.scratch, self.envelope), self.envelope) + + def raw(self, **fields) -> None: + line = {"schema": loop.ENVELOPE_LINE_SCHEMA, "envelope_digest": self.spend.digest, **fields} + with self.spend.path.open("a", encoding="utf-8", newline="\n") as handle: + handle.write(canonical(line) + "\n") + + def refuses(self, code, call, *args, **kwargs): + with self.assertRaises(Rejected, msg=f"{args} {kwargs}") as refused: + call(*args, **kwargs) + self.assertEqual(refused.exception.code, code, refused.exception.detail) + + def test_the_kill_switch_path_stays_out_of_the_digest_and_the_file_is_keyed_by_it(self): + with_switch = loop.new_envelope("m" * 64, dollars_total=1.0, calls_total=3, seconds_total=10.0, kill_switch=str(self.scratch / "stop")) + self.assertEqual(loop.envelope_digest(with_switch), self.spend.digest, "a host path is not a term of the grant") + self.assertEqual(self.spend.path, self.scratch / loop.ENVELOPE_DIR / f"{self.spend.digest}.jsonl") + other = loop.new_envelope("m" * 64, dollars_total=1.0, calls_total=4, seconds_total=10.0) + self.assertNotEqual(loop.envelope_path(self.scratch, other), self.spend.path, "another grant, another file") + + def test_reserve_and_settle_are_strict(self): + self.refuses("envelope_conflict", self.spend.settle, "F1", 1, dollars=0.0, calls=1, seconds=1.0, unknown_models=[]) + self.spend.reserve("F1", 1, "S", "standin") + self.refuses("envelope_conflict", self.spend.reserve, "F1", 1, "S", "standin") + for bad in ({"calls": -1}, {"calls": 1.0}, {"seconds": -0.5}, {"seconds": float("nan")}, {"dollars": float("inf")}, + {"dollars": -0.01}, {"calls": True}): + fields = {"dollars": 0.0, "calls": 1, "seconds": 1.0, **bad} + self.refuses("envelope_unpriced", self.spend.settle, "F1", 1, unknown_models=[], **fields) + self.assertEqual([line["kind"] for line in self.spend.lines()], ["reserve"], "a refused settle wrote nothing") + self.spend.settle("F1", 1, dollars=0.25, calls=1, seconds=2.0, unknown_models=[]) + self.refuses("envelope_conflict", self.spend.settle, "F1", 1, dollars=0.0, calls=1, seconds=1.0, unknown_models=[]) + totals = self.spend.totals() + self.assertEqual((totals["calls"], totals["dollars"], totals["seconds"], totals["unsettled"]), (1, 0.25, 2.0, 0)) + self.assertEqual((totals["calls_left"], totals["dollars_left"]), (2, 0.75)) + + def test_a_file_line_that_is_not_a_priced_envelope_line_refuses_the_file(self): + self.spend.reserve("F1", 1, "S", "standin") + cases = [ + ("envelope_unreadable", {"feature": "F1", "attempt": 1, "dollars": 0.0, "calls": 1, "seconds": 0.0}), # no kind: not a KeyError + ("envelope_unreadable", {"kind": "refund", "feature": "F1", "attempt": 1, "dollars": 0.0, "calls": 1, "seconds": 0.0}), + ("envelope_unpriced", {"kind": "settle", "feature": "F1", "attempt": 1, "dollars": -1.0, "calls": 1, "seconds": 0.0}), + ("envelope_unpriced", {"kind": "settle", "feature": "F1", "attempt": 1, "dollars": 0.0, "calls": -2, "seconds": 0.0}), + ("envelope_unpriced", {"kind": "settle", "feature": "F1", "attempt": 1, "dollars": 0.0, "calls": 1, "seconds": -3.0}), + ("envelope_unpriced", {"kind": "reserve", "feature": "F1", "attempt": 2, "dollars": 0.0, "calls": "1", "seconds": 0.0}), + ("envelope_unreadable", {"kind": "reserve", "attempt": 2, "dollars": 0.0, "calls": 1, "seconds": 0.0}), # no feature + ("envelope_unreadable", {"kind": "stop"}), # no reason + ("envelope_conflict", {"kind": "reserve", "feature": "F1", "attempt": 1, "dollars": 0.0, "calls": 1, "seconds": 0.0}), + ("envelope_conflict", {"kind": "settle", "feature": "F2", "attempt": 1, "dollars": 0.0, "calls": 1, "seconds": 0.0}), + ] + good = self.spend.path.read_text(encoding="utf-8") + for code, fields in cases: + self.raw(**fields) + self.refuses(code, self.spend.totals) + self.refuses(code, self.spend.room, "S") + self.spend.path.write_text(good, encoding="utf-8") + self.raw(kind="stop", reason="x", envelope_digest="0" * 64) + self.refuses("envelope_unreadable", self.spend.totals) + self.spend.path.write_text(good, encoding="utf-8") + self.raw(kind="settle", feature="F1", attempt=1, dollars=0.0, calls=1, seconds=0.0) + self.raw(kind="settle", feature="F1", attempt=1, dollars=0.0, calls=1, seconds=0.0) + self.refuses("envelope_conflict", self.spend.totals) + self.spend.path.write_text(good, encoding="utf-8") + self.assertEqual(self.spend.totals()["calls"], 1, "the hold counts until it settles") + + +class EndToEnd(Scratch): + def test_the_thin_slice_builds_three_features_to_ready_for_owner_merge(self): + project, base = self.project() + home = self.home() + document = promoted_spec(project, home) + mapping = fixture_map(document, base) + featuremap.approve(mapping, OWNER) + recorded = policymod.recorded_answers({"next_feature": "F3", "budget_tier": "S", "proposer_arm": "standin"}) + shadow = policymod.ShadowPolicy(policymod.OfflinePolicy(), recorded, shadow_name="recorded-jev") + run = self.make_loop(project, mapping, document, home, policy=shadow) + tickets = run.compile() + self.assertEqual([t["map_ref"]["feature_id"] for t in tickets], ["F1", "F2", "F3"]) + self.assertEqual(run.promotion["keyid"], OWNER_PUBLIC.hex(), "the approval was re-verified under the key the ledger pins") + opened = [e for e in home.events() if e["kind"] == "ticket_opened"] + self.assertEqual([(e["body"]["feature_id"], e["body"]["source"], e["body"]["human_owner"]) for e in opened], + [("F1", "map", "tester"), ("F2", "map", "tester"), ("F3", "map", "tester")]) + frozen = notes(home.events(), "map_frozen") + self.assertEqual(len(frozen), 1) + self.assertEqual(frozen[0]["body"]["approved_by"], OWNER) + self.assertEqual(frozen[0]["body"]["promoted_by"], OWNER) + self.assertEqual(frozen[0]["body"]["refs"]["promotion"]["approval_digest"], run.promotion["approval_digest"]) + self.assertEqual(frozen[0]["body"]["critical_path"], ["F1", "F2"]) + contracts = sorted(p.name for p in (project / ".syberlabs" / "contracts").glob("*.json")) + self.assertEqual(contracts, sorted(f"syberfactory-{t['ticket_id']}.v1.json" for t in tickets)) + contract = json.loads((project / ".syberlabs" / "contracts" / contracts[0]).read_text(encoding="utf-8")) + self.assertEqual(contract["evolution"]["scope"]["paths"], [tickets[[f"syberfactory-{t['ticket_id']}.v1.json" for t in tickets].index(contracts[0])]["footprint"]["files"][0]]) + self.assertEqual(len(contract["evolution"]["evaluation"]["required"]), 2, "the visible check and the hidden check") + + summary = run.run() + self.assertEqual(summary["stopped"], "nothing_eligible") + self.assertEqual(summary["ready_for_owner_merge"], ["F1", "F2", "F3"], summary) + self.assertEqual(summary["envelope"]["calls"], 3) + self.assertEqual(summary["envelope"]["dollars"], 0.0) + self.assertEqual(summary["envelope"]["unsettled"], 0) + events = loop.autonomy_events(home) + kinds = [(e["kind"] if e["kind"] != "note" else e["body"]["note"]) for e in events] + picks = [e["body"]["answer"]["value"] for e in notes(events, "policy_proposal") if e["body"]["question"] == "next_feature"] + self.assertEqual(picks, ["F1", "F3", "F2"], "critical path first: F1 (bottom level 2, a dependent), then F3 (M), then F2") + shadows = [e["body"]["answer"]["shadow"] for e in notes(events, "policy_proposal") if e["body"]["question"] == "next_feature"] + self.assertEqual([s["agrees"] for s in shadows], [False, True, None], "the shadow disagreed on the first pick and was out of options on the last") + self.assertEqual(shadows[2]["reason"], "policy_invalid") + steps = [(e["body"]["feature"], e["body"]["step"], e["body"]["outcome"]) for e in notes(events, "thread_step")] + self.assertEqual(steps, [("F1", "check", "acceptable"), ("F1", "accept", "succeeded"), + ("F3", "check", "acceptable"), ("F3", "accept", "succeeded"), + ("F2", "check", "acceptable"), ("F2", "accept", "succeeded")]) + self.assertEqual(kinds[:4], ["map_frozen", "ticket_opened", "ticket_opened", "ticket_opened"]) + self.assertEqual(kinds[-1], "loop_stopped") + self.assertEqual(kinds.count("feature_ready"), 3) + self.assertEqual(kinds.count("feature_failed"), 0) + self.assertTrue(home.verify()) + for event in notes(events, "feature_ready"): + self.assertTrue(event["body"]["refs"]["target_ref"].startswith("refs/heads/syberlabs/")) + self.assertEqual(event["body"]["next"], "owner merges; nothing here merges") + self.assertEqual(git(project, "rev-parse", "main"), base, "main never moved") + # The owner's checkout is as it was: HEAD on main at the base commit, a clean tree, no scratch branch left. + self.assertEqual(git(project, "symbolic-ref", "HEAD"), "refs/heads/main") + self.assertEqual(git(project, "rev-parse", "HEAD"), base) + self.assertEqual(git(project, "status", "--porcelain"), "") + self.assertEqual([b for b in git(project, "branch", "--list", "syberfactory/*").split() if b], []) + f2 = summary["features"]["F2"]["commit"] + shown = git(project, "show", f"{f2}:fxapp/rules.py") + self.assertIn("# F1 delivered", shown, "F2 stacked on F1's accepted commit") + walk = loop.why(home, tickets[1]["ticket_id"]) + self.assertEqual((walk["feature"], walk["requirements"], walk["map"]["approved_by"]), ("F2", ["R2"], OWNER)) + self.assertEqual(walk["spec"]["digest"], specmod.spec_digest(document)) + self.assertEqual(walk["spec"]["promoted_by"], OWNER) + self.assertEqual(walk["spec"]["promotion"]["seq"], run.promotion["seq"]) + self.assertEqual(walk["ready_for_owner_merge"]["commit"], f2) + self.assertEqual(run.spend.totals()["stopped"], "nothing_eligible") + self.assertEqual(run.spend.path.parent.name, loop.ENVELOPE_DIR) + # The same list, for the report: every autonomy event in order. + print("\nledger events:", " ".join(kinds)) + + def test_same_fixture_gives_the_same_event_digests(self): + digests = [] + for name in ("one", "two"): + run, _, _, _ = self.ready_loop(name, f"ledger-{name}") + run.run() + digests.append(loop.event_digests(run.ledger)) + self.assertEqual(len(digests[0]), len(digests[1])) + self.assertEqual(digests[0], digests[1], "no timestamp, thread id, commit or approval digest enters an event digest") + self.assertGreater(len(digests[0]), 20) + + def test_compile_needs_the_ledgers_promotion_record_not_the_files_state(self): + """The file's ``state`` decides nothing: the ledger must hold a promote of exactly this spec, citing a + recorded approval by a human, whose envelope is stored and verifies under the owner key the ledger pinned.""" + project, base = self.project() + home = self.home() + in_memory = candidate_spec(project) + payload = specmod.approval_payload(in_memory, "promote", OWNER, verdict_ids=[VERDICT]) + approval = specmod.verify_approval(specmod.sign_approval(payload, OWNER_SEED), OWNER_PUBLIC, in_memory, "promote", verdict_ids=[VERDICT]) + specmod.transition(in_memory, "promote", OWNER, verdict_ids=[VERDICT], approval=approval) + self.assertEqual(in_memory["state"], "promoted", "the file says promoted") + mapping = fixture_map(in_memory, base) + featuremap.approve(mapping, OWNER) + run = self.make_loop(project, mapping, in_memory, home) + with self.assertRaises(Rejected) as refused: + run.compile() + self.assertEqual(refused.exception.code, "spec_unpromoted", "promoted in memory, never on the ledger") + self.assertEqual(loop.autonomy_events(home), [], "nothing was logged") + + document = promoted_spec(project, home) + edited = json.loads(json.dumps(document)) + edited["capability"] = "edited after promotion" + mapping = fixture_map(edited, base) + featuremap.approve(mapping, OWNER) + with self.assertRaises(Rejected) as refused: + self.make_loop(project, mapping, edited, home).compile() + self.assertEqual(refused.exception.code, "spec_unpromoted", "the file is not the spec the ledger promoted") + + mapping = fixture_map(document, base) + featuremap.approve(mapping, OWNER) + # A promotion whose approval was signed under a key the ledger never pinned, or on a ledger that pins no + # key, is a record without a trust root: refused whatever the file and the events say. + other_seed, other_public = generate() + for name, pin, code in (("unpinned", False, "owner_key_unset"), ("other-key", True, "owner_key_mismatch")): + forged_home = self.home(name) + if pin: + forged_home.set_owner_key(OWNER_PUBLIC) + forged = candidate_spec(project) + envelope = specmod.sign_approval(specmod.approval_payload(forged, "promote", OWNER, verdict_ids=[VERDICT]), other_seed) + approval = specmod.verify_approval(envelope, other_public, forged, "promote", verdict_ids=[VERDICT]) + record = specmod.transition(forged, "promote", OWNER, verdict_ids=[VERDICT], approval=approval) + event = forged_home.record_approval(envelope, approval) + record["approval"] = {**record["approval"], "event_hash": event["hash"], "approval_digest": event["body"]["approval_digest"]} + forged_home.append("spec_changed", {"spec_id": forged["spec_id"], "version": forged["version"], **record, + "spec_digest": specmod.spec_digest(forged), "footprint": forged["footprint"]["components"]}) + self.assertEqual(specmod.spec_digest(forged), specmod.spec_digest(document), "the same spec, differently approved") + with self.assertRaises(Rejected, msg=name) as refused: + self.make_loop(project, mapping, forged, forged_home).compile() + self.assertEqual(refused.exception.code, code, name) + + approvals = next(p for p in (home.home / "approvals").glob("*.json")) + kept = approvals.read_text(encoding="utf-8") + approvals.write_text(kept.replace('"', "'", 1), encoding="utf-8") + with self.assertRaises(Rejected) as refused: + self.make_loop(project, mapping, document, home).compile() + self.assertEqual(refused.exception.code, "spec_unpromoted", "a damaged approval envelope refuses") + approvals.write_text(kept, encoding="utf-8") + + run = self.make_loop(project, mapping, document, home) + run.compile() + self.assertEqual(run.promotion["approver"], OWNER) + self.assertEqual(run.promotion["verdict_ids"], [VERDICT]) + self.assertEqual(run.promotion["keyid"], OWNER_PUBLIC.hex()) + self.assertEqual(loop.promotion_record(home, document)["seq"], run.promotion["seq"]) + + def test_compile_reads_the_owner_key_through_the_ledgers_lineage(self): + """M3 of the #164 verification, on the autonomy side: the owner key is the ledger's verified lineage, read + with the ledger's stored rotations. A spec promoted after an honest rotation (bootstrap -> rotation signed + by the first key, the approval signed by the second) compiles under the second key; that lineage cannot be + read without the loader. A bare second ``owner_key_set`` appended through the public ``append`` after the + promotion (the hash chain intact) leaves the ledger pinning nothing: ``compile`` refuses + ``owner_key_lineage``, not ``owner_key_unset`` and not the earlier key, and writes nothing.""" + project, base = self.project() + home = self.home() + second_seed, second_public = generate() + home.set_owner_key(OWNER_PUBLIC) + home.set_owner_key(second_public, seed=OWNER_SEED) + self.assertEqual(home.owner_key(), second_public.hex()) + document = promoted_spec(project, home, seed=second_seed, public=second_public) + with self.assertRaises(Rejected) as refused: + specmod.pinned_owner_key(home.events()) + self.assertEqual(refused.exception.code, "owner_key_lineage", "a rotated ledger is unreadable without the loader") + mapping = fixture_map(document, base) + featuremap.approve(mapping, OWNER) + run = self.make_loop(project, mapping, document, home) + run.compile() + self.assertEqual(run.promotion["keyid"], second_public.hex(), "the key pinned before the approval, through the rotation") + self.assertTrue(home.verify()) + self.assertEqual(len(notes(home.events(), "map_frozen")), 1) + + # The M3 repro: an honest bootstrap and promotion, then a bare owner_key_set appended with the chain intact. + project, base = self.project("tampered") + tampered = self.home("tampered-ledger") + document = promoted_spec(project, tampered) + _, attacker_public = generate() + tampered.append("owner_key_set", {"keyid": attacker_public.hex(), "previous": None, "rotation_digest": None}) + self.assertTrue(ledger._verify_chain(tampered.events()), "the chain is intact; only the lineage is broken") + self.assertFalse(tampered.verify()) + mapping = fixture_map(document, base) + featuremap.approve(mapping, OWNER) + before = len(tampered.events()) + with self.assertRaises(Rejected) as refused: + self.make_loop(project, mapping, document, tampered).compile() + self.assertEqual(refused.exception.code, "owner_key_lineage") + self.assertIn("a bare pin does not re-root the ledger", refused.exception.detail) + self.assertEqual(loop.autonomy_events(tampered), [], "nothing was logged") + self.assertEqual(len(tampered.events()), before, "the ledger was not written") + self.assertFalse((project / ".syberlabs" / "contracts").exists(), "no contract was written") + + def test_a_map_without_owner_approval_is_refused(self): + project, base = self.project() + home = self.home() + document = promoted_spec(project, home) + mapping = fixture_map(document, base) + run = self.make_loop(project, mapping, document, home) + with self.assertRaises(Rejected) as refused: + run.compile() + self.assertEqual(refused.exception.code, "map_unapproved") + self.assertEqual(loop.autonomy_events(home), [], "nothing was logged") + featuremap.approve(mapping, OWNER) + mapping["features"][0]["depends_on"] = ["F2"] + run = self.make_loop(project, mapping, document, home) + with self.assertRaises(Rejected) as refused: + run.compile() + self.assertEqual(refused.exception.code, "map_unapproved", "an edit after approval leaves the approval stale") + featuremap.approve(mapping, OWNER) + run = self.make_loop(project, mapping, document, home) + with self.assertRaises(Rejected) as refused: + run.compile() + self.assertEqual(refused.exception.code, "map_invalid") + self.assertIn("cycle", refused.exception.detail) + + def test_the_envelope_stops_the_loop(self): + run, _, _, _ = self.ready_loop(calls_total=2) + summary = run.run() + self.assertEqual(summary["stopped"], "envelope_exhausted") + self.assertEqual(summary["ready_for_owner_merge"], ["F1", "F3"]) + self.assertEqual(summary["features"]["F2"]["state"], "open", "F2 was eligible and waits for a new envelope") + self.assertEqual(summary["envelope"]["calls_left"], 0) + self.assertEqual(run.spend.totals()["stopped"], "envelope_exhausted") + self.assertEqual([line["kind"] for line in run.spend.lines()], ["reserve", "settle", "reserve", "settle", "stop"]) + + def test_two_maps_in_one_home_keep_their_own_envelopes(self): + """One file per envelope digest: a second map (or a second grant) in the same factory home starts + with its own spend and no inherited stop; the same spent grant is refused, not silently re-run.""" + project_a, base = self.project("a") + home = self.home() + document = promoted_spec(project_a, home) + first = fixture_map(document, base) + featuremap.approve(first, OWNER) + run_a = self.make_loop(project_a, first, document, home) + self.assertEqual(run_a.run()["stopped"], "nothing_eligible") + project_b, base_b = self.project("b") + self.assertEqual(base_b, base, "the fixture's base commit is deterministic") + second = fixture_map(document, base, id="order-markers.m2") + featuremap.approve(second, OWNER) + run_b = self.make_loop(project_b, second, document, home) + self.assertNotEqual(run_b.spend.path, run_a.spend.path) + summary = run_b.run() + self.assertEqual(summary["stopped"], "nothing_eligible") + self.assertEqual(summary["ready_for_owner_merge"], ["F1", "F2", "F3"], "the second map built; it inherited no stop and no spend") + self.assertEqual(summary["envelope"]["calls"], 3) + self.assertEqual([line["kind"] for line in run_b.spend.lines()], ["reserve", "settle"] * 3 + ["stop"]) + self.assertEqual(len(notes(home.events(), "loop_stopped")), 2) + again = self.make_loop(project_a, first, document, home) + with self.assertRaises(Rejected) as refused: + again.run() + self.assertEqual(refused.exception.code, "envelope_stopped", "the first grant is spent; a new run needs a new envelope") + self.assertEqual(len(notes(home.events(), "loop_stopped")), 2, "a refused re-run writes nothing") + self.assertEqual([line["kind"] for line in run_a.spend.lines()].count("stop"), 1) + + def test_await_merge_leaves_a_dependent_waiting_for_the_owner(self): + run, _, _, _ = self.ready_loop(map={"integration": "await_merge"}) + summary = run.run() + self.assertEqual(summary["ready_for_owner_merge"], ["F1", "F3"]) + self.assertEqual(summary["features"]["F2"]["state"], "waiting_owner_merge") + self.assertEqual(summary["stopped"], "nothing_eligible") + + def test_a_failing_candidate_is_refused_by_the_kernel_and_the_feature_is_parked(self): + run, _, _, _ = self.ready_loop(attempts=1, mode="--break") + summary = run.run() + self.assertEqual(summary["ready_for_owner_merge"], []) + self.assertEqual({row["state"] for row in summary["features"].values()} - {"open"}, {"parked"}) + self.assertTrue(summary["features"]["F1"]["last_reason"].startswith("candidate_check_failed:"), summary["features"]["F1"]) + self.assertIn(summary["features"]["F1"]["last_reason"].split(":")[1], ("tests", "hidden.f1.marker"), "both checks fail; one is named") + self.assertEqual(summary["features"]["F2"]["state"], "open", "F2 waits on F1 and never started") + events = loop.autonomy_events(run.ledger) + questions = [(e["body"]["question"], e["body"]["answer"]["value"]) for e in notes(events, "policy_proposal") if e["body"]["feature"] == "F1"] + self.assertEqual(questions[-2:], [("worth_continuing", "stop"), ("after_failure", "park")]) + self.assertEqual(summary["envelope"]["calls"], 2, "one call per parked feature; the envelope counted both") + + def test_retry_escalate_arm_runs_the_stronger_arm_on_the_next_attempt(self): + """The recorded decision is the one that happens: after ``retry_escalate_arm`` the next ``proposer_arm`` + offers only the arms ranked above the one that failed, and the attempt runs on that arm.""" + arms = [{"id": "cheap", "tier_rank": 0, "revision": "1"}, {"id": "strong", "tier_rank": 1, "revision": "1"}] + run, _, _, _ = self.ready_loop(attempts=3, mode="--break", arms=arms) + summary = run.run() + self.assertEqual(summary["features"]["F1"]["state"], "parked") + self.assertEqual(summary["features"]["F1"]["attempts"], 3) + events = loop.autonomy_events(run.ledger) + f1 = lambda note: [e["body"] for e in notes(events, note) if e["body"].get("feature") == "F1"] + self.assertEqual([b["arm"] for b in f1("attempt_started")], ["cheap", "cheap", "strong"]) + self.assertEqual([b["arm"] for b in f1("envelope_reserved")], ["cheap", "cheap", "strong"]) + self.assertEqual([b["answer"]["value"] for b in f1("policy_proposal") if b["question"] == "after_failure"], + ["retry_same", "retry_escalate_arm", "park"]) + offered = [b["options"] for b in f1("policy_proposal") if b["question"] == "proposer_arm"] + self.assertEqual(offered, [["cheap", "strong"], ["cheap", "strong"], ["strong"]], "after the escalation only the stronger arm is an option") + choices = [b["options"] for b in f1("policy_proposal") if b["question"] == "after_failure"] + self.assertNotIn("retry_escalate_arm", choices[2], "on the strongest arm there is nothing to escalate to") + self.assertEqual([line["arm"] for line in run.spend.lines() if line["kind"] == "reserve" and line["feature"] == "F1"], + ["cheap", "cheap", "strong"]) + + def test_a_candidate_outside_its_scope_never_reaches_accept(self): + run, _, _, _ = self.ready_loop(attempts=2, mode="--outside") + summary = run.run() + self.assertEqual(summary["ready_for_owner_merge"], []) + self.assertEqual(summary["features"]["F1"]["last_reason"], "candidate_out_of_scope") + self.assertEqual(summary["features"]["F1"]["attempts"], 2, "retry_same once, then parked") + self.assertEqual(summary["features"]["F1"]["state"], "parked") + + def test_the_kill_switch_stops_before_the_next_decision(self): + switch = self.scratch / "stop-now" + switch.write_text("") + run, _, _, _ = self.ready_loop(kill_switch=str(switch)) + summary = run.run() + self.assertEqual(summary["stopped"], "kill_switch") + self.assertEqual(summary["ready_for_owner_merge"], []) + self.assertEqual(summary["envelope"]["calls"], 0) + + def test_a_rejection_inside_run_still_ends_the_record(self): + """A ``Rejected`` while running (here an answer outside the options) propagates, but not before the + envelope has its ``stop`` line and the ledger its ``loop_stopped`` with the refusal's code.""" + class Rogue: + name = "rogue" + + def decide(self, state, question): + return policymod.Answer(question.key, "F9", self.name, "off the menu") + + run, _, _, _ = self.ready_loop(policy=Rogue()) + with self.assertRaises(Rejected) as refused: + run.run() + self.assertEqual(refused.exception.code, "policy_invalid") + events = loop.autonomy_events(run.ledger) + last = events[-1]["body"] + self.assertEqual((last["note"], last["reason"]), ("loop_stopped", "policy_invalid")) + self.assertIn("rogue answered 'F9'", last["refs"]["detail"]) + self.assertEqual([(line["kind"], line["reason"]) for line in run.spend.lines()], [("stop", "policy_invalid")]) + self.assertEqual(run.spend.totals()["stopped"], "policy_invalid") + self.assertTrue(run.ledger.verify()) + + def test_a_dirty_tree_refuses_the_stacked_start_and_leaves_the_owners_checkout_alone(self): + """Stacking moves HEAD only for the kernel's start and puts it back; a tree with uncommitted tracked + changes is refused before any checkout (``repository_dirty``), the hold stays counted, the record ends.""" + run, project, base, _ = self.ready_loop() + (project / "fxapp" / "main.py").write_text((project / "fxapp" / "main.py").read_text(encoding="utf-8") + "# owner's unsaved work\n", + encoding="utf-8") + with self.assertRaises(Rejected) as refused: + run.run() + self.assertEqual(refused.exception.code, "repository_dirty") + self.assertEqual([fid for fid, row in run.state.items() if row["state"] == "ready_for_owner_merge"], ["F1", "F3"], + "the two unstacked features built; F2's stacked start was refused") + self.assertEqual(git(project, "symbolic-ref", "HEAD"), "refs/heads/main") + self.assertEqual(git(project, "rev-parse", "HEAD"), base) + self.assertIn("# owner's unsaved work", (project / "fxapp" / "main.py").read_text(encoding="utf-8"), "nothing was lost") + self.assertEqual(git(project, "status", "--porcelain", "--untracked-files=no").split(), ["M", "fxapp/main.py"]) + totals = run.spend.totals() + self.assertEqual((totals["stopped"], totals["attempts"], totals["unsettled"]), ("repository_dirty", 3, 1), "the refused attempt's hold counts") + last = loop.autonomy_events(run.ledger)[-1]["body"] + self.assertEqual((last["note"], last["reason"], last["states"]["F2"]), ("loop_stopped", "repository_dirty", "open"), + "refused before its thread started, F2 is still open for a clean tree and a new envelope") + + def test_the_slice_holds_no_merge_and_no_network(self): + for name in ("featuremap", "policy", "loop", "autonomy_cli"): + text = (ROOT / "syberlabs" / "factory" / f"{name}.py").read_text(encoding="utf-8") + for forbidden in ('"merge"', "merge_pull", "git_push", "github_pull_request", "urllib", "http.client", "socket", "requests"): + self.assertNotIn(forbidden, text, f"{name}.py holds {forbidden}") + with self.assertRaises(Rejected): + loop.record(ledger.Ledger(self.scratch / "l"), "merged", "x", {}) + self.assertNotIn("merge", loop.ACTIONS) + self.assertNotIn("publish", " ".join(loop.ACTIONS)) + + +if __name__ == "__main__": + unittest.main()