Skip to content

Gates: human gate approves the content past agent verdicts; agent-gate budget from maxTokens, cut-off answers escalate - #63

Merged
0design merged 4 commits into
mainfrom
claude-qf/gate-subject-and-verdict-budget-20261024
Oct 24, 2026
Merged

0design merged 4 commits into
mainfrom
claude-qf/gate-subject-and-verdict-budget-20261024

Conversation

@0design

@0design 0design commented Oct 24, 2026

Copy link
Copy Markdown
Owner

Why

Two findings from real runs of the "… Factory" workflows on main:

  1. Agent-gate answers were capped at 300 tokens. A reviewer that wrote a longer explanation ended in OUTPUT_LIMIT (completion truncated) and the whole run failed, instead of handing the decision to a person.
  2. A human gate after an agent gate showed and hashed the agent's verdict, not the content. The person approved {pass, reason} without seeing the text that goes on to delivery.

What changes

Gate subject

  • A human approval-gate now passes over agent gates directly before it: its subject (and its approvalHash under bind: sha256) is the content those gates judged.
  • The agent verdicts are listed beside the subject as reviews. They are not part of the subject or its hash. q-core approve prints them under the subject.
  • subject: previous keeps the legacy subject (the output directly before the gate, even an agent verdict). Validation accepts only that value, and only on a human gate.
  • One selector (currentSubjectRow / gateSubjectRow in run.mjs) feeds the gate, its bound row and the approve-time check in waitingGate. Before this, waitingGate had its own filter, which also did not skip workflow-return rows.
  • Runs already parked keep their recorded boundSubjectRowId, so their existing hash is still accepted.
  • An escalated agent gate (failed check, cut-off or unreadable answer, missing key) now shows the candidate it judged as subject, bound under sha256. Before, it showed nothing.
  • The GATE_SUBJECT_PARTIAL validator warning applies the same rule. A human gate after agent gates is judged against the content, where before it was skipped.

Agent-gate budget

  • The answer budget is the gate's maxTokens, an integer from 1 to 32000, default 1024. It replaces the fixed Math.min(maxTokens, 300). The Anthropic worst-case reservation uses the same value.
  • An answer cut off at maxTokens (OUTPUT_LIMIT, for both providers) is no verdict. Like an unreadable verdict, it follows escalateOn: by default it goes to a person, and it keeps its billed tokens and cost. Every other provider failure still fails: cost cap, HTTP, invalid response, cancellation.

Docs and pins

  • SPEC-MANIFEST §5.4 and CHANGELOG (Unreleased).
  • core.schemaSha256 updated for the edited SPEC-MANIFEST.
  • Core artifact repinned (registry/catalog*.json, SHA256SUMS), checked with build-registry --check.

Tests

  • New file test/gate-subject-and-verdict-budget.test.mjs, 11 tests. Run against current main, 7 of them fail: subject and hash, escalated subject, maxTokens and default, cut-off escalation and bill, escalateOn for cut-off, validation, warning. The other 4 are compatibility guards and pass on both: the writer fixture, subject: previous, a run parked before the change, and an unreadable verdict escalating.
  • One existing test changed: test/anthropic-agent-gate.test.mjs, "…bound uses its actual 300-token completion cap" → "…uses its configured completion cap". A configured 800 is now sent as-is, and 32000 is refused by the run budget before any call.
  • Full suite locally, branch vs main: the failure lists are identical (comm -23 is empty). The shared local failures come from running inside an agent session and do not occur in CI.

Not in this PR (open questions)

🤖 Generated with Claude Code

0design and others added 4 commits October 24, 2026 14:16
…ent-gate answer budget from maxTokens, cut-off answers escalate

- A human approval-gate's subject passes over agent gates directly before it, so the
  person sees and binds (sha256) the step output that goes on to delivery. Agent
  verdicts are shown beside it as `reviews`, outside the subject and its hash.
  `subject: previous` keeps the legacy subject; parked runs keep their recorded row.
  One subject selector serves the gate, its binding and the approve-time check.
- An escalated agent gate shows the candidate it judged as `subject` (bound under sha256).
- Agent-gate answer budget is `maxTokens` (1..32000, default 1024) instead of a fixed 300.
  An OUTPUT_LIMIT answer is no verdict: it follows escalateOn like a failed check and
  keeps its billed usage. Other provider failures still fail.
- GATE_SUBJECT_PARTIAL judges a human gate after agent gates against the content.
- Docs (SPEC-MANIFEST 5.4, CHANGELOG), schema pin and Core artifact repin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ails; repin

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd child budgets stop an escalation

- The output a step goes on from (a delivery without body, an agent gate's
  candidate, a human gate's subject) is the last one in execution order, not
  the order of step IDs as object keys. Integer-like IDs such as "0" no longer
  let a delivery send a text other than the one a person approved.
- An agent gate whose call takes the run or an enclosing child workflow over
  its budgetUsd (or leaves its spend unknown under one) stops the run by
  budget before any waiting_human state is written or reported, whatever the
  answer: cut off, objection or unreadable.
- Tests fail on the previous head and pass here; repin SPEC-MANIFEST and the
  Core artifact.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@0design
0design merged commit 0c2593c into main Oct 24, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant