Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions docs/development/testing-and-quality.md
Original file line number Diff line number Diff line change
Expand Up @@ -1313,6 +1313,24 @@ fields; this does not introduce a new telemetry system or receipt gate.

### Budget Failure Decisions

The fixed visible-Goal host fixture's 3,200-character regression guard predates
`1d5d2d0f6`, which adds the known Todo choice to the first current guard. On the
same four host inputs at `b201da8ba`, substituting only the pre-change dispatch
rule gives 3,199 / 3,177 / 2,966 / 2,895 normalized body characters; the complete
current rule gives 3,262 / 3,240 / 3,029 / 2,958. The 63-character increase teaches
an already-known choice before admission, while retaining selection/reentry,
ordered settlement, exact identities, actual outcomes and ambiguous-effect
recovery. Preserve the original 3,200-ceiling failure; calibrate only this test's
regression allowance to 3,300, leaving 38 characters above the largest fixed
input. Production prompt bytes, the 4,000-character host limit, longer-route
checks, execution authority and frozen qualification results remain unchanged.
This is no output reduction or model-compliance qualification.

相同四个宿主输入中,首次 guard 的已知 Todo 选择指引增加 63 个归一化字符,
最大值从 3,199 变为 3,262。保留原 3,200 超限失败,仅将该固定测试的回归预算
校准为 3,300,留 38 字符余量;不删指令,不改生产正文、4,000 宿主硬边界、
长路径检查、执行权限或冻结资格结果,也不据此声称输出节省或模型遵从。

The actual-default model portfolio retains the complete capability candidate
context required by the shared diagnostic projection. On the unchanged three
noisy fixtures, `74b3d526ab1c1fd3d5f96308bef2482fcad7e636` emitted 4,944 /
Expand Down
42 changes: 21 additions & 21 deletions tests/control_plane/test_heartbeat_brief_budget.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
from __future__ import annotations

import contextlib
import hashlib
import io
import json
import shlex
Expand Down Expand Up @@ -93,26 +92,27 @@ def test_brief_keeps_complete_routes_with_128_character_windows_root() -> None:
assert json.loads(encoded) == payload
root, _, _, _ = _windows_routes()
assert encoded.count(json.dumps(root)[1:-1]) == 20
# The unchanged fixture on main b5a1a816a measures 10,522 JSON chars and
# 8,579 Markdown. The complete outcome paragraph adds 523 body/Markdown
# chars; omitting absent brief metadata saves 35 JSON chars: 11,010/9,102.
# The later diagnostic-only renderer change retains that full body and JSON,
# prints an identical active/resolved path once and omits absent pre-quota
# metadata: Markdown is 8,867 chars. The expected output was derived from
# b16aeb3's frozen 9,102-char rendering using only those two substitutions.
# These hashes retain every prior clause and route, plus the reviewed
# instruction transition. This is a synthetic spelling, not a Windows run.
assert hashlib.sha256(payload["task_body"].encode()).hexdigest() == (
"cfc65091d2710ef298174e6a5280edf55f6cc7acdce8e5688a2ae213b5f4b4ac"
)
assert hashlib.sha256(markdown.encode()).hexdigest() == (
"e5c10d37ed7cef08e8fa56a450f3c119de92d1ef99e3516ff6b106a8853c8186"
)
# fd1b23b59 raised the existing body allowance, preserving every clause.
# Only that diagnostic number differs from the earlier frozen rendering.
assert hashlib.sha256(markdown.replace("max_chars=`5400`", "max_chars=`4800`").encode()).hexdigest() == (
"29604e41d97963976ab8911e69c6328421aeaf63053cdb12c197dbe9b5291b9b"
)
# Assert the decision obligations rather than a hash of dated prose.
# The full body must reach Markdown, with its executable guard only once.
body = payload["task_body"]
assert body in markdown
assert markdown.count(payload["quota_guard_command"]) == 1
for obligation in (
"true=work+writeback; only false permits no-op",
"Read agent_channel.work_context and required_reads (or envelope) before work",
"retain original and cumulative outcome requirements",
"reuse inline goal_acceptance, inspect only if absent, then refine",
"Known new-Turn choice: add `--todo-id <id>` to first guard; otherwise",
"use selection_command when required; re-enter as instructed, do/verify authorized work",
"Settle with current settlement_plan.ordered_steps, else next_cli_actions",
"preserve exact identities/effect ids and actual outcomes",
"Never reconstruct static refresh/spend; ambiguous effects need readback/recovery, not blind retry",
"Todo done≠Turn settled; no invented successor/--no-follow-up",
"No wait spend",
"Finish Goal only on terminal no-follow-up",
"Destructive Git/production requires explicit authorization",
):
assert obligation in body, obligation
spec = CLI_OUTPUT_MODE_VARIANT_BY_ID["heartbeat_prompt_brief"]
assert spec.max_chars == {"json": 11_400, "markdown": 9_600}
for output_format, text in (("json", encoded), ("markdown", markdown)):
Expand Down
2 changes: 1 addition & 1 deletion tests/test_chat_turn_steering.py
Original file line number Diff line number Diff line change
Expand Up @@ -397,7 +397,7 @@ def record_write(self, payload):
initial_text = starts[0]["input"][0]["text"]
assert "LoopX context (supporting context only):" in initial_text
assert "Inspect a bounded task." in initial_text
assert initial_text.endswith("Current user message: wait for steer")
assert initial_text.endswith("Current user message:\nwait for steer")
updates = [row["params"] for row in writes if row.get("method") == "turn/steer"]
assert updates == [{
"threadId": session.thread_id,
Expand Down
9 changes: 5 additions & 4 deletions tests/test_host_loop_activation.py
Original file line number Diff line number Diff line change
Expand Up @@ -450,10 +450,11 @@ def test_goal_hosts_reuse_thin_dispatch_and_stay_compact() -> None:
"preserve exact identities/effect ids",
):
assert obligation in payload["task_body"]
# Current authority/continuation clauses fit 3,200 normalized
# chars on this fixed fixture. Keep a tight regression guard and the
# native host's separate hard cap, without shortening those obligations.
assert payload["interface_budget"]["budget_char_count"] <= 3_200
# The known first-guard choice adds 63 useful normalized characters:
# the largest fixed host fixture is 3,262, leaving 38 under this
# regression allowance. Keep the native host's hard cap separate.
assert "Known new-Turn choice: add `--todo-id <id>` to first guard" in payload["task_body"]
assert payload["interface_budget"]["budget_char_count"] <= 3_300
assert payload["interface_budget"]["max_chars"] == 4_000
assert payload["interface_budget"]["within_budget"] is True

Expand Down
Loading