Skip to content

Commit a7fed81

Browse files
Add a propose kind: the model authors from what the registry read (#30)
plan_docs could read and summarise but a goal ending "and then create a plan for merging them" was satisfied by the mechanical goal check without the proposal ever existing. The fourth kind closes that: its body hands the deterministic notes to the run's model and records the recommendation as `proposal (model-authored):` — the one body in this registry that speaks to a model, built per-run by build_registry(model) via the CLI's existing factory contract. Honesty at the edges: with no real model behind the run (None, or the planner's own ScriptedChatModel, whose replies a body must not consume) the note says authoring needs --model instead of pretending; a model call that raises becomes a note too, because the deterministic reads already in state are worth keeping when the proposal step cannot run. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 53f7de0 commit a7fed81

3 files changed

Lines changed: 153 additions & 32 deletions

File tree

docs/cookbook/07-slack.md

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -121,17 +121,20 @@ The default planning registry is the incident-response demo: its node bodies
121121
are stubs, so a goal like "summarise the docs here" gets an honest negative
122122
(or, if phrased vaguely enough, a hollow success). The shipped alternative
123123
has bodies that really read — `survey` / `read` / `summarise`, read-only,
124-
confined to the bot's working directory:
124+
confined to the bot's working directory — plus one kind, `propose`, whose
125+
body hands what was read to the model and records the recommendation that
126+
comes back, labelled `proposal (model-authored):`:
125127

126128
```
127-
@grapharc plan "summarise the docs in this workspace" --registry grapharc.examples.plan_docs:build_registry --trace docs.jsonl --run-id docs-1
129+
@grapharc plan "summarise the docs and propose how to merge them" --registry grapharc.examples.plan_docs:build_registry --model claude-cli/claude-sonnet-5 --trace docs.jsonl --run-id docs-1
128130
```
129131

130-
Works with the scripted planner (free) and with `--model`; either way the
131-
notes in the final state carry actual file names, titles and excerpts,
132-
because the reading is operator code, not model output. `--registry` from
133-
Slack accepts exactly these two shipped modules and nothing else — the flag's
134-
general form imports arbitrary code, which stays refused.
132+
The scripted planner (no `--model`) runs the same chain free: the reading
133+
notes are identical — file names, titles, excerpts are operator code, not
134+
model output — and the `propose` note says plainly that authoring needs a
135+
real model rather than pretending. `--registry` from Slack accepts exactly
136+
these two shipped modules and nothing else — the flag's general form imports
137+
arbitrary code, which stays refused.
135138

136139
## The `agent` opt-in
137140

grapharc/examples/plan_docs.py

Lines changed: 93 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -4,16 +4,21 @@
44
like "summarise the docs in this workspace" gets either an honest negative or
55
a hollow success, depending on how vague the goal is. This registry closes
66
that gap for one bounded job: **reading documentation under the current
7-
working directory and reporting what is there.** Three kinds:
7+
working directory and reporting what is there.** Four kinds:
88
99
survey list the documentation files under cwd
1010
read read them and record an excerpt of each
1111
summarise one note distilling title + first paragraph per file
12+
propose the model authors a recommendation from what was read
1213
13-
Everything is deterministic operator code. The model still does the
14-
*planning* — which kinds, in what order, replanning on refusal — but no node
15-
body contains a model call, so a run with a scripted planner produces real
16-
file content and a run with `--model` spends tokens only on rounds.
14+
The first three are deterministic operator code. The model still does the
15+
*planning* — which kinds, in what order, replanning on refusal — and those
16+
bodies never call it, so a scripted run produces real file content and spends
17+
nothing. `propose` is the one deliberate exception: its body hands the notes
18+
the deterministic kinds collected to the model and records what comes back,
19+
labelled as the model's. With no real model behind the run it says so in its
20+
note instead of pretending — a scripted planner cannot author a proposal, and
21+
the note is honest about which kind of run this was.
1722
1823
The confinement matters more than the capability: bodies read, never write,
1924
and only under `Path.cwd()` — which, launched from the Slack bot, is the
@@ -97,43 +102,107 @@ def _summarise_body(state: DocsState) -> dict:
97102
return {"notes": [*state.notes, summary]}
98103

99104

100-
_BODIES = {"survey": _survey_body, "read": _read_body, "summarise": _summarise_body}
101-
105+
def _propose_body_for(model: Any) -> Any:
106+
"""The one body with a model in it, built per-run so it closes over the
107+
run's own backend rather than importing one.
108+
109+
`model` is whatever drove the planner. A scripted planner's model cannot
110+
author anything (invoking it would consume the planner's own scripted
111+
replies), so anything without a usable `invoke` — or a call that fails —
112+
becomes an honest note instead of an exception: the deterministic notes
113+
already in state are worth keeping even when the proposal step cannot run.
114+
"""
115+
116+
def body(state: DocsState) -> dict:
117+
evidence = "\n".join(state.notes)
118+
real = model is not None and not isinstance(model, _scripted_type())
119+
if not real:
120+
note = (
121+
"propose: needs a real model (--model); no proposal was "
122+
"authored — the notes above are deterministic reads only"
123+
)
124+
return {"notes": [*state.notes, note]}
125+
try:
126+
reply = model.invoke(
127+
"You are the propose step of a documentation-review plan. "
128+
f"The goal: {state.goal!r}. The evidence collected so far:\n"
129+
f"{evidence}\n\n"
130+
"Write one concise, concrete recommendation that satisfies "
131+
"the goal, grounded only in the evidence above."
132+
)
133+
content = getattr(reply, "content", reply)
134+
note = f"proposal (model-authored): {str(content).strip()[:2000]}"
135+
except Exception as exc: # noqa: BLE001 — a failed proposal is a note, not a crash
136+
note = f"propose: model call failed ({exc}); deterministic notes stand"
137+
return {"notes": [*state.notes, note]}
102138

103-
def _factory(build: Any) -> Any:
104-
# `build` is the materialiser's NodeBuild: `name` is the instance the
105-
# planner chose ("readme_survey"), `kind` is what the registry licensed.
106-
# Behaviour keys on the kind; the name is the planner's business.
107-
body = _BODIES[build.kind]
108139
body.writes = {"notes"}
109140
return body
110141

111142

112-
WRITES: dict[str, set[str]] = {kind: {"notes"} for kind in _BODIES}
143+
def _scripted_type() -> type:
144+
from grapharc.testing import ScriptedChatModel
145+
146+
return ScriptedChatModel
147+
148+
149+
_BODIES = {"survey": _survey_body, "read": _read_body, "summarise": _summarise_body}
150+
151+
152+
def _factory_for(model: Any):
153+
def factory(build: Any) -> Any:
154+
# `build` is the materialiser's NodeBuild: `name` is the instance the
155+
# planner chose ("readme_survey"), `kind` is what the registry
156+
# licensed. Behaviour keys on the kind; the name is the planner's.
157+
if build.kind == "propose":
158+
return _propose_body_for(model)
159+
body = _BODIES[build.kind]
160+
body.writes = {"notes"}
161+
return body
162+
163+
return factory
164+
165+
166+
WRITES: dict[str, set[str]] = {kind: {"notes"} for kind in (*_BODIES, "propose")}
113167

114168

115-
def build_registry() -> NodeRegistry:
116-
"""Three read-only kinds. Absence is refusal; there is no write kind to deny."""
169+
def build_registry(model: Any = None) -> NodeRegistry:
170+
"""Three read-only kinds and one that writes what the model recommends.
171+
172+
Accepting `model` is the CLI's contract: a registry factory with a
173+
positional parameter is handed the run's backend (`resolve_registry` in
174+
`grapharc.cli.plan`), which is how `propose` speaks with the same model
175+
that planned — and stays honest when that model is scripted.
176+
"""
177+
factory = _factory_for(model)
117178
return NodeRegistry(
118179
[
119180
NodeSpec(
120181
name="survey",
121182
description="list the documentation files under the working directory",
122-
factory=_factory,
183+
factory=factory,
123184
worst_case=CostEstimate(iterations=1, tokens=300),
124185
),
125186
NodeSpec(
126187
name="read",
127188
description="read each documentation file and record an excerpt",
128-
factory=_factory,
189+
factory=factory,
129190
worst_case=CostEstimate(iterations=1, tokens=1500),
130191
),
131192
NodeSpec(
132193
name="summarise",
133194
description="distil what was read into one summary note",
134-
factory=_factory,
195+
factory=factory,
135196
worst_case=CostEstimate(iterations=1, tokens=800),
136197
),
198+
NodeSpec(
199+
name="propose",
200+
description=(
201+
"author a recommendation from the collected notes; needs a real model"
202+
),
203+
factory=factory,
204+
worst_case=CostEstimate(iterations=1, tokens=2500),
205+
),
137206
]
138207
)
139208

@@ -144,15 +213,16 @@ def default_edge_policy() -> EdgePolicy:
144213

145214

146215
def scripted_planner_replies() -> list[str]:
147-
"""One reply: survey → read → summarise. Read by `grapharc plan` when no
148-
`--model` is given, so the free path exercises the same registry the paid
149-
one does. In an empty directory the chain still yields three notes, so the
150-
loop's goal check is satisfied either way."""
216+
"""One reply: survey → read → summarise → propose. Read by `grapharc plan`
217+
when no `--model` is given, so the free path exercises the same registry
218+
the paid one does — including the `propose` note honestly declining to
219+
author without a real model. In an empty directory the chain still yields
220+
enough notes for the loop's goal check either way."""
151221
import json
152222

153223
from grapharc.runtime.graph import END, START
154224

155-
chain = ["survey", "read", "summarise"]
225+
chain = ["survey", "read", "summarise", "propose"]
156226
endpoints = [START, *chain, END]
157227
return [
158228
json.dumps(

tests/test_plan_docs.py

Lines changed: 50 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -77,11 +77,56 @@ def test_the_cli_scripted_path_uses_this_registrys_replies(docs_dir, capsys):
7777
assert "A widget that frobs." in printed
7878

7979

80+
class _FakeReply:
81+
def __init__(self, content: str) -> None:
82+
self.content = content
83+
84+
85+
class _FakeModel:
86+
"""A real-model stand-in: has `invoke`, is not the scripted type."""
87+
88+
def __init__(self) -> None:
89+
self.prompts: list[str] = []
90+
91+
def invoke(self, prompt: str) -> _FakeReply:
92+
self.prompts.append(prompt)
93+
return _FakeReply("merge 07 and 08 into one Slack page; keep the walkthrough")
94+
95+
96+
def test_propose_hands_the_collected_notes_to_the_model(docs_dir):
97+
model = _FakeModel()
98+
body = plan_docs._propose_body_for(model)
99+
state = plan_docs.DocsState(
100+
goal="plan a merge", notes=["read README.md: Widget — A widget that frobs."]
101+
)
102+
notes = body(state)["notes"]
103+
assert any("proposal (model-authored): merge 07 and 08" in n for n in notes)
104+
assert "A widget that frobs." in model.prompts[0]
105+
assert "plan a merge" in model.prompts[0]
106+
107+
108+
def test_propose_without_a_real_model_declines_honestly(docs_dir):
109+
from grapharc.testing import ScriptedChatModel
110+
111+
for model in (None, ScriptedChatModel(responses=["{}"])):
112+
notes = plan_docs._propose_body_for(model)(plan_docs.DocsState())["notes"]
113+
assert any("needs a real model" in n for n in notes)
114+
115+
116+
def test_a_failing_model_becomes_a_note_not_a_crash(docs_dir):
117+
class Exploding:
118+
def invoke(self, prompt):
119+
raise RuntimeError("backend fell over")
120+
121+
notes = plan_docs._propose_body_for(Exploding())(plan_docs.DocsState())["notes"]
122+
assert any("model call failed" in n and "backend fell over" in n for n in notes)
123+
124+
80125
def test_the_governed_loop_reaches_goal_met_with_substantive_notes(docs_dir, tmp_path):
81126
# Instance names differ from kinds on purpose: a real planner invents
82127
# names ("docs_survey" of kind "survey"), and behaviour must key on the
83128
# kind — this is the shape that catches a factory keyed on the name.
84-
chain = ["survey", "read", "summarise"]
129+
chain = ["survey", "read", "summarise", "propose"]
85130
names = [f"docs_{kind}" for kind in chain]
86131
endpoints = [START, *names, END]
87132
reply = json.dumps(
@@ -98,7 +143,9 @@ def test_the_governed_loop_reaches_goal_met_with_substantive_notes(docs_dir, tmp
98143
)
99144
loop = build_loop(
100145
ScriptedChatModel(responses=[reply]),
101-
registry=plan_docs.build_registry(),
146+
# The registry carries a "real" model for `propose` while the planner
147+
# stays scripted — the two roles are deliberately separable.
148+
registry=plan_docs.build_registry(_FakeModel()),
102149
state_schema=plan_docs.DocsState,
103150
writes=plan_docs.WRITES,
104151
edge_policy=plan_docs.default_edge_policy(),
@@ -109,3 +156,4 @@ def test_the_governed_loop_reaches_goal_met_with_substantive_notes(docs_dir, tmp
109156
assert result.succeeded
110157
assert any("summary of 2 file(s)" in note for note in result.state.notes)
111158
assert any("Widget" in note for note in result.state.notes)
159+
assert any("proposal (model-authored):" in note for note in result.state.notes)

0 commit comments

Comments
 (0)