Skip to content

Commit 78dd36f

Browse files
Refuse a Subgraph subclass outright; the first fix repelled the attack, not the mechanism
A second reviewer re-ran the shapeshifter attack against the supposedly-patched tree and a policy-denied `deploy` still executed. It was right. The reported attack overrode `Subgraph.fingerprint()`. The fix called the unbound `Subgraph.fingerprint(proposal)` so a subclass could not intercept it — except that function calls `self.model_dump_json()`, which a subclass overrides just as easily, producing an identical break. Patching the reported entrance while leaving the corridor open is worse than the original bug: the alarm is retired and the hole is not. Every route from a proposal to its authorisation ends in a method call on the proposal, and a subclass reaches all of them, so the type is refused rather than the methods hardened one at a time: `materialize()` now takes exactly `Subgraph`. A planner's reply is parsed into that class, so nothing legitimate is lost. The regression test is parametrised over both override paths, so neither can be fixed without the other. HANDOFF.md corrected against the same review. It said "Fixed" of the registry mitigation when `freeze()` is opt-in and nothing in the shipped path calls it — the exact overclaim these audits keep catching, now stated as a mitigation you must apply. Its own history check was self-defeating: quoting the phrase put it back in the tree, so the search matched HANDOFF.md instead of the README; it now carries a `-- README.md` pathspec. Also corrected: 14 unpushed commits not 13, planner 2696 lines not 2554, "sixth smallest" not "thinnest", 39 basics tests not 38, console blocks re-run on one page rather than none, two leftover "nine attacks" after the count became thirteen, and a Quick-commands block whose every `grapharc` line would have failed because it is not on PATH. Added: how to rebuild the venv, that CI pins 3.12 while the venv is 3.14, which four files hold the live tests, and a warning that concurrent agents moved HEAD four times mid-audit. 1339 passed, 10 deselected. ruff clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 68c85e5 commit 78dd36f

3 files changed

Lines changed: 120 additions & 47 deletions

File tree

HANDOFF.md

Lines changed: 76 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -46,9 +46,9 @@ Full design: `ARCHITECTURE.md`. Thesis and honest scope: `VISION.md`.
4646
|---|---|---|
4747
| Repo | `/home/shashank/Desktop/GraphARC`, branch `main` | |
4848
| Remote | `github.com/CodeGraphContext/GraphARC` | `git remote -v` |
49-
| HEAD | `857f620`, plus the planner hardening below | `git log -1` |
50-
| **Unpushed** | **13 commits.** `origin/main` is still at `feef03d` "Initial commit". | `git log origin/main..HEAD` |
51-
| Tests | **1,338 passed, 10 deselected** (live) | `.venv/bin/python -m pytest` |
49+
| HEAD | see `git log -1`; it moved four times during the last session | `git log -1` |
50+
| **Unpushed** | **14 commits.** `origin/main` is still at `feef03d` "Initial commit". | `git log origin/main..HEAD` |
51+
| Tests | **1,339 passed, 10 deselected** (live) | `.venv/bin/python -m pytest` |
5252
| Lint | clean | `.venv/bin/python -m ruff check .` |
5353
| Build | wheel + sdist; 93 modules import from the wheel in a clean venv | `uv build` |
5454
| Version | `0.1.0a0`**not yet bumped for release** | |
@@ -85,6 +85,14 @@ paid APIs and cost real money.**
8585
> **Use bare `.venv/bin/python -m pytest`.** If you must override addopts,
8686
> always write `-o addopts="" -m "not live"` — never the first half alone.
8787
88+
> ### This repo has been edited by concurrent agents
89+
>
90+
> During one verification session HEAD moved four times and HANDOFF.md was
91+
> rewritten twice *while agents were auditing it*, so several of their findings
92+
> were already stale when reported. If you fan work out, give the agents
93+
> **disjoint file sets**, and re-derive any number you intend to write down
94+
> rather than quoting one an agent gave you. `git status` before you start.
95+
8896
### History rewrite, already done
8997

9098
The README once said "*inspired by* (never copied from)". The user judged the
@@ -94,7 +102,9 @@ deleted going forward. Done via `filter-branch --tree-filter`, then
94102
`gc --prune=now` run. Verified unreachable from every ref:
95103

96104
```bash
97-
git log --all -S "not copied from" # must print nothing
105+
# Pathspec matters: this file quotes the phrase, so an unrestricted
106+
# search now matches HANDOFF.md's own history instead of the README's.
107+
git log --all -S "not copied from" -- README.md # must print nothing
98108
```
99109

100110
Attribution to OpenClaw, Hermes, Claude Code and OpenRouter is intact and
@@ -109,7 +119,7 @@ Source lines excluding tests (`find grapharc/<pkg> -name '*.py' | xargs cat | wc
109119

110120
```
111121
grapharc/
112-
planner/ 2554 proposals, admission, materialisation, the governed loop
122+
planner/ 2696 proposals, admission, materialisation, the governed loop
113123
harness/ 2165 tool registry, permissions, sandbox, container executor, AgentNode
114124
memory/ 2165 claims with provenance, SQLite, traversal, contradiction detection
115125
session/ 2084 long-lived, cross-process resume, interrupt, human approval
@@ -123,8 +133,9 @@ grapharc/
123133
policy/ 867 TOML rules, approval routing, decision audit
124134
```
125135

126-
`planner/` is now the **largest** subsystem. In the previous handoff it was the
127-
thinnest, and that inversion is the story of the last session.
136+
`planner/` is now the **largest** subsystem. One handoff ago it was 1,409 lines
137+
and ranked sixth *smallest* of twelve — not the thinnest, but the one whose
138+
central claim was unbuilt. That inversion is the story of the last session.
128139

129140
38 test files. `tests/test_planner_loop.py` (62 tests) is the one to read first
130141
if you are touching the gate. The seven core tools are `read_file`,
@@ -197,14 +208,27 @@ matters more than the count.
197208
itself — a planner emits JSON and has no channel to build a Python object.
198209
Now documented in `materialize.py`'s docstring rather than left to be
199210
over-read.
200-
- **Subclass `Subgraph` and override `fingerprint()`** to return an admitted
201-
proposal's hash → used to run a denied `deploy`. **Fixed**: the check now
202-
calls the unbound `Subgraph.fingerprint(proposal)`, so it hashes what the
203-
object holds rather than what it reports.
211+
- **Subclass `Subgraph` and lie about what it contains** → ran a denied
212+
`deploy`. **Fixed, on the second attempt, and the first attempt is the lesson.**
213+
Overriding `fingerprint()` was the reported attack; the obvious fix was to call
214+
the unbound `Subgraph.fingerprint(proposal)`. That is not a fix — the function
215+
calls `self.model_dump_json()`, so overriding *that* instead defeats it
216+
identically, and a second reviewer demonstrated `deploy` executing against the
217+
supposedly-patched tree. Every route to matching a proposal ends in a method
218+
call on the proposal, and a subclass reaches all of them, so the *type* is now
219+
refused: `materialize()` takes exactly `Subgraph`, never a subclass. Both
220+
override paths are parametrised in one test so neither can regress alone.
221+
222+
Worth internalising: a fix that repels the reported attack while leaving the
223+
mechanism intact is not a fix, and it is more dangerous than the bug because
224+
it retires the alarm.
204225
- **Widen the registry between rounds.** "Every round is checked against the
205226
same registry" meant the same *object*; `register()` stayed callable, so a
206-
node body could add a kind that round 2 is then admitted against. **Fixed**:
207-
`NodeRegistry.freeze()` added, and `loop.py` now points at it.
227+
node body could add a kind that round 2 is then admitted against.
228+
**Mitigation added, not on by default**: `NodeRegistry.freeze()` exists and
229+
works, `loop.py` points at it — and *nothing in the shipped path calls it*, so
230+
a registry is mutable unless you freeze it yourself. Saying this is "fixed"
231+
would be the same overclaim the audits keep catching.
208232

209233
The security claim that survives all of this, stated exactly: **a planner
210234
cannot get unauthorized work executed.** Claims about an operator being unable
@@ -238,14 +262,15 @@ exempt snippet can never acquire an output block.
238262
restate the promise without these):
239263

240264
- **`01-basics.md` is not extracted.** `tests/test_cookbook_basics.py` never
241-
opens the markdown file — its 38 tests *reproduce* each recipe in Python
265+
opens the markdown file — its 39 tests *reproduce* each recipe in Python
242266
rather than parsing the page and byte-comparing. The page says "reproduces",
243267
which is honest; but it means page 1 alone can drift from its tests silently.
244268
The other five pages do parse and byte-compare. **Worth closing.**
245-
- **`console` blocks are never re-run.** The extractors pair ```python blocks
246-
only, so the five ```console transcripts in `06-serving-and-ops.md` are
247-
checked by nothing. That page now says so explicitly instead of claiming
248-
"every snippet below was executed".
269+
- **`console` blocks are re-run on one page only.** `test_cookbook_models.py`
270+
sets `EXECUTABLE_LANGS = {"python", "console"}` and really does execute them;
271+
the agents and serving extractors pair ```python blocks alone, so the five
272+
```console transcripts in `06-serving-and-ops.md` are checked by nothing. That
273+
page now says so instead of claiming "every snippet below was executed".
249274

250275
Writing those tests immediately caught a real flake: the session-interrupt
251276
example posted a stop from a thread and hoped it beat the next superstep. It
@@ -315,8 +340,8 @@ review it. So:
315340
and `ROADMAP.md` §0, not smoothed over.
316341
4. Never weaken a test to make something pass.
317342
5. **A green test is not proof the property holds.** Attack the property
318-
yourself before believing it. The nine attacks above were run *after* the
319-
suite was green, and that is the standard.
343+
yourself before believing it. The attacks above were run *after* the suite was
344+
green — and three of them landed, including one whose first fix was wrong.
320345

321346
`ASSESSMENT.md` is the honest self-assessment — independent reviewers were asked
322347
whether this is a thin LangGraph wrapper, told not to be kind, and three of four
@@ -349,7 +374,8 @@ here.
349374
be stated properly: a planner proposes topology at runtime, a deterministic
350375
model-free checker admits or refuses it, materialisation is bound to that
351376
authorisation by content hash, and *every* replanning round is re-checked.
352-
The nine refused attacks above are the evidence.
377+
The ten refused attacks above are the evidence, and the three that succeeded
378+
are the boundary.
353379

354380
**Do not overstate it.** It is novel as a *composition*, not as new
355381
technology, and it is undercut by gap #1: nothing shipped drives it. Until
@@ -365,7 +391,8 @@ here.
365391
that a reader can run and watch. This converts the project's single most
366392
defensible claim from a test fixture into a demo. Nothing else comes close in
367393
value. (ROADMAP §12.1)
368-
2. **Decide the version and push.** 13 commits are unpushed. I would argue for
394+
2. **Decide the version and push.** 14 commits are unpushed, and they are the
395+
whole project — there is no second copy anywhere. I would argue for
369396
`0.1.0`, not `1.0.0` — a 1.0 implies API stability and several subsystems are
370397
days old.
371398
3. **Publish to PyPI.** `.github/workflows/release.yml` is tag-driven
@@ -445,19 +472,39 @@ Do not let these get quietly dropped from the docs.
445472
## Quick commands
446473

447474
```bash
448-
.venv/bin/python -m pytest # bare -q ONLY; see the -qq trap above
475+
# `grapharc` is NOT on PATH — it lives only at .venv/bin/grapharc. Either
476+
# activate the venv (`source .venv/bin/activate`) or use the full path, as
477+
# below. Bare `python` is likewise not the venv interpreter.
478+
479+
.venv/bin/python -m pytest # bare, no extra -q; see the traps above
449480
.venv/bin/python -m ruff check .
450481
uv build # wheel + sdist
451482

452-
grapharc --help # run agent serve models replay diff trace metrics viz
453-
grapharc models # what backends are configured (redacts secrets)
483+
.venv/bin/grapharc --help # run agent serve models replay diff trace metrics viz
484+
.venv/bin/grapharc models # which backends are configured (redacts the key)
454485

455-
# These two cost money. Read the -m live warning first.
456-
grapharc run capstone --model openrouter/anthropic/claude-haiku-4.5 \
457-
--reviewer-model openrouter/openai/gpt-4o-mini
458-
python -m grapharc.examples.agent_fixit --model openrouter/anthropic/claude-haiku-4.5
486+
# Run one cookbook page's tests after editing it — five of six extract and
487+
# byte-compare, so a changed output block fails until you paste the real one:
488+
.venv/bin/python -m pytest tests/test_cookbook_governance.py
489+
490+
# These two cost money. Read the live-test trap above first.
491+
.venv/bin/grapharc run capstone --model openrouter/anthropic/claude-haiku-4.5 \
492+
--reviewer-model openrouter/openai/gpt-4o-mini
493+
.venv/bin/python -m grapharc.examples.agent_fixit \
494+
--model openrouter/anthropic/claude-haiku-4.5
459495
```
460496

497+
If `.venv/` is missing or stale, rebuild it with `uv sync --all-extras --group
498+
dev`. CI pins Python 3.12 in every job while this venv is 3.14, so a passing
499+
local run is not proof CI passes — `.github/workflows/ci.yml` runs four jobs
500+
(`lint`, `live-marker-guard`, `test` on 3.12/3.13/3.14, `build`).
501+
502+
The ten live tests are `test_gateway_openrouter.py` (7),
503+
`test_gateway_gate.py` (1), `test_gateway.py` (1) and `test_v0_gate.py` (1) —
504+
the last being a full agentic run, so it is the expensive one. List them
505+
without running them:
506+
`.venv/bin/python -m pytest -m live --collect-only -o addopts="" -q`.
507+
461508
Key docs: `ARCHITECTURE.md` §7 (**current** gap analysis, re-derived) ·
462509
`ROADMAP.md` (numbered backlog) · `ASSESSMENT.md` (honest self-critique of an
463510
earlier tree) · `VISION.md` (thesis) · `docs/cookbook/` (112 runnable recipes) ·

grapharc/planner/materialize.py

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -297,9 +297,21 @@ def _require_admission(self, admitted: AdmissionResult, proposal: Subgraph) -> N
297297
f"{type(admitted).__name__}; a proposal on its own carries no "
298298
"authorisation and there is no entry point that accepts one"
299299
)
300-
if not isinstance(proposal, Subgraph):
300+
# Exact type, not `isinstance`. Every way of matching a proposal to its
301+
# authorisation ends in a method call on the proposal, and a subclass can
302+
# override any of them: overriding `fingerprint()` was the first attack,
303+
# and overriding the `model_dump_json()` that `fingerprint()` calls
304+
# defeats an unbound `Subgraph.fingerprint(proposal)` just as well. There
305+
# is no method here a subclass cannot reach, so the type is refused
306+
# instead of the methods being hardened one at a time. A planner's reply
307+
# is parsed into exactly this class, so nothing legitimate is lost.
308+
if type(proposal) is not Subgraph:
301309
raise NotAdmitted(
302-
f"materialize() takes the Subgraph second, got {type(proposal).__name__}"
310+
f"materialize() takes a Subgraph second, got "
311+
f"{type(proposal).__name__}. A subclass is refused rather than "
312+
"trusted: the fingerprint match is computed by calling methods on "
313+
"this object, and a subclass can override them to report an "
314+
"admitted proposal's hash while carrying different nodes"
303315
)
304316
if not admitted.admitted:
305317
raise NotAdmitted(

tests/test_planner_loop.py

Lines changed: 30 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -988,18 +988,24 @@ def body(state: LoopState) -> dict:
988988
)
989989

990990

991-
def test_a_subclass_cannot_lie_about_its_own_fingerprint():
992-
"""The match is computed on what the object holds, not on what it reports.
993-
994-
`proposal.fingerprint()` is a virtual call. A `Subgraph` subclass that
995-
overrides it to return an *admitted* proposal's hash passed the check while
996-
carrying entirely different nodes — a policy-denied `deploy` ran. The
997-
unbound `Subgraph.fingerprint(proposal)` hashes the real contents.
998-
999-
This does not make an `AdmissionResult` unforgeable: a caller who builds one
1000-
by hand with a matching fingerprint still materialises, and nothing in a
1001-
library can stop code that already has the interpreter. It closes the
1002-
cheapest lie, and the boundary is documented in the module docstring.
991+
@pytest.mark.parametrize("overridden", ["fingerprint", "model_dump_json"])
992+
def test_a_subclass_cannot_lie_about_what_it_contains(overridden):
993+
"""Matching a proposal to its authorisation ends in a call *on the proposal*.
994+
995+
Two attacks, one root cause. Overriding `fingerprint()` to return an
996+
admitted proposal's hash ran a policy-denied `deploy`. Calling the unbound
997+
`Subgraph.fingerprint(proposal)` looked like the fix and was not: that
998+
function calls `self.model_dump_json()`, so overriding *that* instead
999+
defeats it identically — verified, `deploy` executed again.
1000+
1001+
There is no method here a subclass cannot reach, so the subclass is refused
1002+
rather than the methods hardened one at a time. Both variants are
1003+
parametrised so neither can be fixed without the other.
1004+
1005+
This still does not make an `AdmissionResult` unforgeable — a caller who
1006+
builds one by hand with a matching fingerprint materialises anything, and no
1007+
library check stops code that already has the interpreter. See the
1008+
trust-boundary note in the module docstring.
10031009
"""
10041010
bodies = Bodies()
10051011
reg = registry(bodies)
@@ -1008,9 +1014,17 @@ def test_a_subclass_cannot_lie_about_its_own_fingerprint():
10081014
approval = gate(reg, policy=DENY_DEPLOY).check(honest)
10091015
assert approval.admitted
10101016

1011-
class Liar(Subgraph):
1012-
def fingerprint(self) -> str:
1013-
return honest.fingerprint()
1017+
if overridden == "fingerprint":
1018+
1019+
class Liar(Subgraph):
1020+
def fingerprint(self) -> str:
1021+
return honest.fingerprint()
1022+
1023+
else:
1024+
1025+
class Liar(Subgraph): # type: ignore[no-redef]
1026+
def model_dump_json(self, **kwargs) -> str:
1027+
return honest.model_dump_json(**kwargs)
10141028

10151029
smuggled = Liar(
10161030
nodes=denied.nodes,
@@ -1019,7 +1033,7 @@ def fingerprint(self) -> str:
10191033
proposal_id=honest.proposal_id,
10201034
)
10211035

1022-
with pytest.raises(NotAdmitted, match="what runs must be what was admitted"):
1036+
with pytest.raises(NotAdmitted, match="A subclass is refused rather than trusted"):
10231037
Materializer(registry=reg, state_schema=LoopState).materialize(approval, smuggled)
10241038
assert bodies.ran == []
10251039

0 commit comments

Comments
 (0)