Skip to content

Latest commit

 

History

History
49 lines (41 loc) · 29.5 KB

File metadata and controls

49 lines (41 loc) · 29.5 KB

Changelog

Defects that have been closed, kept in full rather than summarised: what broke, how it surfaced, and what the fix actually guarantees. These lived in the README's Status and limits section, which made that list a mix of "still true" and "used to be true" — the two things a reader most needs kept apart.

Entries are newest-last within a release, matching the order they were written.

0.1.5

  • a documentation and demo release; no runtime code changed between 0.1.4 and this wheel. The demo film was re-cut to open on the graph itself — frame one is the nine-node incident graph with its first node already running, then the question that built it, then the finished audited run — and the README now leads with it. The README and website stopped describing the project as early and unstable: the status line states the version and the testing discipline, and Status and limits became Limits, framed as edges that are documented and tested rather than confessed. A PyPI downloads badge joined the badge row. This release exists mostly so the PyPI project page, which renders the README frozen at publish time, catches up with all of it.

0.1.4

  • a run stopped for overspending reported spending nothing. Tokens were attributed from end events, and a node the budget interrupts emits error instead — so grapharc metrics answered tokens: 0 for a run whose own enforcement message named the figure that stopped it (max_tokens reached (51/5)). The audit trail lost precisely the number the stop was about, and per-node attribution dropped the most expensive node in the run. Every error event is now stamped with what its node spent, exactly as end is, and both summarize and the cost report count it; sub-events inside a node remain a breakdown of its total rather than an addition, so the disjointness that kept ends + orphans from double-counting is unchanged, and RunCost.tokens == RunMetrics.tokens still holds.
  • the .env credential loader walked up parent directories to /, while the config layer next door refuses exactly that on principle — so the file that spends money was discovered more eagerly than the one that constrains a run. A run started in a scratch subdirectory picked up an OPENROUTER_API_KEY from any ancestor: a .env in $HOME billed every user's experiment on a shared box to that key, a demo checked out under a client project quietly used the client's key, and since redact() is the only thing that ever prints a key, nothing in normal operation said which file paid. The rationale cli/config.py wrote down for grapharc.toml — "a run must never be silently governed by a file in a directory you didn't know about" — applies with more force to the file that pays than to the file that restrains, so find_env_file now reads the start directory (default: the working directory) and no ancestor of it. This is a behaviour change: anyone relying on a parent-directory .env must move it into the directory they run from, export the variable, or pass env_file= naming the file. Neither escape hatch moved — a real environment variable still beats any file, and an explicit env_file= still reads a file anywhere on disk — and no "search boundary" was added in place of the walk, because stopping at a git root is still an upward search.
  • the one edge-declaration path that still deferred its error. add_conditional_edge passed the router and its mapping straight through to LangGraph, so a mapping pointing at a node nobody added was accepted, an empty mapping was accepted, and the first run to take that branch died on self.ends[key] — a bare KeyError raised from inside LangGraph's branch machinery, naming neither the graph, the source node, nor the router that produced the key. Everywhere else this kernel fails at declaration: an undeclared write raises at add_node, a write to a field the schema does not have raises at add_node, a cycle is refused at compile(). The mapping's targets were knowable all along. They are checked now, at add_conditional_edge, with an empty mapping refused and every unreachable target named alongside the key that leads to it; a router that annotates what it returns — a Literal, an Enum — has those members held against the mapping's keys, using the same hash lookup LangGraph will use, so the check predicts the failure rather than approximating it. A router that annotates nothing is still not second-guessed: predicting an arbitrary function's return value is not a check, and inventing a requirement would be worse than the gap. That last case is no longer a KeyError, though — the router is wrapped so an unmapped key raises GraphRoutingError naming the node, the key and the keys that were declared, which is what the rest of the kernel raises for a transition it cannot make. The wrapper keeps the router's name and annotations, because LangGraph names the branch after the one and infers the branch's input schema from the other.
  • a reused --run-id silently welded two runs into one record. Every executing command appends to its --trace file — by design, since grapharc diff reads two runs out of one file — and nothing checked whether the id the operator passed was already in there. Running the same plan twice with one --trace/--run-id pair produced a single "run" whose metrics summed both runs' tokens and node counts, whose viz drew the second path welded onto the end of the first, and whose replay reconstructed a chimera; the operator got no signal at any point, and the trace is documented as the record the metrics cannot disagree with. The file being appendable was never the defect — the id being reused was, so the guard sits at the start of the run rather than in the recorder: plan, run and agent (both executors) refuse an explicit --run-id that already has events in the target trace, with exit 2 naming the id, the count and the file, before a single event is written. Fail closed rather than auto-renaming, because a run id is the name an operator will look the run up under later and picking a different one silently is the same class of surprise. Generated ids are untouched — fresh by construction, so they pay for no scan — and different ids in one file stay exactly as they were.
  • the planner's system prompt withheld the edge policy, so a model had to learn it one refusal at a time. The prompt states the catalog, the START/END literals and the structural rules, and its own comments say why — "stating the rule up front is cheaper than three wasted rounds" — but the rule models actually trip over was the one it never stated. Observed with qwen3:8b against the incident registry: the goal said "find the cause and propose a fix", the policy denied *->deploy, and the planner proposed an edge into deploy in all three rounds (edge_denied; edge_denied + cycle; edge_denied) until the loop stopped admission_refused — about 3.5 minutes of local inference spent discovering one sentence, and a run that reads as a model failure when it is an information failure. The refusal came back every round and edge_denied names the check, not the rule, so "no edge may enter deploy, ever" was never on the page. EdgePolicy.disclosure() and NodePolicy.disclosure() now render a policy's deny rules as one line each (edges into 'deploy' are denied by policy — do not propose them), PlannerNode(edge_policy=…, node_policy=…) puts them directly under the catalog, and the shipped loop builders hand the planner the same policy object the checker holds, so the prompt cannot describe a policy the gate is not applying. Allow rules and the default are left out — they say what is permitted, which the catalog already covers — and so is ask, whose remedy is an approval rather than a different proposal. The refusal side is enriched to match: EdgeRule carries the reason NodeRule already had, PolicyEngine.edge_policy() compiles it out of the document instead of dropping it on the floor, and policy/edge_denied quotes it, so a planner reads why and not only what. None of this is enforcement. No check consults the disclosure, the admission gate is byte-identical, and a model that ignores what it was told is refused exactly as one that was never told — pinned by a test that compares the rejections of a disclosed and an undisclosed planner field by field, and by the shipped demo, whose scripted round 1 still proposes the denied deploy and is still refused.
  • the /live token was accepted in the query string on every route, and a URL is the one place a secret cannot be taken back from: the uvicorn request line, the nginx access log, browser history, and the referrer of anything the page opens. The index made it worse by writing the token into every link it rendered, so clicking a trace filed the secret in history a second time. It is refused off /live/api/stream now — that route keeps it because a browser EventSource cannot set a header and has no other way in — with a 401 whose reason says where to put the token rather than that it is wrong. A browser gets a sign-in page instead of a bare 401 and trades the token for a cookie: a SHA-256 digest of it rather than the token itself, HttpOnly, SameSite=Strict, scoped to /live, and always ASCII, so a non-ASCII secret survives the latin-1 header encoding that a Bearer header cannot. Links carry no token at all. The residual exposure — the SSE request line — is now named in the cookbook next to --live-token, with what to scrub. Every confinement the reader already enforced is untouched: ../, %2e%2e%2f, absolute paths, NUL bytes and symlinked traces are the same 404s, and a hostile token is still a 401 rather than a crash. (#41)
  • the live page was blind for the whole planning phase, which is where a governed run spends its budget and does its refusing. plan, admission and round events were on disk — 2,081 tokens spent before any node ran, in the report — and the page rendered none of them, because it keys the graph off the topology event that only lands once a round is admitted and materialised. A run refused on every round produces no topology at all, so the most governance-relevant run there is showed nothing from start to "finished". The snapshot now carries a planning block folded from those same events (no new trace events): per round, the proposal size, the admission status, the checks that failed and the rejection codes, the planner tokens, and whether it executed; plus the loop's stop reason and detail when it stopped without a graph. The page renders it as a panel, and a round that has begun and not closed reads as active rather than idle — a planner mid-inference writes nothing for a minute at a time, which is exactly the "is it thinking or is it wedged?" the report describes. A run that never planned has no planning field and renders exactly as before. (#47)
  • a finished trace rendered as a done deal: instantly all-green, with the amber running styling unreachable for every run that is already over — and for any live run whose nodes finish between two SSE polls. ?replay=1 on the stream walks the recorded events in timestamp order and emits the snapshots the run would have sent, so a node is amber for its recorded window and green after; &speed=N divides the wall clock and the whole replay is capped at 40 seconds, so a 40-minute incident trace is watchable. Frames are rebuilt by the same snapshot code a live stream uses, pointed at a prefix of the file, and depend on no clock: a trace replayed twice renders identically. Without the parameter nothing changed. (#48)
  • a qwen3-class model's <think> block could beat its own answer: JSON extraction ranked object spans longest-first, so a longer draft inside the reasoning block outranked the real reply outside it, and a fenced draft inside the block won outright (only the first fence was ever tried). The visible text — reasoning tags stripped — is scanned first now, the original text is a fallback tier (a reply that is entirely think-block still parses, and a <think> inside a JSON string is data, because a reply that already parses whole is never rewritten), every fence is tried in order, and a trailing comma is repaired only on candidates that already failed to parse — a trailing comma is never valid JSON, so no valid document can be rewritten.
  • the planner pushed Subgraph's own JSON schema at local grammar-constrained decoders — recursive (ProposedNode.subgraph → Subgraph), every field required under strict mode including the proposal_id/origin it discards on arrival, ~3.5 KB of embedded docstrings — and small models reliably choked on it. Backends now declare reliable_structured_output; Ollama says no and gets the text path: a three-key slim shape (nodes, edges — pair, object and from/to forms all accepted — rationale) with a worked example in the prompt, re-validated through the real constructors so admission judges exactly what it always judged. A parse failure's retry note now shows the model a truncated snippet of its own reply plus the example, instead of a bare error string; --max-planning-failures makes the allowance operator-settable.
  • the generated-policy cache was not keyed by registry, so a .grapharc/generated-policy.toml written for the incident demo (deny *->deploy) silently governed a later stdlib run — overriding stdlib's own deny *->apply_change and making the mutating kind reachable with no operator decision anywhere. Generated policies are keyed by registry target now (generated-policy.<slug>.toml); a legacy un-keyed file is never honoured implicitly when the run can say which registry it is — the run falls through to generation or the registry default and says so — and the file survives untouched for an explicit --policy.
  • the CLI never joined the runs it starts to the live view that draws them. plan's default trace went to a tempdir no server serves, and no command printed a URL. Defaults compose now: traces land under .grapharc/runs/<stamp>/, serve --live-root writes a discovery marker (.grapharc/live-server.json — URL, root, pid, never the token; removed on clean shutdown), and plan/go end with a watch : line — the exact page URL when a marker names a server that answers one loopback connect, the command that would start one otherwise. The goal now rides the loop's topology and approval events (operator-supplied text, deliberately shown — the second state field after termination_reason), so the page can say what a graph is for; a parked run shows its proposed nodes in violet with a copy-ready grapharc approve <dir> banner.
  • plan planned nothing and executed everything — the name lied. The verbs are split now: grapharc plan proposes, the gate admits, and the run STOPS with the admitted plan saved to plan.json next to its trace (exit 0, stopped: planned); grapharc go executes the newest saved plan (go <run-dir> for a specific one), replaying the stored proposal through the full governed loop so admission judges it again on the way in — a hand-edited plan.json is a new proposal, not a pre-approved one; plan --go (and go "a goal") does both in one run. Looking at a plan and then typing go is the approval; --approve remains for parking one-shot runs mid-flight. Registry resolution is now one visible chain shared by both commands: flag/config first, else a registry.py in the directory (yours wins), else the built-in general-purpose kinds — with --default forcing the built-ins past everything.
  • new commands for the first hour: bare grapharc orients instead of erroring (exit 0); grapharc start is the guided tour; grapharc init scaffolds a commented registry.py (whose first free run reproduces refuse-then-admit), a grapharc.toml, and .grapharc/runs/ — refusing to overwrite either authored file, with no --force; grapharc go is plan with doing-defaults (the stdlib tool-using registry, --model required); --registry path/to/file.py:attr loads a registry file directly; --workspace confines the stdlib kinds' tools to a directory (refused when a registry cannot take one — never silently un-confined); --model-arg KEY=VALUE reaches the backend constructor. The stdlib planner is now told the deterministic completion rule (end with summarize) instead of discovering it by burning rounds.
  • the live graph's nodes are clickable now. A click pins an inspector card to the node with everything the snapshot already carries about it: status, executions, tokens (a running node shows its live "so far" spend), recorded cost, duration, its share of the whole run's tokens, each execution's window on the run's timeline, the full error text, and every edge in and out with its kind. The pin survives snapshot patches — a running node's spend ticks in place — and clears itself when a replan produces a topology without that node; Escape, the × or a click on the graph background closes it. The exposure boundary did not move: state_delta contents still never reach a live byte — the panel renders only fields the viewmodel already shipped, and the test pinning that sentence is unchanged.

0.1.3

  • grapharc plan drives the governed loop; PolicyEngine.edge_policy() compiles the TOML document into the gate AdmissionChecker consults, and grapharc plan --policy is the caller; grapharc demo --memory PATH hands the shipped graphs the durable SQLite store.
  • the shipped registry withheld the trace recorder from its PlannerNode and Materializer, so grapharc plan wrote a file with no plan event and no start/end pair for any node it executed — the paragraph above claiming otherwise was true of a hand-wired loop and false of the one the command drives. Both now get the recorder, and a test asserts the phase counts.
  • a max_seconds past the platform's time_tfloat("inf"), or a plausible "effectively unlimited" like 1e10 — used to disable the deadline guard for the rest of the process. setitimer raised after the SIGALRM handler was installed and the process-wide slot taken, leaking both, so every later run silently fell back to the mechanism that cannot unwind a blocking syscall: a 0.3s ceiling then took a 5s sleep to notice. Arming is undone on failure now, and the armed delay is clamped to what both mechanisms accept.
  • every async def node double-charged its token re-reports. The re-report ledger was keyed by thread ident, but on_llm_end is sync — under ainvoke LangChain dispatches it to a worker thread while the body stays on the event loop — so the automatic charge found no ledger and the node's named re-report was charged again. Any node using the shipped charge_usage, AgentNode._charge_tokens or planner.proposal._charge reported double its real spend and hit max_tokens at half its declared allowance. The ledger is a contextvars scope now, which also fixes an inner scope discarding the enclosing node's.
  • a bracket anywhere in a model's prose hijacked JSON extraction, because only the first {/[ was ever tried. Based on the context [lines 3-5]: {…} was rejected as unparseable, and — worse — Analysis (note [1]): {"supported": false} returned a perfectly valid [1], substituting a fabricated value for the verifier's actual answer. Every opener is tried now, and length alone turned out not to be a safe rank — a citation list like [101, 205, 309, …] longer than the verdict still won — so object spans are tried before array spans, each longest-first; junk still returns None, so fail-closed is unchanged.
  • a bare backend name was read as a model name, because split_spec only consulted the backend list when the spec contained a slash. --model claude-cli — the backend models --check reports as usable — shelled out to claude -p --model claude-cli and was refused by the CLI on every call, and --model mock named the paid subscription backend and spawned the real binary, so the double documented as "never reaches a provider" reached for one. A bare backend name now resolves to that backend (claude-cli to its own default model, mock to the scripted double, which ignores the model segment anyway); openrouter, openai and ollama front catalogues rather than a model, so those are refused with an example spelling instead of a guess about what to bill you for. The slash forms and bare model names are unchanged.
  • a failing claude -p reported no reason at all. The CLI exits non-zero with an empty stderr and puts its explanation in the JSON envelope on stdout; the gateway read only stderr, so the error was claude -p exited 1: — a sentence that stops at the colon. Since the wrong-model bug above presented itself exactly that way, the two hid each other. stdout is read first now, and the recovered text also feeds the transient-vs-deterministic classifier, which was previously deciding from "".
  • repeating --registry walked a Slack user straight past the agent opt-in. The gate reads a flag's value to decide admission and argparse's store action then runs the last occurrence, but _flag_value returned the first — so plan … --registry grapharc.examples.plan_docs:build_registry --registry grapharc.stdlib:build_registry was judged against the demo registry and executed against the one that builds agent kinds on the host, with GRAPHARC_SLACK_ALLOW_AGENT never consulted and the forced --approve skipped in the same step. No privilege and no special knowledge needed: typing the flag twice was the whole exploit. Repeats of any admitted flag are refused outright now — the fail-closed reading, which retires the entire first-vs-last family rather than the one flag that exposed it — with a carve-out for the options the CLI itself accumulates (agent --allow/--deny, argparse action="append"), where every occurrence reaches the run and nothing can diverge. A duplicated --model is refused on the same rule, opted in or not, and _flag_value reads the last occurrence regardless, so the two readers can no longer disagree. A sweep over the whole allowlist asserts the duplicated form of every gated flag, so a future gate cannot reopen the gap.
  • a NUL byte in a path came back as silence, the worst answer a chat bot can give: Path(raw).resolve() raises ValueError, handle_text_live catches only SlackCommandError, so trace a\x00b escaped the bolt listener as an unhandled exception and the requester saw no reply at all — indistinguishable from the bot being down. A NUL anywhere in the request is now a refusal in the same voice the core tools already use ("cannot name a file"), and _confined turns any ValueError/OSError out of the filesystem into a refusal too, for callers of its own. Folded in from the same report: the flag allowlist tested token.startswith("--"), so a single-dash token slipped it and was spent as a positional — trace -h was admitted with -h as the path. Any leading dash is a flag now, and one not on the list is refused like any other.
  • the /live token check crashed on the strangers it exists to refuse. secrets.compare_digest rejects str outside ASCII, and _authorized handed it the raw query parameter, so ?token=café raised TypeError through the handler: an unauthenticated 500 with a traceback in the log on all four /live routes, where every ASCII guess correctly got a 401. The 500-vs-401 split was itself an oracle about how the token is compared. Both sides are encoded to UTF-8 now, which drops the ASCII restriction and keeps the constant-time comparison that is the whole reason compare_digest is there. A NUL byte in ?trace= was the same shape one function over — resolve_trace raises ValueError, not the LivePathError the route caught — and is a 404 like any other malformed path now.
  • the /live index advertised traces the reader refuses to serve. scan_traces walked the live root with rglob("*.jsonl"), which matches a symlinked file by name, then parsed it and published its name, size, mtime and run ids on GET /live/api/runs and the HTML index — for a file outside the root that /live/api/stream then 404s, the 404 being the proof of intent. One contract, two code paths, and only the reader enforced it; the live root is documented as the Slack bot's working directory, i.e. somewhere other things write. scan_traces routes every candidate through resolve_trace now and skips symlinks outright, so a refactor of either check cannot reopen the leak. The reader's confinement — ../, %2e%2e%2f, absolute paths, sub/../../, symlinked directories — is unchanged.
  • a deny rule naming a tool literally failed open when the name carried fnmatch metacharacters. PermissionPolicy.decide matched with fnmatch(name, pattern) alone, so DENY "exfil[all]" read as a character class, did not match the tool it spells, and evaluation fell through to whatever came next — typically a broad ALLOW "*". The operator got no error, no warning and no deny; worse, visible() decides the same way, so the tool the operator had just forbidden was described to the model as available and then ran when it asked. The failure was inconsistent as well as silent: DENY "tool?x" happened to hold, because a ? glob matches a literal ?. This was the one place in the tree where a deny failed open — an unmatched tool, an unregistered kind and an unreachable backend all refuse. A deny or ask rule now also fires on an exact literal match. The widening is bound to those two tiers on purpose: equality can only add a rule that refuses or gates a call, never one that permits it, so it cannot loosen a policy the way the same change on allow could. For the allow case there is PermissionRule.literal(action, name), which glob.escapes the name rather than widening the match, and which default_harness now uses for the registry names it allows. Glob semantics are untouched: rm* still spans rmdir, * still matches everything, the tier order and the deny default are unchanged.
  • fan-out handed every worker the same payload object, and never held it to the schema the worker declared. _enter deep-copied only a BaseModel, so two Sends built from one dict gave both parallel workers the same live dict — each reading the other's mutations, through a channel no node declared a write to and no trace event records, in the one place the isolation matters most. _check_goto_target validated Send.node against exactly this class of silent failure and left Send.arg alone, so input_schema — documented as typing a worker's payload — enforced nothing: a dict where a model was declared reached the worker and surfaced as a bare AttributeError frames away from the dispatcher that produced it, and a wrong model class sharing a field name never surfaced at all. Every payload is deep-copied now whatever its type, and one contradicting a declared input_schema is refused at dispatch with StateTypeError naming the node, the schema and what arrived. Declaring no input_schema stays legal — no claim, nothing to check — but the copy is unconditional.
  • the front door was the one door the state contract did not hold. update_state refuses an unknown field and GraphARCState forbids extras, but invoke/stream/ainvoke/astream handed input straight to LangGraph, which filters a dict down to known channels before the state model is ever constructed — so extra="forbid" never saw the typo. invoke({"quesiton": …}) ran the whole graph on default values and returned a complete, plausible answer to an empty question, with nothing said to the caller: the quietest failure in the runtime, on the door every user goes through first. All four entry points, and astream_events, now refuse an unknown input key in the same words update_state uses. A wrongly typed input value was already loud and still raises Pydantic's ValidationError.
  • a node stopped by Ctrl-C left no ending in the trace. The sync wrapper caught Exception while its async twin catches BaseException for the reason its own comment gives — "a stop with no trace line is a stop nobody can audit afterwards" — so a KeyboardInterrupt or SystemExit inside a sync node escaped with no terminal error event, and metrics.summarize then reported errors: 0 for a run an audit reads as having simply stopped between nodes. Ctrl-C is not an exotic ending; it is the commonest way a human stops a long run. The sync wrapper catches BaseException now and re-raises it untouched: only the record is new.
  • a policy document's resource = "node" rules were silently discarded. edge_policy() compiled the edge half and nothing compiled the other one, AdmissionChecker gated node kinds on registry membership alone, and check_node — correct, documented, advertised in the engine's own docstring — had no runtime caller anywhere. So a document denying the kind deploy admitted it and ran it, and the only hint that half the file had been dropped was an oblique 1 edge rule(s) in a line that reads as a summary. The shipped example.toml led with exactly that shape: an operator who copied no-shell-nodes got a policy that denied nothing. PolicyEngine.node_policy() now compiles the node half as edge_policy() does the edge half, AdmissionChecker(node_policy=...) consults it for every proposed node, and a refusal comes back as policy/node_denied quoting the rule's own reason — a code the planner replans against, exactly like edge_denied. A document that declares no node rules still leaves kinds to the registry: saying nothing about nodes is not the same statement as denying all of them, and the banner now counts both halves so a reader can tell which was said.