Skip to content

Commit ecfea58

Browse files
A deny rule naming a tool literally failed open when the name was also a glob (#75)
`PermissionPolicy.decide` matched rules with `fnmatch(tool_name, rule.pattern)` and nothing else. `pattern` is documented as a glob, so the obvious way to write a rule — paste the tool's exact name — silently stopped matching the moment that name carried `[`…`]`, because fnmatch reads it as a character class: DENY "exfil[all]" does not match the tool exfil[all] ALLOW "*" does -> allow The operator got no error, no warning and no deny. `ToolRegistry.visible()` decides through the same call, so the tool the operator had just forbidden was also described to the model as available — the model was actively invited to call the thing it may not have, and the call went through. Verified end to end, not only at the policy layer. The failure was inconsistent as well as silent: `DENY "tool?x"` happened to hold, because a `?` glob matches a literal `?`. And it was the one place in this tree where a *deny* failed open — an unmatched tool defaults to DENY, an unregistered kind is refused, an unreachable backend raises. A rule now matches on `fnmatch(name, pattern) or name == pattern`, with the equality bound to the DENY and ASK tiers. That binding is the whole of the safety argument: adding a match to deny or ask can only ever refuse or gate a call that would otherwise have run, so it cannot loosen any policy. The same widening on ALLOW could grant a tool on a pattern the operator wrote as a glob, so ALLOW stays glob-only, and the case it needs is served by `PermissionRule.literal(action, name)`, which stores `glob.escape(name)` and so names one tool exactly at any tier. `default_harness` builds its ALLOW rules from registry names rather than from operator patterns, so it uses `literal` now; the core tool names carry no metacharacters, so nothing there changes behaviour today. Glob semantics are untouched, and tested as such: `rm*` still spans `rmdir`, `*` still matches everything, a glob still matches through every tier rather than only its own text, the deny -> ask -> allow ordering is unchanged, and an unmatched tool still falls to the DENY default. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 72270f7 commit ecfea58

5 files changed

Lines changed: 135 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,3 +20,4 @@ Entries are newest-last within a release, matching the order they were written.
2020
- a **NUL byte in a path came back as silence**, the worst answer a chat bot can give: `Path(raw).resolve()` raises `ValueError`, `handle_text_live` catches only `SlackCommandError`, so `trace a\x00b` escaped the bolt listener as an unhandled exception and the requester saw no reply at all — indistinguishable from the bot being down. A NUL anywhere in the request is now a refusal in the same voice the core tools already use ("cannot name a file"), and `_confined` turns any `ValueError`/`OSError` out of the filesystem into a refusal too, for callers of its own. Folded in from the same report: the flag allowlist tested `token.startswith("--")`, so a single-dash token slipped it and was spent as a positional — `trace -h` was admitted with `-h` as the path. Any leading dash is a flag now, and one not on the list is refused like any other.
2121
- the `/live` **token check crashed on the strangers it exists to refuse**. `secrets.compare_digest` rejects `str` outside ASCII, and `_authorized` handed it the raw query parameter, so `?token=café` raised `TypeError` through the handler: an unauthenticated 500 with a traceback in the log on all four `/live` routes, where every ASCII guess correctly got a 401. The 500-vs-401 split was itself an oracle about how the token is compared. Both sides are encoded to UTF-8 now, which drops the ASCII restriction and keeps the constant-time comparison that is the whole reason `compare_digest` is there. A NUL byte in `?trace=` was the same shape one function over — `resolve_trace` raises `ValueError`, not the `LivePathError` the route caught — and is a 404 like any other malformed path now.
2222
- the `/live` **index advertised traces the reader refuses to serve**. `scan_traces` walked the live root with `rglob("*.jsonl")`, which matches a symlinked file by name, then parsed it and published its name, size, mtime and **run ids** on `GET /live/api/runs` and the HTML index — for a file outside the root that `/live/api/stream` then 404s, the 404 being the proof of intent. One contract, two code paths, and only the reader enforced it; the live root is documented as the Slack bot's working directory, i.e. somewhere other things write. `scan_traces` routes every candidate through `resolve_trace` now and skips symlinks outright, so a refactor of either check cannot reopen the leak. The reader's confinement — `../`, `%2e%2e%2f`, absolute paths, `sub/../../`, symlinked directories — is unchanged.
23+
- a **`deny` rule naming a tool literally failed open** when the name carried fnmatch metacharacters. `PermissionPolicy.decide` matched with `fnmatch(name, pattern)` alone, so `DENY "exfil[all]"` read as a character class, did not match the tool it spells, and evaluation fell through to whatever came next — typically a broad `ALLOW "*"`. The operator got no error, no warning and no deny; worse, `visible()` decides the same way, so the tool the operator had just forbidden was described to the model as available and then ran when it asked. The failure was inconsistent as well as silent: `DENY "tool?x"` happened to hold, because a `?` glob matches a literal `?`. This was the one place in the tree where a deny failed open — an unmatched tool, an unregistered kind and an unreachable backend all refuse. A `deny` or `ask` rule now also fires on an exact literal match. The widening is bound to those two tiers on purpose: equality can only add a rule that refuses or gates a call, never one that permits it, so it cannot loosen a policy the way the same change on `allow` could. For the `allow` case there is `PermissionRule.literal(action, name)`, which `glob.escape`s the name rather than widening the match, and which `default_harness` now uses for the registry names it allows. Glob semantics are untouched: `rm*` still spans `rmdir`, `*` still matches everything, the tier order and the `deny` default are unchanged.

docs/cookbook/03-agents-and-tools.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -546,6 +546,16 @@ Patterns match the **tool name** only, never its arguments. `DENY "run_command"`
546546
stops the shell tool entirely; it cannot express "deny `rm` but allow `ls`". That
547547
distinction belongs in a pre-hook, two recipes down.
548548

549+
**A name that is also a pattern.** `fnmatch` reads `[``]` as a character class,
550+
so a tool called `exfil[all]` — or an MCP-style `mcp__srv__do[all]` — is not the
551+
same string as the pattern that spells it. A `deny` or `ask` rule therefore also
552+
fires on an **exact literal match**, so pasting a tool's name into a rule refuses
553+
it whatever characters it holds. That fallback is deliberately not extended to
554+
`allow`: equality can only ever add a refusal, never a grant. To *allow* one tool
555+
whose name carries `*`, `?` or `[`, build the rule with
556+
`PermissionRule.literal(Decision.ALLOW, name)`, which escapes the name instead of
557+
widening the match.
558+
549559
---
550560

551561
## How do I make sure a denied tool is never even offered to the model?

grapharc/harness/permissions.py

Lines changed: 47 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,17 @@
44
harness, never by prompts — instructions are advisory, this is not. A broad
55
deny always beats a narrower allow (no allowlist exceptions inside a deny),
66
matching the semantics that survived contact with reality in Claude Code.
7+
8+
A pattern is a glob, but a DENY or ASK rule also fires on an exact literal
9+
match, so a rule that simply names a tool refuses it even when the name carries
10+
fnmatch metacharacters (`exfil[all]`). ALLOW keeps glob-only matching — see
11+
`PermissionRule.matches` for why the two tiers differ, and
12+
`PermissionRule.literal` for naming one tool exactly at any tier.
713
"""
814

915
from __future__ import annotations
1016

17+
import glob
1118
from enum import StrEnum
1219
from fnmatch import fnmatch
1320

@@ -24,9 +31,47 @@ class PermissionDenied(Exception):
2431
"""A tool call was refused by policy (or by an absent/negative approval)."""
2532

2633

34+
#: Tiers where an exact literal name is also honoured as a match, on top of the
35+
#: glob. Restricting a tool can only ever *narrow* what runs, so widening these
36+
#: two is safe in the direction this module already fails; see `matches`.
37+
_LITERAL_TIERS = frozenset({Decision.DENY, Decision.ASK})
38+
39+
2740
class PermissionRule(BaseModel):
2841
action: Decision
29-
pattern: str # fnmatch pattern over the tool name
42+
pattern: str # fnmatch pattern over the tool name; DENY/ASK also match literally
43+
44+
@classmethod
45+
def literal(cls, action: Decision, name: str) -> PermissionRule:
46+
"""A rule matching exactly one tool name, whatever characters it holds.
47+
48+
`pattern` is a glob, so a name containing `*`, `?` or `[` is not the
49+
rule that names it — `exfil[all]` reads as a character class. This
50+
escapes the name (`glob.escape`) so the rule means the tool and nothing
51+
else. Use it whenever the name comes from a registry rather than from an
52+
operator writing a pattern by hand, and especially for ALLOW, where the
53+
literal fallback in `matches` deliberately does not apply.
54+
"""
55+
return cls(action=action, pattern=glob.escape(name))
56+
57+
def matches(self, tool_name: str) -> bool:
58+
"""Does this rule fire for `tool_name`?
59+
60+
The glob, plus — for DENY and ASK only — an exact string equality. A
61+
tool whose name contains fnmatch metacharacters (`mcp__srv__do[all]`)
62+
would otherwise slip past the rule that names it exactly, and the
63+
evaluation would fall through to a broader ALLOW: the one place in this
64+
tree where a *deny* failed open.
65+
66+
Bound to DENY/ASK on purpose. Equality can only add rules that refuse or
67+
gate a call, never ones that permit it, so it cannot loosen a policy;
68+
the same widening on ALLOW could grant a tool the operator never allowed.
69+
For an ALLOW rule naming a metacharacter-bearing tool, write it with
70+
`PermissionRule.literal`, which escapes rather than widens.
71+
"""
72+
return fnmatch(tool_name, self.pattern) or (
73+
self.action in _LITERAL_TIERS and tool_name == self.pattern
74+
)
3075

3176

3277
class PermissionPolicy(BaseModel):
@@ -42,6 +87,6 @@ class PermissionPolicy(BaseModel):
4287
def decide(self, tool_name: str) -> Decision:
4388
for tier in (Decision.DENY, Decision.ASK, Decision.ALLOW):
4489
for rule in self.rules:
45-
if rule.action == tier and fnmatch(tool_name, rule.pattern):
90+
if rule.action == tier and rule.matches(tool_name):
4691
return tier
4792
return self.default

grapharc/stdlib.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -256,8 +256,11 @@ def default_harness(tools: tuple[str, ...], workspace: Any = None) -> Any:
256256
registry = ToolRegistry()
257257
for spec in core_tools(Path(workspace or Path.cwd()), include=tools):
258258
registry.register(spec)
259+
# `literal`, not a bare pattern: these names come from a registry, not from
260+
# an operator writing globs, and an ALLOW rule is the one tier where a name
261+
# read as a pattern could grant more than was asked for.
259262
policy = PermissionPolicy(
260-
rules=[PermissionRule(action=Decision.ALLOW, pattern=name) for name in tools],
263+
rules=[PermissionRule.literal(Decision.ALLOW, name) for name in tools],
261264
default=Decision.DENY,
262265
)
263266
return Harness(registry=registry, policy=policy, executor=LocalExecutor())

tests/test_harness_gate.py

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,79 @@ def test_denied_tools_are_never_visible():
6868
assert visible == ["read"] # rm's schema is never exposed
6969

7070

71+
def test_deny_by_literal_name_beats_a_broad_allow_when_the_name_is_a_glob():
72+
"""A DENY rule that *is* the tool's name refuses it, metacharacters and all.
73+
74+
`exfil[all]` reads as a character class to fnmatch, so the rule naming it
75+
used to miss, evaluation fell through to `ALLOW "*"`, and the tool both
76+
appeared in `visible()` and ran. This is the one place a deny failed open.
77+
"""
78+
name = "exfil[all]"
79+
reg = ToolRegistry()
80+
reg.register(ToolSpec(name=name, description="dangerous", fn=_echo))
81+
policy = _policy([{"action": "deny", "pattern": name}, {"action": "allow", "pattern": "*"}])
82+
83+
assert policy.decide(name) is Decision.DENY
84+
assert [t.name for t in reg.visible(policy)] == [] # never offered to the model
85+
with pytest.raises(PermissionDenied):
86+
Harness(reg, policy).call(name, {}) # and never runs if it asks anyway
87+
88+
89+
def test_ask_by_literal_name_gates_a_glob_shaped_name():
90+
"""The same widening on ASK: a gate that reads right is a gate that holds."""
91+
name = "mcp__srv__do[all]"
92+
reg = ToolRegistry()
93+
reg.register(ToolSpec(name=name, description="", fn=_echo))
94+
policy = _policy([{"action": "ask", "pattern": name}, {"action": "allow", "pattern": "*"}])
95+
96+
assert policy.decide(name) is Decision.ASK
97+
assert [t.name for t in reg.visible(policy)] == [name] # ASK is not DENY
98+
with pytest.raises(PermissionDenied, match="requires approval"):
99+
Harness(reg, policy).call(name, {}) # no approval callback
100+
101+
102+
def test_allow_stays_glob_only_and_literal_escapes_instead():
103+
"""Literal equality is bound to DENY/ASK — it may only ever refuse more.
104+
105+
Widening ALLOW the same way would grant a tool on a pattern the operator
106+
wrote as a glob, so `PermissionRule.literal` escapes the name instead.
107+
"""
108+
name = "exfil[all]"
109+
glob_rule = PermissionPolicy(rules=[PermissionRule(action=Decision.ALLOW, pattern=name)])
110+
assert glob_rule.decide(name) is Decision.DENY # the DENY default still holds
111+
112+
literal_rule = PermissionPolicy(rules=[PermissionRule.literal(Decision.ALLOW, name)])
113+
assert literal_rule.decide(name) is Decision.ALLOW
114+
assert literal_rule.decide("exfila") is Decision.DENY # and nothing the class covers
115+
116+
117+
def test_glob_matching_is_unchanged_by_the_literal_fallback():
118+
"""Regression: every existing glob semantic, at every tier."""
119+
policy = _policy([{"action": "deny", "pattern": "rm*"}, {"action": "allow", "pattern": "*"}])
120+
assert policy.decide("rmdir") is Decision.DENY # prefix glob still spans
121+
assert policy.decide("rm") is Decision.DENY
122+
assert policy.decide("read_file") is Decision.ALLOW # "*" still matches everything
123+
124+
# A glob still matches through every tier, and never only its own text.
125+
for action in ("deny", "ask", "allow"):
126+
tier = _policy([{"action": action, "pattern": "danger_?"}])
127+
assert tier.decide("danger_1") is Decision(action)
128+
assert tier.decide("danger_1x") is Decision.DENY # unmatched -> default
129+
# deny -> ask -> allow ordering, independent of rule order
130+
ordered = _policy(
131+
[
132+
{"action": "allow", "pattern": "x_*"},
133+
{"action": "ask", "pattern": "x_a*"},
134+
{"action": "deny", "pattern": "x_ab*"},
135+
]
136+
)
137+
assert (ordered.decide("x_abc"), ordered.decide("x_ax"), ordered.decide("x_b")) == (
138+
Decision.DENY,
139+
Decision.ASK,
140+
Decision.ALLOW,
141+
)
142+
143+
71144
def test_ask_without_approval_fails_closed():
72145
reg = ToolRegistry()
73146
reg.register(ToolSpec(name="send", description="", fn=_echo))

0 commit comments

Comments
 (0)