A deny rule naming a tool literally failed open when the name was also a glob - #75
Merged
Conversation
…o a glob
`PermissionPolicy.decide` matched rules with `fnmatch(tool_name, rule.pattern)` and
nothing else. `pattern` is documented as a glob, so the obvious way to write a rule —
paste the tool's exact name — silently stopped matching the moment that name carried
`[`…`]`, because fnmatch reads it as a character class:
DENY "exfil[all]" does not match the tool exfil[all]
ALLOW "*" does
-> allow
The operator got no error, no warning and no deny. `ToolRegistry.visible()` decides
through the same call, so the tool the operator had just forbidden was also described
to the model as available — the model was actively invited to call the thing it may
not have, and the call went through. Verified end to end, not only at the policy layer.
The failure was inconsistent as well as silent: `DENY "tool?x"` happened to hold,
because a `?` glob matches a literal `?`. And it was the one place in this tree where
a *deny* failed open — an unmatched tool defaults to DENY, an unregistered kind is
refused, an unreachable backend raises.
A rule now matches on `fnmatch(name, pattern) or name == pattern`, with the equality
bound to the DENY and ASK tiers. That binding is the whole of the safety argument:
adding a match to deny or ask can only ever refuse or gate a call that would otherwise
have run, so it cannot loosen any policy. The same widening on ALLOW could grant a
tool on a pattern the operator wrote as a glob, so ALLOW stays glob-only, and the case
it needs is served by `PermissionRule.literal(action, name)`, which stores
`glob.escape(name)` and so names one tool exactly at any tier. `default_harness`
builds its ALLOW rules from registry names rather than from operator patterns, so it
uses `literal` now; the core tool names carry no metacharacters, so nothing there
changes behaviour today.
Glob semantics are untouched, and tested as such: `rm*` still spans `rmdir`, `*` still
matches everything, a glob still matches through every tier rather than only its own
text, the deny -> ask -> allow ordering is unchanged, and an unmatched tool still falls
to the DENY default.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #59.
What was wrong
PermissionPolicy.decidematched rules withfnmatch(tool_name, rule.pattern)and nothing else.patternis documented as a glob, so the obvious way to write a rule — paste the tool's exact name — silently stopped matching the moment that name carried[…], because fnmatch reads it as a character class. The rule did not fire, evaluation fell through to whatever came next (typically a broadALLOW "*"), and the tool ran.ToolRegistry.visible()decides through the same call, so the tool the operator had just forbidden was also described to the model as available. The model was invited to call the thing it may not have, and the call went through.Before, on
main:After:
The failure was inconsistent as well as silent:
DENY "tool?x"happened to hold, because a?glob matches a literal?. And it was the one place in this tree where a deny failed open — an unmatched tool defaults to DENY, an unregistered kind is refused, an unreachable backend raises.The fix
A rule now matches on
fnmatch(name, pattern) or name == pattern, moved ontoPermissionRule.matchessodecide()andvisible()cannot drift apart —visible()already goes throughdecide(), and there is now a single place where matching is defined.Which tiers were widened, and why only those. The literal equality is bound to DENY and ASK. That binding is the whole of the safety argument:
allowkeeps glob-only matching.For the case
allowgenuinely needs — permitting one tool whose name carries*,?or[— there isPermissionRule.literal(action, name), which storesglob.escape(name). That escapes rather than widens, so it names exactly one tool at any tier and is the right constructor whenever the name comes from a registry rather than from an operator writing patterns by hand.Callers checked
grapharc/stdlib.py:default_harnessbuildsALLOWrules from literal core tool names, which is exactly the escape-don't-widen case, so it usesPermissionRule.literalnow. The core tool names (read_file,grep,run_command, …) hold no metacharacters andcore_tools(include=…)rejects any name outside that set, so this changes no behaviour today; it is correct by construction rather than by luck if the set ever grows.grapharc/cli/agent.py:build_policyturns--allow/--deny/--askflags into rules. Those are operator-written patterns and stay globs; the deny/ask ones simply pick up the literal fallback.grapharc/policy/engine.py:permission_policycompiles a TOML document'smatchfields, documented as fnmatch, and is unchanged — it inherits the same tier semantics, so the compiled policy andcheck_toolstill agree.Preserved behaviour
Explicitly tested, not just asserted:
rm*still deniesrmdirandrm,*still matches everything, a glob still matches through every tier rather than only its own text, the deny → ask → allow ordering is independent of rule order, and an unmatched tool still falls to theDENYdefault.Tests
Added to
tests/test_harness_gate.py:test_deny_by_literal_name_beats_a_broad_allow_when_the_name_is_a_glob— the issue's acceptance criterion, checked in all three directions:decide(),visible(), and end to end throughHarness.call.test_ask_by_literal_name_gates_a_glob_shaped_name— the same for ASK with an MCP-stylemcp__srv__do[all]; it stays visible (ASK is not DENY) but is refused without an approver.test_allow_stays_glob_only_and_literal_escapes_instead— pins the asymmetry, so a later "make it consistent" cleanup has to argue with a test: anALLOW "exfil[all]"glob rule leaves the tool on the DENY default, andPermissionRule.literalis what grants it.test_glob_matching_is_unchanged_by_the_literal_fallback— the regression guard for every existing glob semantic.The first three fail on
main(two asallow/visible/executed, one asAttributeError: literal); the fourth passes before and after, which is its job.Docs: the permissions recipe in
docs/cookbook/03-agents-and-tools.mdnow states the rule, andCHANGELOG.mdcarries the defect entry in the section's usual form.Verification
1779 passed, 12 deselected— the full suite, green, with no flaky SIGALRM timing failures on either this branch or untouchedmain.ruff check .clean.🤖 Generated with Claude Code