Skip to content

fix(tools): semantic input coercion + formatZodValidationError-parity validation errors#700

Merged
ericleepi314 merged 3 commits into
mainfrom
fix/tool-input-validation-parity
Jul 12, 2026
Merged

fix(tools): semantic input coercion + formatZodValidationError-parity validation errors#700
ericleepi314 merged 3 commits into
mainfrom
fix/tool-input-validation-parity

Conversation

@ericleepi314

Copy link
Copy Markdown
Collaborator

Incident

A live session (third-party model) called Grep with a parameter literally named n instead of the schema's -n:

⏺ Grep(TODO|FIXME|dangerouslySetInnerHTML|…)
  ⎿ Error: <tool_use_error>InputValidationError: Grep.n: unexpected field</tool_use_error>

The rejection itself is parity-correct — the original's GrepTool schema is z.strictObject (GrepTool.ts:34) and the advertised schema correctly names the field -n on every provider path — but the incident exposed two real gaps in the same validation path vs the original.

Gap 1 — error text (recovery hint quality)

The original renders failed parses via formatZodValidationError (typescript/src/utils/toolErrors.ts:68-134). For the same input it replies:

InputValidationError: Grep failed due to the following issue:
An unexpected parameter `n` was provided

The port said Grep.n: unexpected field. This PR ports the original's format: the three curated categories (missing / unexpected / type-mismatch) grouped in the original's order, JS-flavored received-type names (number, never integer), singular/plural header. Uncategorized issues (enum, oneOf/anyOf) keep the port's readable per-issue fallback — the original dumps zod's raw error.message JSON there, which is strictly worse (deliberate, documented deviation).

Gap 2 — semantic coercion (behavioral parity bug)

The original wraps model-facing boolean/number fields in semanticBoolean / semanticNumber (typescript/src/utils/semanticBoolean.ts, semanticNumber.ts) — zod preprocess steps inside the validation schema used by Bash, FileEdit, FileRead, Grep, PowerShell, CronCreate, SendMessage and TaskOutput. Quoted scalars — "-n": "true", "head_limit": "30", "replace_all": "false" — the most common third-party-model flake — validate and coerce there, and the coerced data (parsedInput.data) is what flows to hooks/permissions/call (toolExecution.ts:669→821).

The port validated the raw JSON schema first, hard-rejecting all of those (expected boolean, got string), which also made the tools' call-time coercion helpers (_semantic_bool/_semantic_int in grep.py) dead code.

Fix: semantic_coerce() runs before validation in the new validate_tool_input() entrypoint, and both dispatch paths (services lane tool_execution.py Step 3; registry.dispatch for mcp-serve) carry the coerced input forward. Semantics mirror the TS wrappers exactly:

  • booleans: only the exact literals "true"/"false" (JS-truthiness would turn "false" into True)
  • numbers: only /^-?\d+(\.\d+)?$/ literals, gated on JS-finiteness (Number("9"×400) is Infinity in JS and is not coerced there — mirrored)
  • anyOf/oneOf nodes never coerce (the original's unwrapped unions — ConfigTool.value keeps "true" a string)
  • copy-on-write: the original input dict is never mutated (the assistant message's recorded tool_use block shares it)

Deviation: coercion is type-driven at the boundary (every boolean/number/integer schema node) instead of replicating the per-field wrapper list — covering the wrapped fields exactly, plus tolerant-direction coverage of the few unwrapped fields (run_in_background, multiSelect) and MCP tool schemas, without a per-tool list to keep in sync.

validate_json_schema keeps byte-identical behavior for generic callers (workflow structured output, advisor).

Bonus: SendMessage approve truthiness trap (critic catch)

TS wraps approve: semanticBoolean() inside the message union branches (SendMessageTool.ts:55,61), which the boundary coercion deliberately skips (anyOf/oneOf rule), and the port's loose oneOf never type-checks union internals. The runtime read was bool(message_obj.get("approve")) — so approve: "false", "no", "0", or 1 all counted as an approval on control-plane messages (shutdown / plan-approval, the latter applying its permission_mode). _approve_flag() now mirrors semanticBoolean→z.boolean exactly: "true"/"false" coerce, anything else (junk strings, numbers, missing) raises ToolInputError and rejects the message.

Tests

tests/test_tool_input_validation.py (28 tests): byte-exact live-repro message, missing/type/multi-issue grouping + pluralization, the full coercion matrix (accepted and refused literals, JS-Infinity gate, ASCII-only digits like JS \d, unions, nested arrays, copy-on-write/non-mutation), generic-path stability, registry + services-lane integration (coerced input reaches call(); wrapped <tool_use_error>InputValidationError: … text), _approve_flag strict semantics.

Full suite: 8506 passed / 0 failed. Critic-reviewed: REVISE (junk-string approve truthy-approval, fixed as above) → APPROVE; critic re-verified every other semantic-wrapped field is a top-level typed node with no repeat of the union-nesting failure mode.

🤖 Generated with Claude Code

ericleepi314 and others added 3 commits July 12, 2026 12:34
… messages

Live incident: a model sent Grep the key `n` instead of `-n` and got the
port's terse 'Grep.n: unexpected field'. The rejection is parity-correct
(z.strictObject refuses unknown keys), but the incident exposed two gaps
in the validation path vs the original:

- Error text now mirrors formatZodValidationError (toolErrors.ts:68):
  'Grep failed due to the following issue:
   An unexpected parameter `n` was provided'
  with the missing / unexpected / type-mismatch categories grouped in the
  original's order and JS-flavored received-type names. Uncategorized
  issues (enum etc.) keep the port's readable fallback (the original dumps
  zod's raw error.message there — deliberate deviation).

- Semantic coercion (semanticBoolean.ts / semanticNumber.ts): string
  'true'/'false' for boolean fields and decimal-literal strings for
  number fields now coerce BEFORE validation, and the coerced input is
  what flows to hooks/permissions/call (TS carries parsedInput.data
  forward). Previously the validator hard-rejected quoted scalars the
  original accepts, e.g. '-n': 'true' or head_limit: '30' — the most
  common third-party-model flake — and the tools' call-time coercion
  helpers never ran. Type-driven at the boundary (every boolean/number/
  integer schema node), never inside anyOf/oneOf unions; copy-on-write so
  the assistant message's recorded tool_use input stays raw.

validate_json_schema keeps its exact old rendering and strictness for
generic callers (workflow structured output, advisor).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e the union

TS wraps approve in semanticBoolean() inside the message union branches
(SendMessageTool.ts:55,61) — the boundary coercion deliberately skips
anyOf/oneOf, and the port's runtime bool() JS-truthied approve:"false"
into an approval.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
approve is a required semanticBoolean inside the union — junk strings/
numbers/missing must reject the message, not JS-truthy into approving a
shutdown or plan. Also: ASCII-gate the number-literal regex (JS \d) and
refresh two stale 'model-original input' comments.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ericleepi314 ericleepi314 merged commit 9b24698 into main Jul 12, 2026
1 of 2 checks passed
@ericleepi314 ericleepi314 deleted the fix/tool-input-validation-parity branch July 12, 2026 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant