Channel overrides: pin per-channel resolution of skill_only_tools, and advise on uncovered sibling channels

[eldonm]

Summary

Two commits on the same problem, follow-up to #138.

channel_overrides is resolved by the exact visitor.channel string — no prefix matching, no aliasing — so a block written for whatsapp does nothing on a whatsapp_call (voice) turn. The mis-keyed override silently no-ops and the action-level value applies instead, which presents as "channel overrides are broken" rather than as a config error. Both keys are valid channels, and the example app ships blocks for both, so this is easy to hit.

1. test(orchestrator) — pins the per-channel resolution behavior, which was only partly covered.
2. feat(validate) — makes jvagent validate advise when this specific mistake is present.

No change to resolution semantics: exact-key matching is correct and is now pinned against anyone "fixing" it into fuzzy matching.

Commit 1 — resolution tests

tests/action/orchestrator/test_skill_only_channel_overrides.py (7 tests). The existing suite covered only the "override with a different list" case.

Test	Pins
test_empty_override_ungates_that_channel	an explicit [] means "gate nothing here", not "fall back" — a truthiness-based resolver gets this backwards
test_override_gates_when_action_level_is_empty	empty globally, gated only on the override channel
test_alternating_channels_do_not_leak_state	voice → web → voice; resolution is per-turn
test_unknown_channel_falls_back_to_action_level	unknown and default both take the action-level list
test_override_key_must_match_the_channel_exactly	the trap — whatsapp does not cover whatsapp_call
test_channel_resolved_list_drives_the_lean_pool	under lean, the pre-surface exclusion follows the channel-resolved list
test_denied_channel_override_still_beats_the_gate	precedence holds per-channel

Each asserts the gate by calling the tool and checking for the steer string, not by inspecting config.

Commit 2 — the validate-time advisory

Emits an advisory when a subtractive knob (skill_only_tools, denied_tools, pinned_tools) is set for one channel of a family but not for its reachable sibling:

validate advisory: agent.yaml [actions[0].context.channel_overrides]
  'skill_only_tools' is overridden for channel 'whatsapp' but not for its sibling
  'whatsapp_call', which is reachable on this agent (provided by
  jvagent/whatsapp_voice_action). Turns on 'whatsapp_call' will use the
  action-level 'skill_only_tools' instead.
  Hint: channel_overrides is matched on the exact channel string. Add a
  'whatsapp_call' block setting 'skill_only_tools' if that is not intended.

Deliberately conservative, because a lint operators learn to ignore is worse than no lint:

• Fires only when the sibling channel is genuinely reachable — its providing action is enabled on that agent. An agent with no voice action hears nothing.
• Covers only knobs whose absence changes behavior silently. Per-channel divergence in history_limit, ack knobs and system_prompt_extra is normal and is ignored.
• Symmetric: gating voice but not chat is as likely a mistake as the reverse.
• Reports the consequence, not the omission — the omission is often correct.

New advisory severity

Every existing AgentYamlWarning is CI-fatal (run_validate returns 1 on any warning). Shipping a heuristic at that severity would break the build of any app whose asymmetry is deliberate. So AgentYamlWarning gains severity, defaulting to "warning" — every existing call site is unchanged — and advisories are printed without affecting the exit code. jvagent validate --strict promotes them to failures for teams that want them enforced.

A note on why one test matters more than the others

CHANNEL_PROVIDERS maps a channel to the action that provides it, and that map is the lint's reachability gate. My first draft keyed it on package directory names (jvagent/whatsapp_voice) rather than published package names (jvagent/whatsapp_voice_action) — they differ. Every behavioral test still passed, because the fixtures used the same wrong constant, and the lint would simply never have fired on a real app. Only an end-to-end run against a real agent.yaml caught it.

test_channel_providers_match_real_action_package_names now reads the actual info.yaml files and pins the refs. It was mutation-checked: reverting the ref turns it red.

Verification

• pytest tests/core/test_channel_override_coverage.py tests/cli/test_validate_advisories.py → 17 passed
• pytest tests/action/orchestrator/test_skill_only_channel_overrides.py → 7 passed
• pytest tests/ → exit 0
• pre-commit run --all-files → 8/8 hooks pass, no modifications
• End-to-end against a real app reproducing the original incident: advisory fires, validate exits 0, --strict exits 1
• The bundled example app, which sets pinned_tools on both whatsapp and whatsapp_call, produces no advisory (true negative)

Design notes, including what was considered and dropped, are in .planning/specs/2026-08-04-channel-overrides-validation-design.md.

🤖 Generated with Claude Code