docs(orchestrator): pin and document channel_overrides REPLACE semantics - #146
Open
eldonm wants to merge 1 commit into
Open
docs(orchestrator): pin and document channel_overrides REPLACE semantics#146eldonm wants to merge 1 commit into
eldonm wants to merge 1 commit into
Conversation
Two properties of channel_overrides account for most "channel overrides don't work" reports. Both are by design and neither announces itself. List knobs REPLACE rather than merge. pinned_tools, denied_tools and skill_only_tools inside a channel block stand in for the action-level list on that channel, so anything still wanted must be repeated there. That is why a config which works with the block commented out can stop working once it is uncommented: the block is not adding to the global list, it is standing in for it. An explicit [] means "none here", not "fall back". Keys match visitor.channel exactly. No prefixes, no aliases, so a block written for whatsapp does not cover whatsapp_call -- voice is its own channel string and both ship in the example app. A mis-keyed block no-ops with nothing logged. Document both in ORCHESTRATOR.md under Configuration, with a table mapping each assembly warning to the failure it indicates, and state them at the two places in the example agent.yaml an operator actually reads -- the channel_overrides block and skill_only_tools. The example previously showed pinned_tools inside a channel block without noting that it replaces. Add tests/wire/test_channel_overrides_replace.py. The orchestrator suites set ex.channel_overrides in Python, which pins the resolver but never exercises agent.yaml -> bootstrap -> persisted attribute -> resolution; a loader that dropped or reshaped the nested block would leave them all green while the feature did nothing. These boot a real graph and read the orchestrator back out of the database. Mutation-checked per the wire tier's convention, which caught a hole in the first draft: the exact-match test asserted on whatsapp_call, which has its own block, so the exact lookup succeeded and a prefix-matching mutation never reached the fallback path -- the test passed while proving nothing. It now asserts on whatsapp_media, a prefix-sharing channel with no block of its own, and fails under that mutation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two properties of
channel_overridesaccount for most "channel overrides don't work" reports. Both are by design, neither announces itself, and neither was stated anywhere an operator reads.pinned_tools,denied_toolsandskill_only_toolsinside a channel block stand in for the action-level list on that channel — anything still wanted must be repeated there. This is why a config that works withchannel_overridescommented out can stop working once it is uncommented: the block is not adding to the global list, it is standing in for it. An explicit[]means "none here", not "fall back".visitor.channelexactly. No prefixes, no aliases — a block forwhatsappdoes not coverwhatsapp_call. Both ship in the example app. A mis-keyed block no-ops with nothing logged.Related issues
Follow-up to a field report that
skill_only_toolsandchannel_overrides"were not working". Investigation found no code defect — #142's 7 resolution tests pass against unmodifiedmain— so this closes the documentation and coverage gap that let a config error read as a product bug.Type of change
Changes
tests/wire/test_channel_overrides_replace.pydocs/ORCHESTRATOR.mdexamples/.../orchestrator_agent/agent.yamlchannel_overridesand atskill_only_toolsChecklist
CONTRIBUTING.mdand the localtests/CLAUDE.md.pre-commit run --all-filespasses.pytest tests/passes — 0 failures.file:linein the description.CHANGELOG.mdentry: no behaviour changes.Notes for reviewers
Why a wire test rather than another orchestrator unit test. The existing suites — including #142's — set
ex.channel_overrides = {...}in Python. That pins the resolver, but never exercises the trip a real deployment takes:agent.yaml→ bootstrap → persisted attribute → resolution. A loader that dropped or reshaped the nested block would leave every one of those tests green while the feature did nothing in production. This tier boots a real graph and reads the orchestrator back out of the database, which is exactly the gaptests/CLAUDE.md§3.1 describes.Mutation-checking caught a bug in my own test, which is worth reading. Per the wire tier's convention I broke the behaviour deliberately:
The exact-match test asserted on
whatsapp_call, which has its own block, so the exact lookup succeeded and the prefix fallback was never reached. It now asserts onwhatsapp_media— a prefix-sharing channel with no block of its own, the only shape that detects it — and fails under that mutation withassert ['whatsapp__send_flow'] == ['kb__search'].That failure mode is the same one this PR documents: something that looks correct and silently does nothing.
Relationship to #142. Complementary, no overlap. #142 pins resolution semantics in Python and adds a
jvagent validateadvisory for the sibling-channel case; this pins the YAML→attribute path and writes the rules down. #142's advisory would have turned this particular report into a startup warning, so it is worth landing.Steps to test
To see the trap directly, add to an agent's orchestrator context: