feat: named profiles so different agents call different models - #79
Merged
Merged
Conversation
Provider and model were one global configuration shared by every agent, and swapping it per agent broke under async calls. A profile is a second config file loaded under a name with its own provider, key and model; any agent can be bound to one and its calls go through that profile's provider. Unbound agents use the global configuration exactly as before. New primitives: llm:load-profile name file (validated like load-config; a rejected reload leaves the old profile intact), llm:use-profile name (the reserved name "default" unbinds), llm:profile, llm:profiles. llm:active and llm:config report the calling agent's effective configuration. Each profile caches one provider instance, created on first use, so an async call keeps the provider it started with when the agent is rebound. Profiles survive clear-all like the global config; bindings do not. Setters (set-model, set-thinking, ...) keep acting on the default config. Tests: ProfileStoreSpec (11) covers load/replace/reserved names/caching; 14 tests.txt cases cover routing, async capture, reload, failed reload, clear-all, and that history and usage stay per agent, via a __TEST_ECHO_MODEL marker. T10 in the live harness binds a turtle to a second Groq model: 41/41 pass. Closes #68
…wait budget Three problems a review found in how profiles interact with the rest of the extension. Throttle gates were replaced whenever the configured cap or interval changed. Two profiles on one endpoint with different settings alternate every tick, so each call got a fresh gate with a full set of permits and the cap meant nothing. The gate is now reconfigured in place: a smaller cap admits nothing until in-flight requests drain below it, a larger one hands the new permits to the oldest waiters at once. release() repays a shrunk-cap deficit before admitting anyone. A profile config was built on ConfigStore.withDefaults(), which carries model=gpt-4o-mini, so an Anthropic profile that named no model asked for an OpenAI model. Profiles now load the file as the whole configuration, the way llm:load-config does, and the provider's own default applies. Every Await read the global timeout and retry budget. Calls now use the calling agent's effective config, and an async call captures the budget at launch so a later config change or rebinding cannot alter it. Tests: throttle spec covers in-place reconfigure, shrink under load, and grow with waiters; tests.txt covers a model-less Anthropic profile, a profile's own one-second budget timing out while the observer succeeds, and an async call keeping its budget after the global config changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #68.
What
Provider and model were one global configuration shared by every agent, and swapping it per agent broke under async calls (an in-flight request resolved against whatever config the next agent installed). This adds named profiles: a second config file loaded under a name, with its own provider, key, model, thinking, retry and throttling settings. Any agent can be bound to a profile; unbound agents use the global configuration exactly as before.
Primitives
llm:load-profile name file: validated exactly likellm:load-config(unknown provider, missing key, unreachable local server). Reloading a name replaces it; a rejected reload leaves the old profile intact."default"is reserved.llm:use-profile name: binds the calling agent."default"unbinds. Unknown name errors and lists what is loaded.llm:profile,llm:profiles.llm:activeandllm:confignow report the calling agent's effective configuration.Semantics
ProfileStoreinsrc/main/config). An async call captures its provider at start, so rebinding an agent affects its next call, never a pending one.clear-alllike the global config does; bindings are cleared with the agents.llm:set-model,llm:set-provider,llm:set-api-keyand the thinking setters keep acting on the default configuration only. A profile is what its file says.Tests
ProfileStoreSpec: 11 tests. Load, sorted names, reserved and blank names, case-insensitive lookup, provider created once and reused, reload drops the cached provider, unknown name lists loaded ones, factory failure not cached, concurrent first use creates one instance.tests.txt: 14 cases with two tracked fixtures (demos/test-profile-a,-b) and a__TEST_ECHO_MODELmarker so a reply reveals which config served it. Covers default profile, bad provider, missing file, reserved name, unknown profile, two turtles on two models with the observer on the default,llm:active/llm:configper agent, return to default, async keeps its provider after rebinding,set-modelleaves profiles alone, reload replaces, failed reload keeps the old profile,clear-allresets bindings and keeps profiles, history and usage per agent.demos/e2e-testsderives a second profile from the active Groq config withopenai/gpt-oss-120b, binds one turtle, and checks routing, reporting, a real reply, and usage credit. 41/41 pass.Deterministic suite: 249 passing.
Docs
docs/API-REFERENCE.md: new Profiles section.docs/CONFIGURATION.md: profiles subsection under provider examples.Review follow-up
A Codex (gpt-6-astra) review of today's diff produced six findings. Three were profile-specific and are fixed in this PR with tests first:
llm:load-config.Three predate this work and are filed: #80 (async completion after clear-all), #81 (
llm:chooseon models that reject json_schema), #82 (refusal payloads lose usage).Deterministic suite: 254 passing. Live harness: 41/41 on Groq.