Skip to content

feat: named profiles so different agents call different models - #79

Merged
JNK234 merged 2 commits into
mainfrom
feat/agent-profiles-68
Sep 9, 2026
Merged

JNK234 merged 2 commits into
mainfrom
feat/agent-profiles-68

Conversation

@JNK234

@JNK234 JNK234 commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Closes #68.

What

Provider and model were one global configuration shared by every agent, and swapping it per agent broke under async calls (an in-flight request resolved against whatever config the next agent installed). This adds named profiles: a second config file loaded under a name, with its own provider, key, model, thinking, retry and throttling settings. Any agent can be bound to a profile; unbound agents use the global configuration exactly as before.

llm:load-config "config.txt"
llm:load-profile "llama"  "config-groq.txt"
llm:load-profile "sonnet" "config-anthropic.txt"
ask turtles with [who mod 2 = 0] [ llm:use-profile "llama" ]
ask turtles with [who mod 2 = 1] [ llm:use-profile "sonnet" ]

Primitives

  • llm:load-profile name file: validated exactly like llm:load-config (unknown provider, missing key, unreachable local server). Reloading a name replaces it; a rejected reload leaves the old profile intact. "default" is reserved.
  • llm:use-profile name: binds the calling agent. "default" unbinds. Unknown name errors and lists what is loaded.
  • llm:profile, llm:profiles.
  • llm:active and llm:config now report the calling agent's effective configuration.

Semantics

  • Each profile caches one provider instance, created on first use (ProfileStore in src/main/config). An async call captures its provider at start, so rebinding an agent affects its next call, never a pending one.
  • Profiles survive clear-all like the global config does; bindings are cleared with the agents.
  • llm:set-model, llm:set-provider, llm:set-api-key and the thinking setters keep acting on the default configuration only. A profile is what its file says.
  • Throttling already keys on provider and endpoint, so profiles on the same endpoint share one cap. History and usage stay per agent.

Tests

  • ProfileStoreSpec: 11 tests. Load, sorted names, reserved and blank names, case-insensitive lookup, provider created once and reused, reload drops the cached provider, unknown name lists loaded ones, factory failure not cached, concurrent first use creates one instance.
  • tests.txt: 14 cases with two tracked fixtures (demos/test-profile-a, -b) and a __TEST_ECHO_MODEL marker so a reply reveals which config served it. Covers default profile, bad provider, missing file, reserved name, unknown profile, two turtles on two models with the observer on the default, llm:active/llm:config per agent, return to default, async keeps its provider after rebinding, set-model leaves profiles alone, reload replaces, failed reload keeps the old profile, clear-all resets bindings and keeps profiles, history and usage per agent.
  • Live: T10 in demos/e2e-tests derives a second profile from the active Groq config with openai/gpt-oss-120b, binds one turtle, and checks routing, reporting, a real reply, and usage credit. 41/41 pass.

Deterministic suite: 249 passing.

Docs

docs/API-REFERENCE.md: new Profiles section. docs/CONFIGURATION.md: profiles subsection under provider examples.

Review follow-up

A Codex (gpt-6-astra) review of today's diff produced six findings. Three were profile-specific and are fixed in this PR with tests first:

  • Throttle gates were replaced on any settings change, so two profiles with different caps on one endpoint bypassed the cap. Gates now reconfigure in place and keep the in-flight count.
  • Profiles were built on OpenAI defaults, so a model-less Anthropic profile asked for gpt-4o-mini. Profiles now load like llm:load-config.
  • Await budgets came from the global config. Calls now use the calling agent's effective config; async calls capture it at launch.

Three predate this work and are filed: #80 (async completion after clear-all), #81 (llm:choose on models that reject json_schema), #82 (refusal payloads lose usage).

Deterministic suite: 254 passing. Live harness: 41/41 on Groq.

Provider and model were one global configuration shared by every agent,
and swapping it per agent broke under async calls. A profile is a second
config file loaded under a name with its own provider, key and model; any
agent can be bound to one and its calls go through that profile's
provider. Unbound agents use the global configuration exactly as before.

New primitives: llm:load-profile name file (validated like load-config; a
rejected reload leaves the old profile intact), llm:use-profile name (the
reserved name "default" unbinds), llm:profile, llm:profiles. llm:active
and llm:config report the calling agent's effective configuration.

Each profile caches one provider instance, created on first use, so an
async call keeps the provider it started with when the agent is rebound.
Profiles survive clear-all like the global config; bindings do not.
Setters (set-model, set-thinking, ...) keep acting on the default config.

Tests: ProfileStoreSpec (11) covers load/replace/reserved names/caching;
14 tests.txt cases cover routing, async capture, reload, failed reload,
clear-all, and that history and usage stay per agent, via a
__TEST_ECHO_MODEL marker. T10 in the live harness binds a turtle to a
second Groq model: 41/41 pass.

Closes #68
…wait budget

Three problems a review found in how profiles interact with the rest of
the extension.

Throttle gates were replaced whenever the configured cap or interval
changed. Two profiles on one endpoint with different settings alternate
every tick, so each call got a fresh gate with a full set of permits and
the cap meant nothing. The gate is now reconfigured in place: a smaller
cap admits nothing until in-flight requests drain below it, a larger one
hands the new permits to the oldest waiters at once. release() repays a
shrunk-cap deficit before admitting anyone.

A profile config was built on ConfigStore.withDefaults(), which carries
model=gpt-4o-mini, so an Anthropic profile that named no model asked for
an OpenAI model. Profiles now load the file as the whole configuration,
the way llm:load-config does, and the provider's own default applies.

Every Await read the global timeout and retry budget. Calls now use the
calling agent's effective config, and an async call captures the budget
at launch so a later config change or rebinding cannot alter it.

Tests: throttle spec covers in-place reconfigure, shrink under load, and
grow with waiters; tests.txt covers a model-less Anthropic profile, a
profile's own one-second budget timing out while the observer succeeds,
and an async call keeping its budget after the global config changes.
@JNK234
JNK234 merged commit b69f8e2 into main Sep 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: per-agent model selection — allow different agents to use different models in one run

1 participant