Skip to content

Fix harness default model states in onboarding#2156

Merged
morgmart merged 1 commit into
mainfrom
buzz-agent-config-3
Jul 20, 2026
Merged

Fix harness default model states in onboarding#2156
morgmart merged 1 commit into
mainfrom
buzz-agent-config-3

Conversation

@morgmart

@morgmart morgmart commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR is a narrow safety/presentation patch for onboarding’s default model configuration, split out before the larger Agent Config Core work.

It fixes the field-test issue where Claude Code / Codex could look unconfigured even though the harness had a valid default, and it prevents stale model choices from one harness from appearing as “custom” models in another harness.

Concretely:

  • Model fields now show a real default/loading state instead of looking blank while harness model discovery runs.
  • Harness-provided default model entries are merged into one canonical default row, so users do not see two different “default” choices for the same behavior.
  • Changing harness selection during onboarding clears dependent model/provider effort state immediately, so a Codex model like gpt-5.5[low] cannot survive into Claude Code as a custom model.
  • Generic Effort is hidden for Claude Code and Codex for now, because those harnesses do not use Buzz Agent’s generic BUZZ_AGENT_THINKING_EFFORT path.

Why this PR exists

While testing onboarding, Morgan noticed this confusing path:

  1. Choose Claude Code as the harness.
  2. Continue without selecting model/effort.
  3. Buzz starts welcome agents successfully.
  4. Settings/onboarding surfaces show blank model/effort fields, leaving it unclear what the agents are using.

The underlying behavior was not completely broken: Claude Code and Codex can run with their own harness defaults. The product problem was that Buzz rendered those unset-but-valid defaults as if the user still needed to choose something.

Then, while testing Codex, we found a second issue:

  1. Choose Codex.
  2. Pick a Codex model such as gpt-5.5[low].
  3. Go Back.
  4. Switch to Claude Code.
  5. Return to config.

Before this PR, the old Codex model could appear under Claude Code as:

  • Custom model…
  • custom value: gpt-5.5[low]

That is misleading. The model belonged to the old harness context and should not carry forward silently.

What we learned

This PR exposed a larger architecture issue: the existing shared config UI is still shaped around the Buzz Agent / Goose model of:

Provider → Model → Effort

That shape does not map cleanly onto every harness.

Claude Code

Claude Code effort is real, but not through Buzz Agent’s generic env var.

Local CLI / adapter inspection showed:

  • Claude Code CLI supports --effort <level>.
  • The Claude ACP adapter exposes effort as a native ACP config option:
    • id: "effort"
    • category: thought_level
    • applied via Claude SDK flag settings.

So the current generic BUZZ_AGENT_THINKING_EFFORT field is not the correct way to configure Claude Code effort.

Codex

Codex model discovery can return model IDs that already include effort, such as:

  • gpt-5.5[low]
  • gpt-5.5[medium]
  • gpt-5.5[high]

So showing a separate generic Effort field next to those choices creates two controls for what is effectively one Codex decision.

Clearing rule

We settled on this policy for the future config core:

Context change clears; catalog mismatch explains.

Meaning:

  • If the user changes harness/provider, dependent values belong to the old context and should reset immediately.
    • Example: Codex → Claude clears gpt-5.5[low].
    • Example: OpenAI → Anthropic clears an OpenAI model choice.
  • If an async model catalog later does not recognize a saved model, that alone should not silently erase state everywhere.
    • Discovery can be delayed, stale, credential-limited, or missing a valid custom/alias value.
    • Outside first-run cleanup, the app should explain the value as custom/unavailable/needs attention rather than destructively clear it.
  • Onboarding gets a named cleanup exception because it does not expose custom model entry and is creating the first global default.

This PR implements the immediate onboarding-safe version of that policy.

What changed in this PR

Model defaults now render honestly

usePersonaModelDiscovery now preserves the harness default/current model as a displayable default row.

Examples:

  • Default model (gpt-5.5[high]) when Codex reports a current/default model.
  • Default model when a harness supports a default but does not report the concrete current model.
  • Loading models… while discovery is still pending.

Duplicate default rows are merged

Some harnesses expose a literal model option with ID default. Buzz also needs an empty-value default row meaning “let the harness choose.”

Those are the same user intent. This PR absorbs harness default catalog entries into the canonical empty-value default row, so the user sees one default choice instead of two.

Harness switching clears dependent state immediately

Onboarding now clears stale dependent values as soon as selected harness context changes, rather than waiting for model discovery to eventually prove the old model is invalid.

This prevents stale model flashes/races while the next harness catalog is loading.

Generic Effort is hidden for Claude Code and Codex

This is intentional and temporary.

  • For Claude Code, effort should come from Claude ACP configOptions, not BUZZ_AGENT_THINKING_EFFORT.
  • For Codex, effort currently appears to be encoded in model options.

Showing the generic Buzz Agent effort control for those harnesses would imply Buzz is applying a setting it is not actually applying correctly.

What this PR intentionally does not do

This PR does not implement the full Agent Config Core.

It does not:

  • Render Claude Code’s native ACP effort config option.
  • Split or normalize Codex model+effort IDs into separate UI fields.
  • Move the harness picker into the shared config component.
  • Make Settings fully honor/edit the selected default harness.
  • Rework definition/instance dialogs.

Those are follow-up steps in the broader Harness / Provider / Model plan.

Follow-up plan

The next architectural step should be an Agent Config Core: one source of truth that maps:

selected harness + scope + persisted config

into a field model that answers:

  • What fields exist for this harness?
  • Where do their options come from?
  • What does unset mean?
  • How is the value persisted?
  • How is the value applied at runtime?
  • What clears on harness/provider changes?
  • What should happen when discovery does not recognize a saved value?

A suggested future policy shape:

type DependentValuePolicy = {
  onContextChange: "resetDependentValues";
  onCatalogMismatch: "explainOnly" | "onboardingCleanup";
};

The important distinction is that future agents should not add a vague clearInvalidModel boolean. Changing context and failing async discovery are different events and should have different behavior.

How to review

Things to look for:

  1. Onboarding should never show a stale Codex model as a Claude Code custom model.
  2. Claude Code and Codex should not show the generic Effort field in this PR.
  3. Buzz Agent / Goose provider-backed flows should still show provider/model/effort normally.
  4. Default model rows should not duplicate harness default entries.
  5. Loading model discovery should read as loading, not as “Select a model.”

Verification

Ran after rebasing onto latest origin/main:

  • cd desktop && pnpm typecheck
  • cd desktop && node --import ./test-loader.mjs --experimental-strip-types --test src/features/agents/ui/agentConfigControls.test.mjs src/features/agents/ui/usePersonaModelDiscovery.test.mjs
  • cd desktop && pnpm exec playwright test tests/e2e/onboarding-agent-defaults.spec.ts --project=smoke --grep "authenticated Claude|multiple CLI harnesses|changing setup-page"

Also passed during push:

  • check-push-org
  • branch-skew
  • desktop-check
  • desktop-test
  • mobile-test
  • desktop-tauri-test
  • rust-tests
  • test

@morgmart
morgmart requested a review from a team as a code owner July 20, 2026 03:59
@morgmart
morgmart merged commit a6432ab into main Jul 20, 2026
25 checks passed
@morgmart
morgmart deleted the buzz-agent-config-3 branch July 20, 2026 04:49
wpfleger96 added a commit that referenced this pull request Jul 20, 2026
…vider

* origin/main: (111 commits)
  fix: skip membership lookup for open relays (#2107)
  fix(desktop): make invite QR downloadable (#2168)
  Archive managed agents when deleted (#2135)
  perf(relay): cache Git pack hydration (#2169)
  chore(deps): update rust crate rustls to v0.23.42 (#2151)
  chore(deps): update dependency @tanstack/react-virtual to v3.14.6 (#2153)
  Add Agent Config Core: harness capability model behind agent config surfaces (#2158)
  chore(deps): update all non-major dependencies (#2152)
  chore(deps): update rust crate getrandom to v0.4.3 (#2150)
  fix(relay): bound and observe Git read operations (#2167)
  fix(desktop): derive default clone URL for relay-hosted repos (#2166)
  fix(desktop): mask composer rounded corners (#2165)
  feat(desktop): add PR merge conflict recovery (#2164)
  fix(desktop): mask scrolled content behind channel and thread composers (#2163)
  feat(desktop): add inline PR diff comments (#2162)
  feat(desktop): add commit-scoped PR review decisions (#2161)
  fix(desktop): batch project work item queries (#2160)
  feat(desktop): make missing project checkouts actionable (#2159)
  Fix harness default model states in onboarding (#2156)
  relay: gate push enqueue on live leases; batch matcher pipeline (T1b/T1a-repair/T2b) (#2145)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants