Skip to content

fix(web): voice setup picks a backend first and never dead-ends - #84

Merged
nullStack65 merged 2 commits into
release/native-0.1.10from
fix/native-0.1.10-voice-setup-flow
Nov 18, 2026
Merged

nullStack65 merged 2 commits into
release/native-0.1.10from
fix/native-0.1.10-voice-setup-flow

Conversation

@nullStack65

Copy link
Copy Markdown
Owner

Setting up AssemblyAI dictation was confusing and could dead-end. Transcribe with listed environments ("On an environment → primary-r720") next to per-environment cloud groups ("Cloud via primary-r720 → AssemblyAI"). You could pick an environment without a backend, or a backend by environment. When the environment wasn't reachable, the dialog said "Connect to primary-r720 with dictation access to continue." and offered only Cancel. Admins also had to type the AssemblyAI key by hand.

What changed

  • Backend first. Transcribe with now lists Off, AssemblyAI (marked Recommended, the fork default), OpenAI, Grok Voice Transcribe, OpenRouter and Local model. The environment is chosen automatically: the current one if it can run the backend, then this device's own environment, then the first connected one.
  • Environment override is optional. Advanced → Transcription environment only appears when more than one environment can run the chosen backend.
  • No dead ends. An offline environment now says "primary-r720 isn't connected. Open Environments to reconnect it." with a real Open Environments button. If a dialog has nothing to do, its only button says Close. A destination whose environment disappeared says to choose a backend again.
  • Key from the vault, never typed or shown. The server accepts ASSEMBLYAI_API_KEY from its launch environment (for example infisical run --path /agent-runtime -- …). The key-presence response reports it as hostManaged, so the dialog offers Use AssemblyAI with no key field and no Remove key. The server reads the key once and deletes it from its own process.env, so agent subprocesses don't inherit it. A key saved in settings still wins over it.
  • First run. When a device that never chose a backend opens voice settings from the composer mic (or the dictation shortcut), the AssemblyAI confirmation opens right away. It's one click to enable, and consent is still explicit.

Surfaces: web and desktop share this panel. Mobile is unchanged: its cloud sheet is still environment-first, but it already sees the host key as configured. That redesign is a follow-up. Providers, connection modes, and the user docs (voice-input.md, faq.md) are updated.

Before

Off Environment-first menu Key typed by hand Dead end: "continue", no button
before-01 before-02 before-03 before-04

After

First run offers AssemblyAI (host key, nothing to type) Backend-first menu Host-managed key: nothing to remove, Close Offline: real Open Environments button
after-01 after-02 after-03 after-04

Screenshots come from a local dev server on the R720 with an empty database. The server was started with a fake ASSEMBLYAI_API_KEY value, not the real vault key. The offline state was produced by stopping that dev server.

Tests

  • apps/web/src/components/settings/CloudVoiceConfiguration.browser.test.tsx: a new no dead end check. Every dialog state must offer a forward action, any "open X" or "press X" must name a button that exists, and "continue" is only allowed next to a Continue button. It covers the offline environment and the host-managed one-click first run. Existing flows are updated to the backend-first menu.
  • apps/web/src/speech/voiceBackends.test.ts: default environment selection.
  • apps/server/src/speech/CloudSpeech.test.ts: covers the host key. It's used without settings, a saved key outranks it, removing the saved key falls back to it, and it's dropped from process.env.
  • Local runs: web 27 tests (24 browser and 3 logic), server 6, all passing. Targeted effect-tsgo typecheck for web, server and contracts, plus vp fmt --check and vp lint on changed files, all clean.

Rollout

Merging makes AssemblyAI the recommended default in the fork. To make it work with no typing, each fleet host has to launch T3 with ASSEMBLYAI_API_KEY from Infisical /agent-runtime in its environment. That step is outside this PR and belongs with the deployment lanes.

Model: claude-opus-5-5
Task: fix

🤖 Generated with Claude Code

Transcribe with now lists backends (AssemblyAI recommended, OpenAI, Grok,
OpenRouter, Local model) instead of environments, and runs the choice on
this device's environment by default. Overriding the environment is an
optional Advanced row that appears only when more than one environment can
run the backend, so there is no "environment without a backend" state.

The setup dialog no longer says "Connect ... to continue" with only Cancel:
an offline environment offers Open Environments, and a dialog with nothing
to do says Close.

The server also accepts an AssemblyAI key from ASSEMBLYAI_API_KEY in its
launch environment (for example injected from the vault), reports it as
host-managed so the UI offers nothing to type or remove, and drops it from
its own environment so agent subprocesses don't inherit it. A key saved in
settings still wins. First-time dictation offers AssemblyAI straight away.

Model: claude-opus-5-5
Task: fix
Agent: claude-code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nullStack65

Copy link
Copy Markdown
Owner Author

🤖 Claude Fable 5.2 via OpenCode, independent cross-harness review on behalf of nullStack65 (overnight coordinator, one bot review round)

Verdict: one blocking finding, so I did not merge. Head reviewed: 7bd2e4446a0a561d85713bcdbf8798381676fd60. The fix is one word plus a test fixture. Everything else passed review.

Blocking

1. "Connected" is tested against the wrong connection phase, so the default-environment rule is inverted in the real app.

apps/web/src/components/settings/VoiceSettingsPanel.tsx:1240:

connected: environment.connection.phase === "available",

In this codebase "available" means a known environment that is not connected. packages/client-runtime/src/connection/presentation.ts shows it as "Available" next to a separate "Connected", and connectionProjectionPhase in model.ts maps it to "disconnected". The live phase is "connected", which is what 36 other comparisons in apps/web/src use (for example apps/web/src/state/usage.ts:39). This line is the only non-test code that treats "available" as connected.

What that does to defaultVoiceEnvironment:

  • A live environment always gets connected: false. A disabled one gets connected: true.
  • The "this device's own environment, if connected" and "first connected one" steps never pick a live environment. The result falls through to primary ?? candidates[0].
  • With one environment the fallback happens to give the right answer, which is why the screenshots look right.
  • With several environments it goes wrong. If the primary is offline or can't run the backend, a connected environment that can is skipped. The dialog then says "X isn't connected. Open Environments to reconnect it." A disabled environment whose config is still cached is preferred over a live one.
  • For a cloud backend there is no direct way out of that. Advanced → Transcription environment only renders once a destination is saved, and the dialog can't save one for a disconnected environment. That is the "no way forward except reconnect" path this PR sets out to remove, and the Closura fleet is multi-environment.

Why the tests pass anyway:

  • CloudVoiceConfiguration.browser.test.tsx:25 mocks the connected "Laptop" with connection: { phase: "available" }, the same wrong value.
  • voiceBackends.test.ts passes connected as a boolean, so it never sees the mapping. Its "an offline primary environment yields to a connected one" case is the behaviour that does not hold in the app.
  • "available" is a valid member of the phase type, so typecheck can't catch it either.

Fix: compare against "connected", change the fixture to phase: "connected", and add one panel-level test with two environments (offline primary, connected second) asserting the dialog targets the connected one.

Passed

  • Backend before any key or permission prompt. The menu lists backends only. The key dialog opens after a backend is chosen. The first-run offer names AssemblyAI in its title and enables nothing until Use AssemblyAI is clicked; Cancel leaves the choice unset. With no destination, the composer mic navigates to settings and returns before any microphone call (useEnvironmentSpeechInput.ts:455-458).
  • Dialog states. Offline and no-dictation-access both get a real Open Environments button that routes to /settings/connections, which exists. Load failure has Retry. A host-managed key has no field and no Remove.
  • Key handling. The host key is read once through Config.Redacted, kept only in the service closure, and used only in transcribe. It is never written to settings. Clients receive booleans plus hostManaged, never the value. Nothing logs it. A saved key outranks it, and removing the saved key falls back to it.
  • Server test is meaningful: it asserts the real Authorization header across host, saved, and removed states, and that the variable leaves process.env.
  • Scope is clean: 12 files, all voice setup, contracts and docs.

Non-blocking

  1. Part of the "no dead end" helper never runs against this PR's copy. The pattern /\b(?:press|click|tap|hit|open)\s+([A-Z]…)/g is case-sensitive, so sentence-initial "Open Environments to reconnect it." produces zero matches (I evaluated it against the offline dialog text). The forward-button and continue checks do work, and the test separately clicks Open Environments and asserts the route, so the test still has value.
  2. CI on this head does not cover the web half. The ARC workflow typechecks contracts, shared, client-runtime, provider-core, server and mobile, then builds web. It does not typecheck web, and it runs neither voiceBackends.test.ts nor the browser tests. The "27 web tests passing" claim is from the author's local run; I did not rerun it. Worth adding voiceBackends.test.ts to the workflow list.
  3. The green run predates the newest base commit. It was created at 22:08Z and the thread-attribution merge (feat(web): show who each thread came from, and filter by it #82) landed on the base at 22:12Z. Files don't overlap and GitHub reports the PR clean, so I'm not concerned, but the next push will retest against current base.
  4. All backends can be disabled with no explanation. If no environment reports a capability yet (config not loaded), every menu item is disabled and the description stays generic. A line such as "Connect an environment to choose a backend" would close that.
  5. Stale saved choices. A local destination on an environment that can't run local models says "Choose another backend", but re-picking Local model is a no-op (same select value) and the environment override is hidden when only one other environment qualifies. Off, then Local model, works. A cloud destination on an environment that lost the cloud capability now reads "Audio goes to … through X" instead of the old "(unavailable)".
  6. A key that fails validation is ignored silently. If ASSEMBLYAI_API_KEY is set but malformed, the host looks unconfigured with no log line. A warning that names the variable (never the value) would save an operator some time.
  7. Deleting from process.env stops inheritance only. On desktop the Electron parent still holds the variable, and on Linux a same-user process can usually still read the server's /proc/<pid>/environ. That is fine under the stated local threat model; the comment and PR text should just not read as a stronger guarantee.
  8. Mobile still shows the environment-first sheet and may offer Remove for a host-managed key, which does nothing. The PR already lists mobile as a follow-up.

Does merging publish anything externally? No.

  • The base branch release/native-0.1.10 carries one workflow, closura-arc-ci.yml. Its push trigger is limited to feat/assemblyai-current, and its pull-request trigger uses default types, which exclude closed.
  • Earlier merges into this branch produced zero workflow runs (checked the feat(web): show who each thread came from, and filter by it #82 and feat(web): one filter menu for multi-select inboxes and environments #79 merge commits).
  • On main, the release workflow with tag and schedule triggers is disabled, and the fork release workflow runs only by manual dispatch. The newest GitHub Release is v0.0.43 from September.
  • The branch has no protection rules, the repo has no webhooks I can see, and no Pages site.
  • Fleet hosts pin an exact source commit and AppImage hash, so they do not follow the branch head.
  • The repository is public, so the merged code is visible, as the PR branch already is.

Status

Not merged. Checks on the exact head are green (ARC CI success; CodeRabbit success, though its description says it skipped review for this base branch). One fix round on finding 1 and this is ready. I pushed nothing, ran no builds, dispatched no workflow, and touched no keys.

The voice panel compared connection.phase to "available", which means a
known environment that is not connected, so the first-connected rule never
picked a live environment. Move the mapping into voiceEnvironmentCandidates,
compare against "connected", fix the browser fixture, and add a
two-environment test.

Model: claude-opus-5-5
Task: fix
Agent: claude-code
Harness: t3-claude
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nullStack65

Copy link
Copy Markdown
Owner Author

🤖 Claude Opus 5.5 repair on behalf of nullStack65 (overnight coordinator)

Fixes the blocking finding in the review above: voice defaults treated phase === "available" as connected.

  • New voiceEnvironmentCandidates in apps/web/src/speech/voiceBackends.ts compares against "connected". VoiceSettingsPanel now uses it.
  • The browser test fixture now uses phase: "connected".
  • A new unit test covers two environments: an available primary and a connected server. The connected one is chosen.
  • New head: 1de0437f378ab2f71ea971fc85178137e8a88f09. This is a normal commit on top of 7bd2e444.

Local results

  • vp test run --project unit src/speech/voiceBackends.test.ts: 4/4 pass.
  • Negative control: with "available" restored, the new test fails (1 failed, 3 passed).
  • effect-tsgo on apps/web: no errors in the changed files. The 8 errors it reported come from borrowed workspace deps, where @t3tools/contracts resolved to another worktree without this PR's hostManaged. The browser tests were not run.

This is fix verification within the same review round, not a new round.

@nullStack65
nullStack65 merged commit 34635c7 into release/native-0.1.10 Nov 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant