Skip to content

thv llm setup: add --lazy mode that defers OIDC login until first token use #5386

@jerm-dro

Description

@jerm-dro

Motivation

thv llm setup currently runs the interactive OIDC browser flow inline (see pkg/llm/setup.go:99-103):

Ensuring you are logged in to the LLM gateway…

This is appropriate for an interactive user at a terminal, but it makes the command unusable in automated provisioning contexts — e.g. an MDM (Mobile Device Management) profile running thv llm setup at first login to pre-configure Claude Code, Cursor, etc. on a fleet of laptops. The browser flow cannot complete without a human present, so setup either fails or hangs waiting for the callback.

The flag validation, tool detection, tool-config patching, and config persistence steps are all non-interactive and safe to run unattended; only the inline login is interactive.

Proposal

Add a --lazy flag to thv llm setup that skips the inline OIDC login and defers it until the user actually tries to access the LLM gateway. When deferred login fires, the experience must match what thv llm setup would have done — the same browser-based OIDC flow, with no extra command for the user to run.

Behavior when --lazy is set

thv llm setup --lazy does everything the normal command does except call login(ctx, &llmCfg) (pkg/llm/setup.go:100):

  1. Validates inline flags and merges them into the config.
  2. Detects supported tools.
  3. Patches each tool's config files (ConfigureLLMGateway + ConfigureEnvFile).
  4. Persists the merged config and ConfiguredTools to disk.
  5. Prints a clear message that login will happen on first use, e.g.:
    Lazy mode: skipping OIDC login. You'll be signed in automatically
    the first time a configured tool accesses the LLM gateway.
    

Deferred login by component

Proxy-mode tools (Cursor, VS Code, Xcode) — login fires at request time, not at proxy startup.
runLLMProxyForeground (cmd/thv/app/llm.go:464-465) already builds an interactive TokenSource, and Proxy.handler calls p.tokenSource.Token(ctx) on every incoming request (pkg/llm/proxy/proxy.go:164). After a lazy setup, the first request through the proxy triggers the OIDC browser flow naturally; the proxy startup path itself should not need to change.

Token-helper tools (Claude Code, Gemini CLI) — thv llm token needs to gain the interactive path.
Today it is non-interactive by design (cmd/thv/app/llm.go:175, interactive=false) and returns ErrTokenRequired (pkg/llm/tokensource.go:20-23) when no cached token exists. To make lazy mode transparent, thv llm token should launch the browser flow whenever no cached or refreshable token is available — the same flow thv llm setup would have run.

Background / subprocess contexts

Both deferred entry points are typically called from non-foreground processes — thv llm token is spawned as a subprocess by Claude Code / Gemini CLI, and thv llm proxy start & is commonly run as a backgrounded shell job. This is fine: the OIDC flow uses github.com/pkg/browser (pkg/auth/oauth/flow.go:234), which shells out to the OS URL opener and does not require a TTY or stdio. As long as the process runs inside the user's desktop session — which is true for both cases above — the browser opens and the OIDC callback completes normally. If the browser cannot be opened (e.g. no desktop session), the existing fallback at flow.go:235-237 prints the auth URL to stderr.

Acceptance criteria

  • thv llm setup --lazy completes successfully without opening a browser, even when no cached token exists.
  • Tool config files and persisted config are identical to a normal thv llm setup run.
  • After a lazy setup, the first request through thv llm proxy start triggers the OIDC browser flow and the request completes once login finishes (without the proxy's per-request token-fetch timeout cutting off the user mid-login).
  • After a lazy setup, the first invocation of thv llm token (including when spawned as a subprocess by Claude Code / Gemini CLI) triggers the OIDC browser flow and prints a fresh token on success.
  • Subsequent invocations use the cached refresh token without prompting.
  • E2E tests cover the lazy setup path and the deferred-login path (using a fake login function).
  • CLI docs regenerated with task docs.

Non-goals

  • Headless / device-code OAuth flows for true non-interactive login on remote machines.
  • Changing the default behavior of thv llm setup for interactive users. Lazy must be opt-in.

Metadata

Metadata

Assignees

No one assigned

    Labels

    cliChanges that impact CLI functionalityenhancementNew feature or requestgoPull requests that update go codellm gatewayLLM gateway authentication feature

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions