Skip to content

feat: per-task model and effort selection for workflows#123

Closed
sasicodes wants to merge 12 commits into
mainfrom
feat/workflow-model-scoring
Closed

feat: per-task model and effort selection for workflows#123
sasicodes wants to merge 12 commits into
mainfrom
feat/workflow-model-scoring

Conversation

@sasicodes

@sasicodes sasicodes commented Jul 6, 2026

Copy link
Copy Markdown
Owner

What changed

Sub-agents in a workflow now pick their own model and effort per task, instead of all inheriting the parent session's single model/effort. The goal is cost efficiency: match each task to the cheapest model and lowest effort that can do it well.

Model scores (models.ts)

  • Every allowed model carries { affordability, intelligence, taste } (0-10, higher is better; affordability higher = cheaper), plus a modelScore(provider, id) lookup.

run_workflow tool

  • Each task is now { prompt, model, effort }, all mandatory (Valibot-validated).
  • The tool description is built dynamically and lists every model that has credentials configured, across all providers, with its scores and supported effort levels.
  • Cross-provider: one workflow can mix OpenAI and Anthropic sub-agents.
  • Guidance tells the agent not to default to the highest scores (wastes tokens) and to match each task to the minimum capability it needs.

Runtime

  • Each sub-agent resolves its own model from the task's key and clamps its own effort, instead of a single shared model/effort.

Testing

  • pnpm run lint, pnpm run format, pnpm --filter @start/desktop typecheck - clean
  • pnpm --filter @start/desktop vitest run - 803 tests passing (added workflow-catalog tests, updated subagent tests)

Note

Stacked on #122; merge that first.

Greptile Summary

This PR introduces per-task model and effort selection for sub-agent workflows, replacing the previous single shared model/effort inherited from the parent session. A ModelScore (affordability/intelligence/taste) is added to each allowed model, and the run_workflow tool description is now built dynamically to list only the models with active credentials, guiding the LLM to pick the cheapest model that fits each task.

  • models.ts gains numeric scores per model; workflowModels() in ChatService assembles the live menu from the registry.
  • SubagentTaskInput now requires model (key string) and effort; Valibot validates both; runtime.ts resolves the model per-task and clamps the thinking level.
  • catalog.ts (new) generates the run_workflow description with scored model lines; tests added for workflow-catalog and updated for subagents-runtime.

Confidence Score: 4/5

Safe to merge after addressing the resolveModel fallback — today it silently runs tasks on the wrong model when the agent provides an unrecognised key.

The resolveModel wiring in chat.ts falls back to pickModel() when findModelByKey returns undefined, so an LLM-supplied key that doesn't exactly match an available model causes the task to run on the user's current session model with no error or log. Every other changed file — the schema, the catalog, the runtime, the types — looks correct and well-tested.

packages/desktop/src/main/chat.ts — the resolveModel closure on line 2008.

Important Files Changed

Filename Overview
packages/desktop/src/main/chat.ts Adds workflowModels() and wires resolveModel; the ?? this.pickModel() fallback silently routes tasks to the wrong model when an invalid key is given.
packages/desktop/src/main/models.ts Adds ModelScore interface and per-model scores; lookup via scoresById map is clean and safe.
packages/desktop/src/main/subagents/runtime.ts Refactors to AgentJob struct, resolves model per-task, clamps thinking level correctly; logic is sound.
packages/desktop/src/main/subagents/utils/catalog.ts New file generating the dynamic run_workflow tool description with per-model scores; straightforward.
packages/desktop/src/main/subagents/utils/input.ts Schema tightened to require model and effort via Valibot; old string-task shorthand correctly dropped.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant PA as Parent Agent
    participant RT as run_workflow tool
    participant RM as resolveModel (ChatService)
    participant SA as Sub-Agent Session

    PA->>RT: "run_workflow({ tasks: [{prompt, model, effort}] })"
    RT->>RT: normalizeSubagentTasks (Valibot validates)
    loop Each task (up to 4 concurrent)
        RT->>RM: resolveModel(task.model)
        alt key found in registry
            RM-->>RT: ResolvedModel
        else key not found
            RM-->>RT: pickModel() fallback (silent)
        end
        RT->>SA: "createAgentSession({ model, thinkingLevel: clampThinkingLevel(model, task.effort) })"
        SA-->>RT: result / error
    end
    RT-->>PA: combined result text
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant PA as Parent Agent
    participant RT as run_workflow tool
    participant RM as resolveModel (ChatService)
    participant SA as Sub-Agent Session

    PA->>RT: "run_workflow({ tasks: [{prompt, model, effort}] })"
    RT->>RT: normalizeSubagentTasks (Valibot validates)
    loop Each task (up to 4 concurrent)
        RT->>RM: resolveModel(task.model)
        alt key found in registry
            RM-->>RT: ResolvedModel
        else key not found
            RM-->>RT: pickModel() fallback (silent)
        end
        RT->>SA: "createAgentSession({ model, thinkingLevel: clampThinkingLevel(model, task.effort) })"
        SA-->>RT: result / error
    end
    RT-->>PA: combined result text
Loading

Reviews (3): Last reviewed commit: "fix: clarify workflow error when no prov..." | Re-trigger Greptile

Comment thread packages/desktop/src/main/providers/tools/subagents.ts Outdated
@sasicodes sasicodes deleted the branch main July 6, 2026 09:38
Base automatically changed from chore/remove-google-workflow-rename-and-update-button to main July 6, 2026 09:38
@sasicodes sasicodes closed this Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant