feat: per-task model and effort selection for workflows#124
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Sub-agents in a workflow now pick their own model and effort per task, instead of all inheriting the parent session's single model/effort. The goal is cost efficiency: match each task to the cheapest model and lowest effort that can do it well.
Model scores (
models.ts){ affordability, intelligence, taste }(0-10, higher is better; affordability higher = cheaper), plus amodelScore(provider, id)lookup.run_workflowtool{ prompt, model, effort }, all mandatory (Valibot-validated).Runtime
Testing
pnpm run lint,pnpm run format,pnpm --filter @start/desktop typecheck- cleanpnpm --filter @start/desktop vitest run- 803 tests passing (added workflow-catalog tests, updated subagent tests)Greptile Summary
This PR upgrades the
run_workflowtool so each task carries its ownmodelandeffortinstead of inheriting the parent session's single selection. A newmodels.tsscore table (affordability,intelligence,taste) drives a dynamically built tool description that guides the agent toward cost-appropriate model selection per task.SubagentTaskInputnow requiresmodel(a key string) andeffort;runSubagentscallsresolveModel(task.model)individually per task and passes the result alongside aclampThinkingLevelcall, replacing the former sharedmodel/thinkingLevelpair.workflowToolDescriptionis exposed as aget description()getter on the tool object so the model menu re-evaluates on each access rather than being frozen at session init;workflowModels()filters the picker list to only models that have a static score entry.taskSchemanow validates a strict object (prompt+model+effort), and the JSON schema marks all three fields asrequired; previously loose inputs (plain strings, prompt-only objects) are silently dropped.Confidence Score: 5/5
Safe to merge; changes are self-contained to the workflow tool and its supporting utilities, with no impact on the main session path.
The core runtime and tool plumbing is clean: per-task model resolution, effort clamping, and the dynamic description getter all behave correctly. The only finding is an unused provider field on WorkflowModelOption, which has no runtime impact. Tests cover the new schema, catalog rendering, and runtime concurrency behaviour.
No files require special attention.
Important Files Changed
Sequence Diagram
%%{init: {'theme': 'neutral'}}%% sequenceDiagram participant Agent as Parent Agent participant Tool as run_workflow tool participant Catalog as workflowToolDescription participant Runtime as runSubagents participant Resolver as resolveModel(key) participant Sub as Sub-Agent Session Agent->>Tool: read description (get accessor) Tool->>Catalog: workflowToolDescription(availableModels()) Catalog-->>Agent: model menu with scores Agent->>Tool: "execute({tasks: [{prompt, model, effort}, ...]})" Tool->>Runtime: "runSubagents({tasks, resolveModel, ...})" loop each AgentJob (max 4 concurrent) Runtime->>Resolver: resolveModel(task.model) alt model found Resolver-->>Runtime: ResolvedModel else not found Resolver-->>Runtime: "pickModel() fallback | null" end Runtime->>Sub: "createAgentSession({model, thinkingLevel: clampThinkingLevel(model, task.effort)})" Sub-->>Runtime: summary end Runtime-->>Tool: SubagentRunResult Tool-->>Agent: tool result text%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% sequenceDiagram participant Agent as Parent Agent participant Tool as run_workflow tool participant Catalog as workflowToolDescription participant Runtime as runSubagents participant Resolver as resolveModel(key) participant Sub as Sub-Agent Session Agent->>Tool: read description (get accessor) Tool->>Catalog: workflowToolDescription(availableModels()) Catalog-->>Agent: model menu with scores Agent->>Tool: "execute({tasks: [{prompt, model, effort}, ...]})" Tool->>Runtime: "runSubagents({tasks, resolveModel, ...})" loop each AgentJob (max 4 concurrent) Runtime->>Resolver: resolveModel(task.model) alt model found Resolver-->>Runtime: ResolvedModel else not found Resolver-->>Runtime: "pickModel() fallback | null" end Runtime->>Sub: "createAgentSession({model, thinkingLevel: clampThinkingLevel(model, task.effort)})" Sub-->>Runtime: summary end Runtime-->>Tool: SubagentRunResult Tool-->>Agent: tool result textReviews (2): Last reviewed commit: "fix: keep workflow model menu fresh as p..." | Re-trigger Greptile
Context used: