Skip to content

feat(tools): agent-callable verify/critique tool (agent decides to spend test-time compute on self-review) #4196

Description

@Hmbown

Idea

Give the agent a tool it can CHOOSE to call to run an adversarial self-review of its own recent work before claiming done — spending extra test-time compute (elevated reasoning + an independent critic) exactly when the agent judges it warranted. This is the model-decided version of the multi-lens verification that catches "green-CI-but-wrong" bugs (it just caught #4181 shipping a CLI-only fix that 16/16 CI called done).

Shape (agent DECIDES to call it)

  • A new tool in crates/tui/src/tools/ (name TBD: verify/review/critique), registered in the tool catalog so the model can invoke it.
  • Input: what to verify (a claim/change description + optional scope: recent diff / files / the requirement).
  • Behavior: run an adversarial critic pass at HIGH/MAX reasoning (regardless of session tier) — ideally an independent critic sub-agent (reuse existing sub-agent infra) prompted to REFUTE / find correctness gaps, missed requirements, edge cases. Return structured findings (issue, severity, fix) the agent must address.
  • The tool DESCRIPTION guides WHEN to call (before claiming a non-trivial change done; after a risky edit; when uncertain) — so the agent decides, not an always-on watcher.

Why

Test-time compute → better answers, spent on demand by the model's judgment. Folds in the intent of #3982 (advisor watcher) and the Advisor-role idea, but as an agent-invoked tool.

Acceptance criteria

  • The model can call the tool; it runs a critic pass at elevated reasoning and returns actionable findings.
  • Bounded (a verify call cannot itself recursively spawn verify); config/opt-out for cost.
  • Tests: tool contract + a critic-finds-a-planted-bug test.

Target: v0.8.69 (post-stopship).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingenhancementNew feature or requestreliabilityReliability, flaky behavior, retries, fallbacks, and robustnesssubagentsSub-agent orchestration, lifecycle, and completion handlingtuiTerminal UI behavior, rendering, or interaction

    Projects

    Status
    In progress

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions