🐛 ask_user is opt-in: spec.execution.askUser - #242
Merged
Merged
Conversation
ask_user shipped default-on (konveyor#161). The gate behind it is fail-closed by design — an unanswered question fails the run — and whether the model asks is its own call, so any run could be failed by a question nobody was attached to see. On the ROKS demo cluster the same Agent and prompt Succeeded once, then Failed at turn 38 on an unanswered ask. It also made mode: auto not the headless-safe policy CONTEXT.md says it is. The tool is now mounted only when the run opts in: spec.execution.askUser on an AgentRun or AgentWorkflow stage, carried next to mode and delivered as execution.askUser in params.json; or HARNESS_HITL_ASK=on for a harness run outside the controller. HARNESS_HITL_ASK=off still wins. The gate itself is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: ibolton336 <ibolton@redhat.com>
Contributor
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: ibolton336 <ibolton@redhat.com>
djzager
approved these changes
Sep 17, 2026
djzager
left a comment
Member
There was a problem hiding this comment.
Reviewed the opt-in path through workflow stages, AgentRuns, params.json, and harness configuration. No issues found.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
ask_user(#161, ADR 0017) shipped default-on, withHARNESS_HITL_ASK=offas the opt-out. The gate behind it is fail-closed by design — an unanswered
question fails the run, the harness never answers for the human — and I think
that part is right. The default is what's wrong:
the same Agent, prompt and gateway Succeeded once (
ui-gckqt), then on thenext run decided at turn 38 to ask a clarifying question and was Failed
180 s later:
stage failed: ask_user question unanswered (HITL gate).Nothing on the run or in the console said that could happen.
there. Mounting the tool also appends
askUserGuidelineto the prompt(
AskUser: len(mcpServers) > 0, same commit as the tool): "A human may bewatching this run. When you need a decision only a human can make — a missing
prerequisite, an ambiguous requirement, a choice between approaches, anything
destructive — call the
ask_usertool and wait…". A migration is made ofchoices between approaches, so a question is the behavior the run was told
to perform, not a model whim — and for an unattended run the premise is
false. The Agent's own prompt in the failing run (236 chars) says nothing
about asking; all of it comes from the harness. The guideline is right for
its purpose (it stops the ask-in-prose-then-end-the-turn bug from before
✨ harness: ask_user tool — the agent can stop and ask the human (ACP elicitation) #161) and is untouched here: it stays gated on the tool being mounted, so
with opt-in it only reaches runs where "a human may be watching" is true.
the inventory and left, anyone who closes the tab. ADR 0017 says "a run that
must complete unattended sets
HARNESS_HITL_ASK=off", which puts the burdenon the wrong runs.
spec.env: [{name: HARNESS_HITL_ASK, value: "off"}], but nothing surfacesthat: the console's create-run modal has no env field, the default Agents
don't set it, and the run itself gives no hint it can be failed this way.
mode: autowasn't headless-safe, which is what CONTEXT.md says it is.The code comment for the default-on block says it exists so "the one E2E path
must exercise the tee (and steering)". That holds for the tee and steering,
which are passive.
ask_useris the only one of the three that can fail a run.What
ExecutionSpec.askUser(bool, default false), next tomode— soit is settable on
AgentRun.spec.executionand perAgentWorkflowstage,and, like mode, not on the Agent (execution-time concern, ADR 0011/0018).
resolveExecutioncarries it from the override;executionSectionwritesexecution.askUser: trueinto params.json only when set.HITLAsk = (params execution.askUser || HARNESS_HITL_ASK=on) && !HARNESS_HITL_ASK=off.onkeeps a path for harness runs outside the controller (hack/harness-*,the live ask test mounts the MCP server itself and is unaffected);
offstill beats everything. The startup log now says which way it went.
rejection of "ask but proceed on timeout", are unchanged); harness README
and agent-base-images env tables; a CONTEXT.md Ask User entry.
I deliberately did not call the field
interactive— the glossary reservesthat as a term to avoid — and did not fold it into
mode: approve:approval gates the agent's tool calls, asking is the agent's initiative, and
a 200-turn migration wants questions without approving every shell call.
Filed as
:bug::mode: autois documented as headless-safe and wasn't, andthis restores that. Upgrade note all the same — a run that relied on the
old default loses the tool until it sets
askUser: true, and the consolecannot set it until the Hub bump and UI checkbox below land (planned as fast
follow-ups, before the next alpha). Opting out never made the agent guess — it
removes the tool, which is the pre-#161 behavior.
Follow-ups (not here)
AgentRunGo type(
type AgentRun = agent.AgentRun), vendored atagentic-controller/api22fee49(2026-08-26). Until Hub bumps that dependency past this PR, anaskUserin the request is dropped on unmarshal — silently. Then theconsole's create-run modal wants a checkbox ("Let the agent ask me questions
— the run fails if one goes unanswered"), unchecked by default.
as the plan was (🐛 Session panel: pin the plan, and stop the usage chip reporting what it cannot know tackle2-ui#3608); it wants pinning and a countdown.
Verification
make generate manifests— three CRDs gainaskUser, nothing else movesmake test— all packages pass (controller envtest suite included)cd harness && go test ./...— pass; newTestHITLAskIsOptIncoversdefault-off, run opt-in, env opt-in, env-off-beats-run, junk env value
TestResolveExecutionextended: askUser comes only from the override, anaskUser-only override stays non-nil, and the key is absent from params.json
unless opted into
make lint— 0 issuesNot run: the kind e2e suite, and no live cluster run on an image built from
this branch.
🤖 Generated with Claude Code