fix(tests): raise guardrail smoke/escalation turn_timeout 300 -> 1200 - #2234
Conversation
Both tasks were hard-killed mid-turn at exactly 300s in the last two smoke runs on PR #1694 (July 7 and today) — the only turn-timeout deaths in the suite. 99 of 106 uipath-agents tasks use 1200; these two pins predate that standard. Neither task sets task_timeout, so the turn budget is the effective cap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Claude finished @mjnovice's task in 1m 20s —— View job SummaryRaises Change-by-Change Review1.
|
| Area | Status | Notes |
|---|---|---|
| Frontmatter | N/A | No frontmatter changes |
| E2E Tests | OK | Only timeout tuning, no behavioral changes |
| Skill Body | N/A | No skill changes |
| References & Assets | N/A | No references changed |
| Repo Hygiene | OK | Scoped change, no secrets, no unrelated edits |
Issues for Manual Review
None found.
Conclusion
Clean, well-scoped fix. Both values now match the suite standard. LGTM — approve.
|
Claude finished @mjnovice's task in 2m 7s —— View job Coder-eval task lint (advisory)2 task YAMLs changed; verdict counts: 0 Critical, 1 High, 0 Medium, 0 Low, 2 OK. Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge. Evidence of passing run❌ High — PR body does not claim the changed tasks have been run and passed with the new timeout value. The PR documents why the change is needed (300s timeout kills on CI) but doesn't assert the tasks pass at 1200s. Please consider editing the PR description to add a line like:
(Given the change is purely a timeout raise, this is largely ceremonial — but the convention asks for it.) Per-task lint
|
| expected_turns: 32 | ||
| max_turns: 40 | ||
| turn_timeout: 300 | ||
| turn_timeout: 1200 |
There was a problem hiding this comment.
I will approve this, but it doesn't solve the real problem: failing cli calls (fixed yesterday), LLM not following the skills steps.
I'm working on a more robust fix: #2219
Why
The uipath-agents smoke suite is flaking on turn-timeout kills. In the last two runs on PR #1694:
skill-agent-guardrail-coded-escalation-smoke—Turn timeout (300s) fired after 300.0s — hard-killing subprocessskill-agent-guardrail-coded-smoke— same 300s hard-killThese were the only turn-timeout deaths in the suite, and they map exactly to the only two smoke-suite tasks pinning
turn_timeout: 300. A single agent turn that wraps a longpip install/uipinvocation legitimately exceeds 5 minutes on CI.What
Raises
turn_timeoutfrom 300 to 1200 in:tests/tasks/uipath-agents/coded/guardrails/smoke/smoke.yamltests/tasks/uipath-agents/coded/guardrails/escalation/escalation.yaml1200 is the suite standard — 99 of 106
turn_timeoutpins undertests/tasks/uipath-agents/already use it, including every sibling guardrail task. Neither task setstask_timeout, so nothing else caps the run sooner.Three other tasks pin
turn_timeout: 300(deploy_folder_and_rebump,push_pull_roundtrip, maestro-flowrouting_listing) but each pairs it with a deliberate shorttask_timeout(300–600s) and none has failed on turn timeout — left untouched.🤖 Generated with Claude Code