fix(harness): classify pi terminal outcomes honestly - #2345
coreplane-switchboard[bot] wants to merge 1 commit into
Conversation
Co-Authored-By: coreplane-switchboard[bot] <318072483+coreplane-switchboard[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Changes requested: The runtime distinction is sound, but two covering behavioral specs still promise the superseded ProviderFailure-only behavior.
Warning
Changes requested · head 5458e8b · 2 findings: 2 minor
| Severity | Finding | Where |
|---|---|---|
| minor | F1 Spec contradiction — docs/reference/specs/harness.md item 1: local and unknown terminal results no longer cross as ProviderFailure | docs/reference/specs/harness.md:11 |
| minor | F2 Spec contradiction — docs/reference/specs/harness-pi.md item 6: a client-side abort is now local-only, not provider-down | docs/reference/specs/harness-pi.md:16 |
Full review
F1 — harness.md still states that every failed model answer crosses as a ProviderFailure.cause. The new local_abort and unknown variants deliberately do not, leaving the harness contract and its validation criterion false. Update item 1 to describe the discriminated terminal-result boundary.
F2 — harness-pi.md item 6 still says an ordinary client-side abort is a provider-down failure that may be released by provider-up. The implementation now correctly records it as local cancellation and excludes provider-up release. Coordinate with #1625 so the held spec and its validation rows are updated before this behavior merges.
Requested by justin · Thread
Pi terminal results now preserve local cancellation, unknown outcomes, provider refusal, and provider failures as separate types. Runs no longer invent a permanent provider refusal when pi retained no such evidence.
#2340 showed two coding runs ending together on pi-local aborts but rendered as provider refusals. The truthful fix belongs at the pi terminal boundary.
Rendered by the plan runner from the coding run's submitted description; the run ended before it could open the pull request itself.