ci: stop swallowing integration test output on failure - #104
Merged
Conversation
Each integration step ran the test runner under `set -e` with `output=$(uv run python ...)`. When the runner exits non-zero, `set -e` aborts the step at the capture line, so the subsequent `echo "$output"` never runs and all failure detail (failed test names, tracebacks, API errors) is discarded. Logs jump straight from `##[endgroup]` to `exit code 1`, making failures impossible to diagnose. Capture the runner's exit status explicitly (`set +e` around the call), always echo the captured output, and treat a non-zero exit as failure in addition to the existing "Failed:" grep. The runner already returns exit code 1 on any failed example, so this is the more reliable signal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When an integration test fails in CI, the job log shows nothing between
##[endgroup]and##[error]Process completed with exit code 1— all the diagnostic output (failed test names, tracebacks, API errors) is discarded. This made the 0.0.36 release failure impossible to diagnose from the logs.Root cause
Each integration step runs:
The runner prints all per-test detail to stdout, which the command substitution captures into
$output. Withset -e, a non-zero exit from the runner aborts the step at the assignment line, soecho "$output"never executes and the captured output is thrown away.Fix
Capture the runner's exit status explicitly, always echo the output, and treat a non-zero exit as failure (in addition to the existing
"Failed:"grep — the runner alreadyexit 1s on any failed example, so the exit code is the more reliable signal):Applied uniformly to all 12 integration-test steps.
Notes
This is a pure CI-observability fix — no test or framework code changes. It does not change which tests run or pass; it only ensures failures are visible in the logs.
🤖 Generated with Claude Code