Skip to content

CI: Test Summary job succeeds even when required jobs fail #419

Description

@debugmcpdev

What happens

The test-summary job in .github/workflows/ci.yml runs with if: always() and its only step unconditionally prints a success message:

  test-summary:
    name: Test Summary
    needs: [build-and-test, container-tests]
    runs-on: ubuntu-latest
    if: always()

    steps:
    - name: Check test results
      run: |
        echo "✅ CI pipeline completed"
        echo "Check the test results in the build-and-test job"

When build-and-test or container-tests fails, the workflow run is red overall, but Test Summary itself shows green with "✅ CI pipeline completed". Anyone scanning the job list (or tooling keying off the job) gets a misleading success signal. This happened in practice during the container-build failures on the PR #392 and #393 runs: the container job failed while Test Summary reported completion.

Fix

Keep if: always() (so the job still runs and can name the failure), but make the step evaluate needs.*.result and exit non-zero unless every needed job succeeded:

    - name: Check test results
      run: |
        echo "build-and-test: ${{ needs.build-and-test.result }}"
        echo "container-tests: ${{ needs.container-tests.result }}"
        if [ "${{ needs.build-and-test.result }}" != "success" ] || [ "${{ needs.container-tests.result }}" != "success" ]; then
          echo "❌ One or more required jobs did not succeed"
          exit 1
        fi
        echo "✅ All required jobs succeeded"

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggithub_actionsPull requests that update GitHub Actions code

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions