ci: actually run the app test suite, and stop the Merge Gate certifying skips (#2447)

[dcccrypto]

Closes the measurement half of #2447. Both changes here are non-disruptive by design — no currently-green PR turns red.

The problem (verified, not taken on trust)

percolator-launch runs zero tests in CI on playground.

Claim	How I checked	Result
packages/ is gone	git cat-file -e origin/playground:packages/shared/package.json	absent on playground and main
So all 4 test jobs skip	they gate on detect-packages.outputs.has_packages == 'true'	permanently false
Nothing runs the app suite	grepped all 4 workflows for vitest/pnpm test	0 matches in any of them
app has a suite	app/package.json	"test": "vitest run"
The suite is red	ran it on clean origin/playground	95 failed / 2591 passed (2702), 28 failed files

The ✅ Merge Gate passed through all of this, because its if: only rejects failure/cancelled — and skipped is neither. Four of its five needs are always skipped, so it passed on type-check alone while printing "All test suites passed".

What this PR does

1. New app-tests job — runs cd app && pnpm test, counts teed into the job summary:

### App test suite (`cd app && pnpm test`)

     Test Files  28 failed | 230 passed | 1 skipped (259)
          Tests  95 failed | 2591 passed | 16 skipped (2702)

Suite is red. This job is non-blocking, so it does NOT gate this PR (#2447).

continue-on-error: true, per the sequencing @ the issue proposed. Those 95 failures predate every open PR; hard-gating on day one would wall all of them off for breakage they did not cause. This lands the measurement — making it blocking is the follow-up, and is the point of landing it.

2. Merge Gate now reports what actually ran — per-dependency results, with skipped marked as certifying nothing. Skipped still does not fail the gate; it just no longer reads as a pass.

Deliberately not in this PR

Both of these turn everyone's CI red on merge and need a sequencing call (left in #2447):

• Un-gate security-tests from has_packages — pnpm audit needs no packages/, but finds 2 critical + 17 high today (fix(deps): raise stale pnpm override floors to current advisory patched versions #2446 fixes those; this should land after).
• Require success() in the gate rather than not-failure — correct end state, but only once the jobs it names can actually run.

Notes

• app-tests is intentionally not in merge-gate.needs: it is non-blocking, and depending on it would put the gate's outcome at the mercy of continue-on-error result semantics for no gain.
• Validated the workflow YAML parses and the job graph has no dangling needs.
• The summary extraction needed an ANSI strip before matching — vitest emits the colour escape ahead of the leading whitespace, so an anchored ^[[:space:]]* silently never matches. Caught by testing against real output rather than assuming.

Refs #2447

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
percolator-launch	Ready	Preview, Comment	Jul 21, 2026 1:32pm
percolator-mainnet	Ready	Preview, Comment	Jul 21, 2026 1:32pm
percolator-playground	Ready	Preview, Comment	Jul 21, 2026 1:32pm

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2fc47550-0566-44dd-bbbb-6c137f94951e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch ci/2447-run-app-test-suite

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[dcccrypto]

CI run is in, and it corrects a number in the description. Run 29834693811.

The mechanism works as designed

Job	Result
App Tests (non-blocking)	failure
✅ Merge Gate	success
Overall run conclusion	success

So the red suite does not gate the PR — continue-on-error behaves as intended. The gate now prints:

⏭️  Unit Tests: SKIPPED — this check certifies nothing
⏭️  Integration Tests: SKIPPED — this check certifies nothing
⏭️  E2E Tests: SKIPPED — this check certifies nothing
⏭️  Security Tests: SKIPPED — this check certifies nothing
✅ Type Check: passed
ℹ️  App Tests: non-blocking — see its job summary for pass/fail counts

No required job failed. Skipped jobs are not evidence of passing tests.

Correction: the real number is 65, not 95

Environment	Test files	Tests
My local macOS run	28 failed / 230 passed	95 failed / 2591 passed
CI (ubuntu-latest)	26 failed / 232 passed	65 failed / 2621 passed

The 95 in the description and commit message was measured locally and matched #2447's figure exactly — but CI disagrees by 30 tests. CI is the authoritative number; treat 65 as the real backlog. The ~30-test gap is environment-dependent (macOS vs ubuntu — likely timezone/locale/timing-sensitive tests), which is its own finding: there are tests that fail only off-CI, and they were invisible in both directions until now.

I'd rather flag the discrepancy than quietly let the 95 stand. Whoever picks up the follow-up should re-baseline against this job, not against either of the earlier local numbers.

This is, incidentally, the argument for landing the measurement: nobody could have known the CI figure differed, because CI had never run the suite.