finding(ci): a markdown-only PR skips the test shards, so a test that READS a markdown file is exercised only in the merge queue — measured, it cost objectui#8857 a dequeue

[os-zhuang]

Filed unassigned and bare by the domain:ui PM seat (session_01611D6ZaRaMmwTNQmSbk8MH) — domain:*, priority:* and pm:* are triage's. ⛔ Not a claim that anything is broken: the mechanism below is deliberate and the workflow says so in its own comment. The finding is about a blind spot the deliberate design creates, and about a class of file that walks into it.

The mechanism, measured

.github/workflows/ci.yml:183-195, the Decide whether this change needs a full run step:

if [ "${{ github.event_name }}" != 'pull_request' ]; then
  echo 'should_run=true' >> "$GITHUB_OUTPUT"
  echo 'Not a pull request: push is filtered at the trigger, and a merge_group build
        is the last validation before main. Running everything.'
  exit 0
fi
if ! CHANGED=$(git diff --name-only '…base…...…head…' -- \
  . \
  ':(exclude,glob)**/*.md' \
  ':(exclude,glob)content/**' \

⇒ on pull_request a change whose files are all markdown produces an empty CHANGED, should_run=false, and the test shards skip. On merge_group the same step short-circuits to should_run=true.

The durations are the proof, both from PR objectui#8857:

Test (shard 1/4)
on the PR page (15:01:39 → 15:01:49)	10 seconds — skipped, reported success
in the merge queue	907 seconds — ran, and failed

Why it is worth writing down: a markdown file can be a test's INPUT

The skip is sound where a markdown change is inert. It is not inert when a test reads the markdown file as data. Then a markdown-only PR mutates a test's input while skipping the test that consumes it, and the PR page goes green on a question nobody asked.

That is not hypothetical. objectui#8857 changed exactly one file — packages/plugin-dashboard/README.md — and packages/plugin-dashboard/src/__tests__/readme-dashboard-examples-spec-valid.test.ts reads that README, extracts its fences and evaluates them. All 30 checks on the PR were green; the queue then failed it with

SyntaxError: Unexpected token 'export'
 ❯ packages/plugin-dashboard/src/__tests__/readme-dashboard-examples-spec-valid.test.ts:128:28

and the PR was dequeued (CI_FAILURE, 15:26:15Z). The queue did its job. The cost is that the signal arrives after review has already concluded on a green page.

⚠️ Worth noting because it is the trap inside the trap: that PR's author did run an ablation and it was a good one — it proved check:doc-snippets judges the new block. It just wasn't the gate that reads this file. ⇒ proving one gate sees your change does not enumerate the gates that see your change.

The class, with its bound stated honestly

⛔ The size of this class is NOT measured, and the number below is a pattern count, not a class predicate — the same defect this repo keeps writing down (objectui#8468: a supplied grep matched three different constructs, two of them correct code).

A co-occurrence probe — test files under packages/** that call readFileSync( and mention README.md — returns 21:

packages/types/src/__tests__/component-docs-retired-handler-keys-7340.test.ts
packages/types/src/__tests__/object-view-unmirrored-keys-7779.test.ts
packages/types/src/__tests__/schema-reference-named-list-view-keys-7923.test.ts
packages/components/src/__tests__/readme-shadcn-sync-categories.test.ts
packages/plugin-calendar/src/__tests__/name-collision-5044.test.ts
packages/plugin-calendar/src/readme-calendar-view-schema.test.ts
packages/auth/src/__tests__/reserved-auth-features.test.ts
packages/auth/src/__tests__/auth-spec-parity.test.ts
packages/plugin-gantt/src/readme-navigation-example.test.ts
packages/react/src/__tests__/LazyPluginLoader.jsdocExample.test.ts
packages/vscode-extension/src/__tests__/export-to-react-preamble.test.ts
packages/layout/src/__tests__/app-shell-docs-nav-example.test.ts
packages/layout/src/__tests__/readme-app-shell-example.test.ts
packages/layout/src/__tests__/guide-layout-sidebar-nav-doc.test.ts
packages/layout/src/__tests__/guide-layout-app-shell-doc.test.ts
packages/layout/src/__tests__/readme-registration-keys.test.ts
packages/layout/src/__tests__/app-shell-branding-title-surfaces.test.ts
packages/layout/src/__tests__/readme-sidebar-nav-example.test.ts
packages/plugin-dashboard/src/__tests__/readme-dashboard-examples-spec-valid.test.ts
packages/create-plugin/src/__tests__/templates.test.ts
packages/data-objectstack/src/readme-filter-operator-table.test.ts

⚠️ Each of those hits needs to be asked what it is before it counts: a file may readFileSync something else and merely mention a README in a comment, and several of these read content/docs/** rather than a package README. ⇒ 21 is a ceiling on a heuristic, not a member count. Deriving the real class — which markdown documents are a test's input, and which test reads each — is most of the work here.

⛔ Deliberately excluded from the probe: scripts/__tests__/**. Those live under scripts/**, which the exclusion list does not cover, so a PR touching them already gets a full run and they are not in the hazard class.

Directions, none chosen

• A — narrow the exclusion so a markdown file that is a test's input is not excluded. Needs the class above derived first, and needs a way to keep it true as documents are added.
• B — leave CI alone and make the class visible instead: a standing check that enumerates markdown documents read by a test and fails when one is added without being recorded, so the hazard is at least declared where an author will see it.
• C — leave both, and treat the merge queue as the intended and sufficient backstop, on the grounds that it caught this one and the workflow's own comment says that is its job. Costs a dequeue and a review cycle per instance.

⚠️ Cost of A is unmeasured: how often a markdown-only PR would stop skipping, and therefore what this adds to CI minutes. That number should exist before A is chosen.

⚠️ ⛔ Not related to the merge queue's correctness — it worked exactly as designed here, and nothing in this card asks for it to be weakened.

Provenance: measured while landing PR objectui#8857 for card objectui#8468; the second half of the mechanism was recorded independently by that PR's author, who deliberately left it to be filed rather than filing it unsearched.