test(upgrade): promote 3 QA cross-version upgrade read-visibility anchors#1900
test(upgrade): promote 3 QA cross-version upgrade read-visibility anchors#1900kriszyp wants to merge 1 commit into
Conversation
…hors Pins three properties of the gh#1865 upgrade-boot read-visibility investigation that the existing suite does not cover, all verified green at 07c2bbc: - cross-version-upgrade-visibility (QA-658/P-437): a REAL 5.1.22 -> current cross-version upgrade boot (real npm-provisioned legacy install, real dataRootDir handoff) keeps every pre-upgrade component-table row visible on all six read surfaces — the gh#1865 refutation's genuine cross-version arm. - cross-version-upgrade-residuals (QA-660/P-438): the same cross-version upgrade stays read-consistent at 55k rows through a SIGKILL-dirty shutdown and writes racing the boot boundary — the scale/shape + dirty-shutdown residual QA-658 left untested. - upgrade-boot-schema-change (QA-647/P-432): a same-build restart plus an attribute-adding schema change across the boot never makes existing rows invisible on any read surface (6-surface matrix, two restarts) — the same-build control arm the two cross-version suites are compared against. cross-version-upgrade-visibility and cross-version-upgrade-residuals npm-provision harper@5.1.22 at runtime and skip cleanly if that install isn't present, so they carry a network dependency in CI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive integration tests and fixtures to verify read visibility across various database surfaces during cross-version upgrades and schema changes. The feedback highlights a redundancy in the cmp helper function within upgrade-boot-schema-change.test.ts, where the parameter b is passed but remains unused because the value is re-derived dynamically. Addressing this redundancy will improve code clarity and maintainability.
| const cmp = (a: string[], b: string[], name: string) => { | ||
| const wasOk = matrixAfterRestart1[name as keyof SurfaceMatrix] as unknown as string[]; | ||
| log( | ||
| `Q4 boot-order check (${name}): restart#1 misses=${(wasOk as string[]).length}, ` + | ||
| `restart#2 misses=${a.length} -> ${ | ||
| a.length === 0 && (wasOk as string[]).length > 0 | ||
| ? 'CLEARED on 2nd boot (order-dependent)' | ||
| : a.length > 0 && (wasOk as string[]).length === 0 | ||
| ? 'NEW MISS introduced by 2nd boot alone' | ||
| : a.length === (wasOk as string[]).length | ||
| ? 'UNCHANGED across boots (stable)' | ||
| : 'CHANGED (different miss set/count)' | ||
| }` | ||
| ); | ||
| }; |
There was a problem hiding this comment.
The cmp function's implementation is a bit confusing. It accepts a b parameter but then re-derives the same value from matrixAfterRestart1 using the name parameter, leaving b unused. This is redundant and makes the code harder to read.
For better clarity and maintainability, you should use the b parameter directly.
const cmp = (a: string[], b: string[], name: string) => {
log(
`Q4 boot-order check (${name}): restart#1 misses=${b.length}, ` +
`restart#2 misses=${a.length} -> ${
a.length === 0 && b.length > 0
? 'CLEARED on 2nd boot (order-dependent)'
: a.length > 0 && b.length === 0
? 'NEW MISS introduced by 2nd boot alone'
: a.length === b.length
? 'UNCHANGED across boots (stable)'
: 'CHANGED (different miss set/count)'
}`
);
};| log(`(b) dirty-shutdown trigger: ${acked}/${numChunks} chunk requests ACKed (client received 200) before kill`); | ||
| const proc = ctx.harper.process; | ||
| const pid = proc.pid!; | ||
| process.kill(-pid, 'SIGKILL'); |
There was a problem hiding this comment.
Suggestion (non-blocking): process.kill(-pid, 'SIGKILL') targets an entire process group, which only does what's intended if ctx.harper.process was spawned with detached: true (making it its own group leader). Two other tests in this repo doing the same "dirty crash" kill (server/crash-replay.test.ts:26, server/replay-stress.test.ts:167) use the simpler ctx.harper.process.kill('SIGKILL') — no negative pid. If the framework doesn't spawn detached, -pid here will likely throw ESRCH (uncaught, since there's no try/catch around this call) instead of delivering the kill. Consider matching the established ctx.harper.process.kill('SIGKILL') pattern unless there's a specific reason this test needs to also reap child processes of the Harper process itself.
|
Reviewed; no blockers found. |
| resolve(process.env.HOME ?? '', 'dev/tmp/qa658-harper51/node_modules/harper/dist/bin/harper.js'); | ||
| const legacy51Available = existsSync(LEGACY_51_BIN_PATH); | ||
|
|
||
| const skipSuite = process.platform === 'win32' || !legacy51Available; |
There was a problem hiding this comment.
High: this anchor never runs in CI, so it provides no actual regression protection for #1865
skipSuite is true whenever !legacy51Available, and LEGACY_51_BIN_PATH defaults to ~/dev/tmp/qa658-harper51/node_modules/harper/dist/bin/harper.js — a path that only exists on a developer machine that manually ran the npm install harper@5.1.22 --no-save step described in the file header. Nothing in this PR provisions HARPER_LEGACY_51_PATH (or that default path) in CI: .github/workflows/integration-tests.yml already provisions a legacy build for the 4.x suite via HARPER_LEGACY_VERSION_PATH (lines 113/192/291), but has no equivalent step for HARPER_LEGACY_51_PATH. So on every CI run this whole suite silently skips — the PR title's "promote into the tracked suite" doesn't hold in practice; the test exists in the repo but never executes anywhere except a machine with that exact local install, which means it can't fail on the cross-version regression it's meant to anchor.
Suggested fix: add a CI step (mirroring the existing HARPER_LEGACY_VERSION_PATH provisioning in integration-tests.yml) that npm-installs harper@5.1.22 into a known path and sets HARPER_LEGACY_51_PATH, or explicitly document in the PR that this anchor is local-only until that's wired up.
—
Generated by Barber AI
| process.env.HARPER_LEGACY_51_PATH ?? | ||
| resolve(process.env.HOME ?? '', 'dev/tmp/qa658-harper51/node_modules/harper/dist/bin/harper.js'); | ||
| const legacy51Available = existsSync(LEGACY_51_BIN_PATH); | ||
| const skipSuite = process.platform === 'win32' || !legacy51Available; |
There was a problem hiding this comment.
High: same CI-provisioning gap as cross-version-upgrade-visibility.test.ts — this anchor also never runs in CI
Same !legacy51Available skip condition, same unprovisioned default path. Combined with the sibling file, that's 2 of the 3 "promoted" anchors in this PR that will perpetually skip in CI (only upgrade-boot-schema-change.test.ts — the same-build control arm, not a cross-version boot — actually runs unconditionally). The two tests that exercise the actual #1865 cross-version scenario are the ones gated off.
Suggested fix: same as the sibling file — provision HARPER_LEGACY_51_PATH in CI (or the shared default path) alongside the existing 4.x legacy-install step, so these anchors run somewhere that can actually catch a regression.
—
Generated by Barber AI
Summary
Promotes 3 gated qa-explorer regression specs into the tracked
integrationTests/upgrade/suite, anchoring findings from the gh#1865 upgrade-boot read-visibility investigation. All three are test-only (no product code touched) and each stays its own file — allrequires-isolation(they boot a real npm-provisionedharper@5.1.22and/or restart/SIGKILL the instance).cross-version-upgrade-visibility.test.ts(from QA-658 / P-437): a REAL 5.1.22 → current cross-version upgrade boot keeps every pre-upgrade component-table row visible on all six read surfaces — the gh#1865 refutation. Measured: 6.7s.cross-version-upgrade-residuals.test.ts(from QA-660 / P-438): the same cross-version upgrade stays read-consistent at 55k rows through a SIGKILL-dirty shutdown and boot-boundary writes — closes the scale/shape + dirty-shutdown gap QA-658 left untested. Measured: 15.9s.upgrade-boot-schema-change.test.ts(from QA-647 / P-432): restart + attribute-adding schema change never makes existing rows invisible on any read surface (6-surface matrix, two restarts) — the same-build control arm the two cross-version suites are compared against. Measured: 8.0s.Network dependency:
cross-version-upgrade-visibilityandcross-version-upgrade-residualsnpm-provisionharper@5.1.22at runtime (via~/dev/tmp/qa658-harper51, overridable withHARPER_LEGACY_51_PATH) and skip cleanly if that install isn't present. This is a CI network dependency — leaving it to review whether/how to provision in CI.Test plan
npm run build(known pre-existingresources/DatabaseTransaction.tstype error unrelated to this change; dist emits fine)timeout:cross-version-upgrade-visibility.test.ts: 4 pass, 1 skip (no-miss-to-heal), 6.7scross-version-upgrade-residuals.test.ts: 5 pass, 1 skip (no-miss-to-heal), 15.9supgrade-boot-schema-change.test.ts: 4 pass, 1 skip (no-miss-to-heal), 8.0soxlintclean,prettier --writeapplied🤖 Promoted by qa-explorer (Claude Opus 4.8 orchestrator / Sonnet promotion agent)