DX-3053: Stop one integration file deleting every other file's snapshots - #248
Merged
Merged
Conversation
Vitest runs integration files in parallel, and delete-snapshots called the account-wide delete twice, once as a test and once in cleanup. Nine other files depend on their own snapshot surviving, so whichever of them happened to be mid-run lost it and failed with "Snapshot is not ready". That is why the red tests moved around between runs: the directory tests one time, a snapshot name test the next. The by-id tests now track what they created and delete only that. The account-wide case is kept but gated behind UPSTASH_BOX_ALLOW_ACCOUNT_WIDE, because it cannot share a key with anything else, including a second CI run on another pull request. Its request shape is already asserted in box-delete-snapshots.test.ts, so CI keeps that guarantee without the hazard. Also adds a case pinning that an empty id list is rejected rather than treated as "everything", which is the bug CLOUD-4771 fixed. Verified by running delete-snapshots, cd and name together against a live API: 37 passed. Those three collided before.
alitariksahin
force-pushed
the
DX-3053
branch
from
September 21, 2026 16:04
81156a1 to
8181245
Compare
ytkimirti
approved these changes
Sep 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The integration job has been red on
mainsince at least 9 September, and which tests fail moves around between runs. Two separate causes. This handles the one that lives in this repo.The suite was sabotaging itself
Vitest runs integration files in parallel, and
delete-snapshots.integration.test.tscalled the account-wide delete twice: once as a test, and once inafterAll. Nine other files depend on their own snapshot surviving long enough to restore from it. Whichever of them happened to be mid-run when that fired lost its snapshot and failed withSnapshot is not ready.That explains the drift. One run it was the directory tests, the next a snapshot naming test, and on a third the directory file failed at a different case. Nothing was flaky. The timing of the wipe was what moved.
It is worse than intra-run interference: two CI runs on two pull requests share one key, so a wipe on one can break the other.
What changed
The by-id tests now record what they created and delete only that. Cleanup deletes the same list rather than everything.
The account-wide case is kept, not deleted, but gated behind
UPSTASH_BOX_ALLOW_ACCOUNT_WIDE=1. It cannot safely share an account with anything else, so CI leaves it unset and it can be run deliberately against a scratch account. Its request shape is already asserted inbox-delete-snapshots.test.ts, which pins the?all=truequery, so the guarantee CI cares about is unchanged.One case added along the way: an empty id list must be rejected rather than read as "everything". That is the bug CLOUD-4771 fixed, and nothing in the integration suite held it in place.
Verified
Ran the three files that previously collided together against a live API, in parallel, as CI does:
The two skips are the account-wide describe, correctly opted out.
The other cause is not in this repo
The two
browser.integration.test.tsfailures are a product bug: closing the last tab in a box makes Chromium exit, and the close then reports that teardown as an error although the tab is gone. Reproduced against production, three times out of three, with closing a non-last tab succeeding every time.That is fixed in upstash/box-backend#272, which is open. Until it merges those two tests stay red here, and this pull request does not paper over them.
A third failure, uncovered by this change
With the snapshot wipe gone,
cd / cwd > agent.run respects cwd after cdruns its assertions for the first time in a while, and fails on its own merits:The snapshot bug was masking it: the test used to die in
beforeEachbefore reaching the assertion.It is not a flaky model answer. The agent names its directory explicitly, and reproducing against production gives the same result twice:
So this is not session pinning. The reason is narrower than "the folder is ignored", and my first reading of it was wrong.
runner-claude.tsdoes receive and use the folder. It deliberately pins Claude's process directory to/workspace/homeso that session transcripts stay in one place across different folder values, and communicates the selected folder two other ways: it adds it toadditionalDirectories, and it tells Claude in the system prompt that this is the working directory and to prefix shell commands with acd. A prompt instruction is not a process working directory, so relative paths still resolve against the real one, which is what the failure shows.runner-codex.tshas no such workaround and passesworkingDirectory: WORK_DIRstraight through, so the inconsistency is specific to the Claude runner.That is a backend gap rather than anything this pull request can fix, and I have deliberately not skipped the test to make the job green.
It needs its own ticket: Claude agent runs must honour the selected working directory without breaking session continuation. Documenting agents as exempt from
cd()would paper over an inconsistency rather than fix it, so the test stays as written. The fix is not simply swapping thecwdvalue either, since the pinned directory exists to keep sessions resumable, and that has to keep working.