Skip to content

fix(cli): share injected source pairs between process inject and upload - #91427

Merged
trunk-io[bot] merged 2 commits into
masterfrom
cat/cli-sourcemap-process-pairs
Sep 1, 2026
Merged

fix(cli): share injected source pairs between process inject and upload#91427
trunk-io[bot] merged 2 commits into
masterfrom
cat/cli-sourcemap-process-pairs

Conversation

@cat-ph

@cat-ph cat-ph commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Problem

posthog-cli sourcemap process can abort a user's production build with Chunk ID not found (exit 1) when something writes into the scanned directory while the command runs. Next.js 16.3+ users hit this on every build: Turbopack's filesystem cache (on by default there) keeps flushing files into .next in the background while @posthog/nextjs-config runs the CLI, reported in posthog-js#4667.

The cause is a time-of-check/time-of-use race inside process: the inject pass and the upload pass each walk the directory roots and read the files independently. A chunk pair that materializes (or changes) between the passes reaches upload without an injected chunk ID, and one such pair fails the entire upload.

This PR is an alternative to #91191 (path-snapshot approach). Review on that PR surfaced three semantic gaps the snapshot needs shims for: upload re-reads contents so an overwritten chunk still aborts, inject re-validates existence of every snapshotted path so a concurrently deleted cache file aborts, and deriving cleanup roots from snapshot entries narrows the --delete-after safety guard under --include/--exclude. This PR removes the whole class instead: land one of the two, close the other.

Changes

  • process now reads the source pairs from disk once, injects them, and uploads those same in-memory pairs. There is no second directory walk and no re-read, so a build no longer fails when files appear, change, or vanish mid-run, and upload sends exactly the bytes inject stamped.
  • --delete-after cleanup keeps deriving its safety roots from the original selection, so its authorization scope is unchanged.
  • The Chunk ID not found error now names the offending file and says how to recover.
  • --delete-after cleanup no longer fails the build after a successful upload when a file vanished or was replaced mid-run: every artifact is fingerprinted when read, and cleanup only strips or deletes what still matches - through the verifying file handle for sources, and via an atomic rename-verify-remove for maps. Skipped files warn instead of aborting, a map shared by several chunks is only deleted once every source referencing it verified, and read-only sources that need no stripping no longer require write access.
  • Overlapping selection roots now process each file once, instead of stamping it repeatedly and uploading a stale copy.
  • The injecting selection log line is bounded by entry count and byte budget. A large stdin-provided selection used to render as one ~1 MB line, which killed the CLI with EAGAIN when stderr was a non-blocking pipe (e.g. spawned from Node.js with inherited stdio).
  • Mechanical: inject_impl and plain::inject::inject return the injected pairs; upload is split into upload (walks, standalone command) and upload_pairs (consumes given pairs); standalone sourcemap inject / sourcemap upload behavior is unchanged.

How did you test this code?

  • New unit test inject_impl_returns_the_injected_pairs: injects a fixture pair and asserts the returned in-memory pair carries a chunk ID and matches the bytes written to disk — the property upload_pairs relies on. The freeze itself is structural: upload consumes the pairs it is given and cannot see later files.
  • New unit tests display_is_bounded_for_large_selections, display_is_bounded_for_long_paths, and display_truncates_a_single_oversized_path pin the log-line byte bound.
  • Seven cleanup unit tests pin the --delete-after hardening: happy path, missing source, replaced source, replaced map, shared map with a skipped source, shared map matching any uploaded copy, and overlapping-roots dedupe.
  • Full cargo test for the CLI crate, cargo clippy --all-targets, and cargo fmt are green locally.
  • Not checked: an end-to-end next build against a live PostHog instance with this CLI build.

Automatic notifications

  • Publish to changelog?

Docs update

Sampo changeset included (cli/.sampo/changesets/sourcemap-process-shared-pairs.md); no docs describe the process internals.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

  • Claude Code (Fable 5) built this after review on fix(cli): freeze sourcemap process selection across inject and upload #91191 (Greptile + Codex bots) showed the path-snapshot approach needed accumulating shims; threading the pairs makes the shared-set invariant structural instead of patched.
  • Considered stacking on fix(cli): freeze sourcemap process selection across inject and upload #91191 and rejected it: this supersedes rather than extends, so landing both in sequence would add and immediately delete the snapshot code.
  • Structured second-model review (Codex) ran repeatedly over the diff; its rounds shaped the log-line truncation and the entire cleanup hardening (fingerprints over chunk-id checks, shared-map grouping, handle-bound writes, rename-verify-remove, non-clobbering restore). The review-round cap was reached; the one finding consciously not addressed is multiple symlink aliases of the same map inside the selection, where deletion operates on the first alias and later aliases survive - the failure direction is a retained file, and the case has no known real-world producer (noted in the code).

@cat-ph cat-ph self-assigned this Aug 31, 2026
@trunk-io

trunk-io Bot commented Aug 31, 2026

Copy link
Copy Markdown

😎 Merged successfully - details.

@github-actions

Copy link
Copy Markdown
Contributor

🤖 CI report

Trunk lane — non-backend lane

This PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes.

@cat-ph
cat-ph marked this pull request as ready for review August 31, 2026 18:58
@posthog

posthog Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

🦔 PostHog Review reviewed this pull request

Nothing worth raising this time, so here's a calming picture instead:

A happy dog on a sunny path

Resolved comments: 2 declined

@greptile-apps

greptile-apps Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
### Issue 1
cli/src/commands.rs:424-429
**Comments preserve change history**

These comments contrast the new handoff with the former directory walk and cite the originating regression, with the same history repeated in `plain/upload.rs` and `source_pairs.rs`. Repository guidance reserves that context for the PR or commit message; keeping it in the implementation creates stale commentary that obscures the current invariant.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "fix(cli): harden delete-after cleanup ag..." | Re-trigger Greptile

Comment thread cli/src/commands.rs
Comment thread cli/src/sourcemaps/args.rs
@trunk-io
trunk-io Bot merged commit 13dc107 into master Sep 1, 2026
328 of 329 checks passed
@trunk-io
trunk-io Bot deleted the cat/cli-sourcemap-process-pairs branch September 1, 2026 22:50
@deployment-status-posthog

deployment-status-posthog Bot commented Sep 1, 2026

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-09-01 23:12 UTC Run
prod-us ✅ Deployed 2026-09-01 23:23 UTC Run
prod-eu ✅ Deployed 2026-09-01 23:23 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants