fix(shallow): honour the shallow boundary across clone, gc, fsck and rev-list

[mizchi]

Fixes #184, #185, #186, #187, #188, #189.

A shallow clone records which commits have no parents on disk. bit got that record wrong and then walked past it in four places — the worst of which deleted history without saying so.

None of this is covered by the git-compat suite: t5537-fetch-shallow.sh and friends run upstream git's porcelain against bit's four shimmed plumbing commands, so bit clone --depth, bit gc and bit rev-list never execute. Reproducing any of it also needs --no-git-fallback; without it bit delegates these commands to the real git binary and every difference disappears.

What was wrong

clone --depth recorded the wanted ref tips (upload_pack_process.mbt) instead of the boundary the server reported. Those coincide only at depth 1, so any deeper clone claimed its tip was parentless:

depth | git writes | bit wrote
  1   |     c6     |    c6
  2   |     c5     |    c6
  3   |     c4     |    c6

repack -ad believed it, packed the tip alone and dropped the pack holding the rest — exit 0, no warning, objects unreadable by git afterwards:

depth 1: 1 -> 1      depth 3: 3 -> 1   <-- data loss
depth 2: 2 -> 1      depth 4: 4 -> 1   <-- data loss

The native HTTP clone never wrote the file at all, because prepare_clone_with_http used the fetch_pack_with_http variant that discards the shallow lines. The repository then reported itself complete, and fetch --unshallow — gated on that file existing — short-circuited as already up to date, so such a clone could never be completed (git 6 commits, bit 3). #183 closed the same gap on the JS path; the native one still had it.

gc, prune and repack walked from the ref tips with no boundary and aborted with Missing commit object. #183 added the shallow~ parameter for exactly this and left the call sites unwired.

fsck followed the same absent parents and reported a well-formed shallow clone as corrupt.

rev-list emitted the boundary's parent — an id with no object behind it — so it counted one more commit than the repository holds while log counted correctly.

The fix

• clone_process_to_fs takes fetch_pack_process_result and hands the response's lines to @repo.apply_shallow_updates.
• PreparedClone carries shallow/unshallow; both the sync and async HTTP writers apply them. New write_shallow_boundaries_async covers the async writer, which has no AsyncRepoFileSystem to merge against — a fresh clone has nothing to merge.
• gc_git_dir, repack_git_dir, prune_git_dir and the reflog walk pass @repo.read_shallow_boundaries.
• fsck_connectivity_check takes shallow~ and stops following parents at a boundary commit; the CLI passes the repository's.
• The rev-list walk stops at a boundary commit, as log already did.

Verification

A differential harness ran git × bit × {full clone, shallow clone} over both file:// and smart HTTP, reporting only what breaks on the shallow clone while matching on the full one — so unimplemented-command noise is excluded. 17 shallow-specific divergences before, 0 after (matched 14 → 60).

t/t9021-shallow-clone-integrity.sh covers all six against real git over both transports. I built the pre-fix tree to confirm it is a real guard: 9 of its 13 assertions fail there, all 13 pass after. A unit test pins the fsck graft directly.

Two flaws in my own test were caught and fixed during that check, both of which had made it pass against broken code:

• asserting the boundary via git log was tautological, since git log is itself bounded by .git/shallow; it now compares the commit objects on disk against what a boundary-honouring walk shows.
• the HTTP test ended its && chain with kill … || true, which swallowed every assertion before it. Tear-down moved to an EXIT trap.

About the t/ suite — this test does not run in CI

Worth stating plainly, since the new test's placement suggests otherwise: no CI job runs anything under t/. The workflow's jobs are check, the JS/WASM and native test runs, the cmd-native-test shards, distributed-test, nix-build, git-compat and js-build; none of them invokes the test-subdir task. t9021 is therefore reached only by pkf run test-subdir or bash t/run-tests.sh locally, as is the existing t9020.

Within that local task, the filter is a substring match, so t900 alone skipped t9011–t9021. This PR adds a t902 pass so the task at least covers t9020 and the new test. t9011–t9018 remain uncovered by any filter, and two of those tests are red today — see #191, where fixtures/workspace_flow/bootstrap.sh uses a bare git init and the tests hardcode main. Wiring the whole t/ tree into CI wants that fixed first, so it is left out of this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VDVUJHu38YtVAr6sKZtRDB