Skip to content

build: follow moonbit 0.10.13 and moonbitlang/async 0.22, update all deps - #179

Merged
mizchi merged 10 commits into
mainfrom
claude/wizardly-pascal-i79mco
Sep 17, 2026
Merged

mizchi merged 10 commits into
mainfrom
claude/wizardly-pascal-i79mco

Conversation

@mizchi

@mizchi mizchi commented Sep 16, 2026 •

Copy link
Copy Markdown
Member

Summary

The workspace no longer built on the current MoonBit toolchain (0.1.20260915 / moonc 0.10.13): the pinned moonbitlang/x@0.4.40 and bobzhang/toml@0.1.7 fail to compile, and moonbitlang/async changed its HTTP header and cancellation APIs. This PR updates every third-party dependency to its latest release and adapts the code.

Dependency bumps (modules/*/moon.mod, component/moon.mod.json)

package before after
moonbitlang/async 0.19.4 0.22.1
moonbitlang/x 0.4.40 0.5.5
bobzhang/toml 0.1.7 0.4.3
mizchi/x 0.2.0 0.6.1
mizchi/zlib 0.4.8 0.4.9
mizchi/simd 0.4.1 0.6.1
mizchi/llm 0.2.2 0.3.2

mizchi/tempfile, mizchi/libgit2 and mizchi/experimental_crypto were already at their latest release.

Code changes

  • HTTP headers: @http now keys request/response headers by CaseInsensitiveString. bit keeps plain String maps in its transport-neutral interfaces and converts at the boundary (bit_io_native/http_client_native.mbt, cmd/bit/serve_http.mbt).
  • @fs.read_dir returns an ArrayView; the three RepoFileSystem::readdir impls materialise it.
  • moonbitlang/core/strconv is empty now; parse_int/parse_int64 come from @string. Dead imports removed.
  • async cancellation is no longer an error and cannot be caught, and @async.is_cancellation_error is deprecated (it always returns false). The 123 err if @async.is_cancellation_error(err) => raise err guards are therefore dead code and were removed; the is_being_cancelled() checks stay.
  • fragile_catch_all lint (new in this compiler, and fatal under pkf run check's --deny-warn): removed the 60 pure re-raise catch { err => raise err } handlers and moved the three cleanup-in-catch sites (bit cat, bit tree temp dirs, HTTP client close) to errdefer, which also makes them run on cancellation.
  • npm CLI payload is now an ES module: moonc emits import statements for the #module("node:fs") FFI bindings that moonbitlang/x 0.5 uses, so the JS build of modules/bit can no longer be required as bit.cjs. It is now copied to npm/bit.mjs; npm/bit.js re-exports it and npm/bin/bit.mjs imports it. package.json exports/files, .gitignore and the pkf task (sync-npm-bit-mjs) follow.
  • JS bundle guardrail rebaselined: the tree-shaken minimal lib bundle grew from ~163.5k to 174,458 raw / 40,265 gzip bytes with the new toolchain (same exported surface). Ceilings raised to 180k / 44k in tools/verify-lib-js-treeshake.mjs; the minimal/git-ops ratio guards are unchanged and still hold (0.56 raw, 0.54 gzip vs 0.70).
  • CI's setup-moonbit action now pins 0.10.13+cbb11c36f (the old pin cannot compile the new dependency releases).
  • moonc stack overflow on the cmd/bit debug build: moonc 0.10.13's link-core pass aborts with an ICE (Error: Stack overflow) when building modules/bit in the debug configuration under the default 8 MiB stack. The same step succeeds with ulimit -s unlimited, and the release build is unaffected. The cmd-native-test job now lifts the limit before building/testing, and AGENTS.md documents it.
  • flake.lock refreshed for nix-build: moon-registry advanced to today's mooncakes index (the 2026-08-12 one lacked bobzhang/toml@0.4.3 and moonbitlang/async@0.22.1, failing with attribute '"0.4.3"' missing), and moonbit-overlay to its current HEAD (86fff168, ships the v0.10.13+cbb11c36f toolchain) together with the nixpkgs/treefmt-nix pins from the overlay's own lock. The overlay entry was hashed with nix flake prefetch over git because the sandbox blocks GitHub tarball downloads; the tree has no export attributes, so it is the hash the github: fetcher yields.
  • Two pre-existing cmd-bit-1 test failures fixed (93e8778, 44b919f), so the shard is green for the first time in a while: bit grep --column -o gave every match after the first a column that summed the preceding matches' ends — the same bug git fixed upstream in "grep: fix --column --only-match for 2nd and later matches" (2.55, the release third_party/git pins) — and serve --http: a repository resolves under the root asserted a .git suffix strip that http_resolve_repo never did and must not do, since a bare repository is the directory acme/app.git.

Verification

  • moon check --deny-warn --warn-list=-27-20-82 (what pkf run check runs) passes for native, js and wasm (also green in CI's test job).
  • moon build --target native --release modules/bit succeeds; bit --version runs.
  • moon build --target native modules/bit (debug) succeeds with ulimit -s unlimited; the resulting binary passes a smoke run (init, add, two commits, status, log) and real git fsck/git log read the repository cleanly.
  • moon test --target native for every non-cmd workspace module plus mizchi/bit/tests: all pass (same loop as the CI full run; also green in CI's test job).
  • moon test --target wasm for bit_runtime, bit_diff3, bit_repo, bit_grep: pass.
  • moon test --target js -p mizchi/bit -p mizchi/bit_lib: 361/363 pass. The two failures are the ssh-ed25519 signing tests, which shell out to ssh-keygen; it is not installed in this sandbox (ssh-keygen failed with status null). They should pass on the CI runner.
  • JS build pipeline reproduced locally (moon build --target js --release for bit_lib and bit, bun bundles, node --test tools/js-build.test.mjs tools/npm-lib.test.mjs tools/npm-cli.test.mjs): 5/5 pass; CI's js-build is green.
  • nix flake metadata (with the overlay resolved over git) reports no pending lock changes. A full local nix build was not possible: the sandbox blocks the GitHub tarball downloads the other flake inputs need, so CI's nix-build is the check.
  • node tools/check-layers.mjs and tools/check-layer-guards.sh pass.
  • The grep --column -o fix was checked against the native debug binary with GIT_CONFIG_* unset (otherwise the shim delegates to the system git): foo_mmap bar mmap now reports columns 5 and 14, abmmapXmmap 3 and 8 — the true columns, where it printed 13 and 9 before.
  • All 22 CI checks are green on 44b919f.

Not done / follow-ups

  • Remaining deprecation warnings (@sys.get_env_var → @env, StringBuilder::new() → StringBuilder(), ~1.5k sites) are warning-only and left as is.

🤖 Generated with Claude Code

https://claude.ai/code/session_018UDsHQQ4RZzjsqix2HYGBv

Bump every third-party dependency to its latest release and adapt the
code to the toolchain and async API changes that broke the build:

- moonbitlang/async 0.19.4 -> 0.22.1, moonbitlang/x 0.4.40 -> 0.5.5,
  bobzhang/toml 0.1.7 -> 0.4.3, mizchi/x 0.2.0 -> 0.6.1,
  mizchi/zlib 0.4.8 -> 0.4.9, mizchi/simd 0.4.1 -> 0.6.1,
  mizchi/llm 0.2.2 -> 0.3.2 (the old moonbitlang/x and toml releases no
  longer compile on the current compiler).
- @http keys headers by CaseInsensitiveString: convert bit's plain
  String header maps at the transport boundary (bit_io_native, bit serve).
- @fs.read_dir now returns an ArrayView: materialise it in the three
  RepoFileSystem::readdir implementations.
- moonbitlang/core/strconv is empty now; use @string.parse_int /
  parse_int64 and drop the dead imports.
- async cancellation is no longer an error and cannot be caught, so the
  `err if @async.is_cancellation_error(err) => raise err` guards are dead
  code (the function is deprecated and always returns false). Remove them
  and keep only the is_being_cancelled() checks.
- Pin CI to the 0.10.13 compiler and bump the component pins to match.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018UDsHQQ4RZzjsqix2HYGBv
`pkf run check` runs moon check with --deny-warn, and the new compiler
adds a `fragile_catch_all` lint for `catch { err => raise err }` and for
cleanup done inside a catch-all handler (which no longer sees async
cancellation). Drop the 60 pure re-raise handlers, move the three
temp-dir / client cleanups to `errdefer`, and fix the fallout:

- two merge-state cleanup helpers no longer raise, so drop `raise Error`
- `@async.sleep` cannot fail any more, so the test doubles in
  lfs_client_wbtest stop catching it
- bit_archive and bit_fast_import imported the now-empty strconv package

moon check --deny-warn --warn-list=-27-20-82 passes for native, js and
wasm.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018UDsHQQ4RZzjsqix2HYGBv
moonc 0.10.13's link-core pass overflows the default 8 MiB stack while
building the debug configuration of modules/bit (an ICE reporting
"Stack overflow"); the same step succeeds with `ulimit -s unlimited`,
and the release build is unaffected. Apply that in the cmd-native-test
job, which builds and tests cmd/bit in debug, and note it in AGENTS.md.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018UDsHQQ4RZzjsqix2HYGBv
moonc now emits `import` statements for the `#module("node:fs")` FFI
bindings that moonbitlang/x 0.5 uses, so the JS build of modules/bit is
an ES module and can no longer be loaded with `require` as bit.cjs
(js-build failed with "Cannot use import statement outside a module").

Copy the payload to npm/bit.mjs instead and import it from the ESM
wrapper (bit.js) and the bin script; update package.json exports/files,
.gitignore and the pkf task name (sync-npm-bit-mjs) to match.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018UDsHQQ4RZzjsqix2HYGBv

mizchi commented Sep 16, 2026

Copy link
Copy Markdown
Member Author

nix-build is red and I cannot fix it from this session.

The job fails inside the moonbit-overlay dependency resolver with

error: attribute '"0.4.3"' missing
Did you mean one of 0.2.3, 0.4.1, 0.4.2, 0.1.5 or 0.1.6?

flake.lock pins the moon-registry input (the mooncakes index) at a 2026-08-12 revision. It predates bobzhang/toml@0.4.3 (2026-08-16) and moonbitlang/async@0.22.1 (2026-09-16), which this PR now depends on. The moonbit-overlay input is from the same date and will need the newer toolchain as well.

Required fix, to be run by someone with nix available (it is not installed in the sandbox this PR was prepared in, and a git+https lock entry needs a narHash I cannot compute without it):

nix flake update moon-registry moonbit-overlay
nix build   # then push the refreshed flake.lock to this branch

Everything else on this PR is unaffected by the lock file.


Generated by Claude Code

The tree-shaken minimal lib bundle grew from ~163.5k to 174,458 raw /
40,265 gzip bytes with the new toolchain and dependency releases
(moonbitlang/x 0.5, async 0.22); the exported surface is unchanged.
Raise the raw and gzip ceilings to 180k / 44k, keeping the same kind of
headroom the previous bumps left. The minimal/git-ops ratio guards are
untouched (now 0.56 raw, 0.54 gzip against a 0.70 ceiling).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018UDsHQQ4RZzjsqix2HYGBv

mizchi commented Sep 16, 2026

Copy link
Copy Markdown
Member Author

cmd-native-test (cmd-bit-1) is red on main too, with the same two tests.

The shard fails 2/242 on this branch:

  • cmd/bit/serve_http_wbtest.mbt:41 "serve --http: a repository resolves under the root": http_resolve_repo("/srv/git", "/acme/app.git") returns /srv/git/acme/app.git, the test expects the .git suffix stripped (/srv/git/acme/app).
  • cmd/bit/grep_wbtest.mbt:1481 "grep: command path cases stay compatible": git grep --column -n -o reports column 14 for the second match on a line where git reports 13.

The last CI run on main (run 34554739985, the merge of #178) has the identical failure in the same shard (Total tests: 242, passed: 240, failed: 2, same diff), on the old toolchain. Neither test touches code this PR changes, and no fix exists on any branch yet, so I am not widening this PR with one. Happy to open a separate fix if wanted.


Generated by Claude Code

CLAUDE.md is a symlink to AGENTS.md, so the note was appended twice.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018UDsHQQ4RZzjsqix2HYGBv
The locked mooncakes index (2026-08-12) predates bobzhang/toml 0.4.3
and moonbitlang/async 0.22.1, so nix-build failed resolving the new
dependency pins (`attribute '"0.4.3"' missing`). Advance moon-registry
to today's index and moonbit-overlay to its current HEAD, which ships
the v0.10.13+cbb11c36f toolchain the rest of CI is pinned to.

moon-registry was relocked with `nix flake update`. The overlay entry
was written from `nix flake prefetch` of the same revision over git
(the sandbox blocks GitHub tarball downloads); the tree has no export
attributes, so the NAR hash is the one the github fetcher produces.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018UDsHQQ4RZzjsqix2HYGBv
The new moonbit-overlay revision locks nixpkgs a831408e and
treefmt-nix 27b3b12a; mirror them in the nested lock nodes so
`nix build` does not have to rewrite flake.lock in CI.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018UDsHQQ4RZzjsqix2HYGBv

mizchi commented Sep 16, 2026

Copy link
Copy Markdown
Member Author

Update on nix-build: flake.lock is now refreshed on this branch (8930986). moon-registry points at today's mooncakes index and moonbit-overlay at 86fff168 (ships v0.10.13+cbb11c36f), with the overlay's own nixpkgs/treefmt-nix pins mirrored. A full local nix build is still not possible here (GitHub tarball downloads are blocked in the sandbox), so the CI nix-build run on this head is the verification.


Generated by Claude Code

`git grep --column -o` prints one line per match, and each line carries
the column where that match starts. bit added up the *ends* of the
preceding matches instead, which double-counts the gap in front of every
match after the first: on `foo_mmap bar mmap` the second match starts at
column 14 and bit printed 13.

Upstream git had the same bug and fixed it in "grep: fix --column
--only-match for 2nd and later matches" (René Scharfe, 2026-04), shipped
in 2.55 — the release `third_party/git` pins. It computes
`cno = bol - start + match.rm_so + 1`; tracking the offset of the
unscanned remainder and adding the match's own start column is the same
thing.

`grep: command path cases stay compatible` shells out to the system git,
so its expectation had been captured from a pre-fix git, and the shard
has been failing ever since the runners moved to 2.55. It fails the same
way on `main`, unrelated to this branch's dependency bump.

Verified with the native debug binary, with GIT_CONFIG_* unset so the
shim does not delegate to the system git (2.43 here, still pre-fix):
`bit grep --column -n -o -e mmap HEAD` now reports 5 and 14 on
`foo_mmap bar mmap` and 3 and 8 on `abmmapXmmap` — the true columns —
where it printed 13 and 9 before. `moon check --deny-warn
--warn-list=-27-20-82 --target native` passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0134zgvUroAxAAggLS9yufPX
`serve --http: a repository resolves under the root` asserted that
`http_resolve_repo("/srv/git", "/acme/app.git")` strips the suffix and
answers `/srv/git/acme/app`. It never did: the resolver hands the path to
`normalize_repo_root`, which folds a trailing `/.git` — a working copy's
git directory — to the work tree that holds it, and leaves `<name>.git`
alone.

Leaving it alone is the point. A bare repository *is* the directory
`acme/app.git`, so stripping the suffix would make the normal server
layout unreachable; `bit upload-pack`, `receive-pack` and
`upload-archive` read a `.git` suffix the same way, and
docs/scalable-git-server.md records the route as verbatim ("Repository
paths are used verbatim, never percent-decoded").

So the expectation was wrong, not the resolver, and it never passed:
neither function has changed since the test landed in dd09350, whose own
message notes the end-to-end run had not been executed yet. Assert what
the listener implements, and add the `/<repo>/.git` case that
`normalize_repo_root` does fold.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0134zgvUroAxAAggLS9yufPX

mizchi commented Sep 17, 2026

Copy link
Copy Markdown
Member Author

The two cmd-native-test (cmd-bit-1) failures are fixed on this branch (93e8778, 44b919f), so the earlier "left for a separate fix" note above no longer applies. Both were real, both pre-date this branch, and neither was a flake.

grep: command path cases stay compatible — git grep --column -o prints one line per match, each carrying the column where that match starts. bit summed the ends of the preceding matches instead, which double-counts the gap in front of every match after the first: on foo_mmap bar mmap the second match starts at column 14 and bit printed 13. Upstream git had the identical bug and fixed it in "grep: fix --column --only-match for 2nd and later matches" (René Scharfe, 2026-04), shipped in 2.55 — the release third_party/git pins — computing cno = bol - start + match.rm_so + 1. The test shells out to the system git, so its expectation had been captured from a pre-fix git and the shard started failing when the runners moved to 2.55. Both bit and the expectation now report the true column.

Verified locally with the native debug binary, GIT_CONFIG_* unset so the shim does not delegate to the system git (2.43 here, still pre-fix):

                        before        after / git 2.55
foo_mmap bar mmap       5, 13         5, 14
abmmapXmmap             3, 9          3, 8

serve --http: a repository resolves under the root — the expectation asserted that http_resolve_repo strips a .git suffix; it never did. The resolver hands the path to normalize_repo_root, which folds a trailing /.git (a working copy's git directory) to the work tree and leaves <name>.git alone — deliberately, since a bare repository is the directory acme/app.git, bit upload-pack/receive-pack/upload-archive read the suffix the same way, and docs/scalable-git-server.md records the route as verbatim. Neither function has changed since the test landed in dd09350, so this case had never passed. The expectation now matches the listener, plus the /<repo>/.git case that normalize_repo_root does fold.

moon check --deny-warn --warn-list=-27-20-82 --target native passes; CI on this head is the check for the rest.


Generated by Claude Code

@mizchi
mizchi merged commit 520edf2 into main Sep 17, 2026
22 checks passed
mizchi pushed a commit that referenced this pull request Sep 17, 2026
A minor bump: every workspace module, its intra-workspace dependency
pins, `bit_version_tag` and `npm/package.json` move from 0.46.4 to
0.47.0 together, the way the previous releases did.

Since v0.46.4, `main` gained new capability rather than fixes alone,
which is what makes this a minor and not a patch:

- `bit serve --http`, a real listener for the Git smart protocol over
  many repositories (#178), with protocol v2 `ls-refs` on the
  upload-pack side and side-band-64k framing on receive-pack.
- The object-storage layer underneath it: `bit_objstore` with
  compare-and-swap writes, a write-ahead log over it, and an S3
  transport plus response headers and DELETE in `bit_io_native`.
- The toolchain move to MoonBit 0.10.13 / moonbitlang/async 0.22 with
  every third-party dependency at its latest release (#179), which
  changed the npm CLI payload to an ES module.
- A deprecation-free tree: `moon check` now reports no warnings on
  native, js or wasm (#180).

`moon check` passes for native, js and wasm after the bump — the
intra-workspace pins resolve consistently — and the JS build reports
`git version 2.47.0 (bit v0.47.0)`. Publishing is left to the
maintainer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0134zgvUroAxAAggLS9yufPX
@mizchi mizchi mentioned this pull request Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants