Skip to content

fix(runtime): account for ArrayBuffer ownership and external memory - #113

Merged
steipete merged 2 commits into
mainfrom
claude/w143-arraybuffer-accounting
Oct 5, 2026
Merged

steipete merged 2 commits into
mainfrom
claude/w143-arraybuffer-accounting

Conversation

@steipete

@steipete steipete commented Oct 4, 2026 •

Copy link
Copy Markdown

Connect Bun's memory reporting to the allocation and external-storage accounting in the published OpenClaw WebKit engine integrated by #128. Typed-array and ArrayBuffer payloads become visible before GC, JS heap reports exclude backing storage, and worker transfers preserve the allocating VM's charge while moving ownership of the external payload.

Native storage is classified explicitly at the Rust/C++ boundary. node:v8 serializes into unshared external storage while retaining its existing Buffer wire envelope. Worker heap inspection runs on the owning VM thread during JavaScript loops and atomic waits, and pending requests settle during termination. This builds on oven-sh#34406 and oven-sh/WebKit#303; thanks @robobun. Documentation and the fork changelog describe the accounting contract.

Rebased onto #128 (0d23479165e219eec05ebc2ec0d83550571a827c), including the published f1e1ca1156c8cb3b468bec0e1989fbfa08899661 WebKit ARM64 arithmetic fix. The committed production manifest is unchanged by this PR.

Validation

Final candidate: 6facf8734f92f9a2ee0434d286481aca51c595ef. The final follow-up changes only test fixtures; the runtime sources and WebKit manifest are identical to the previously built and qualified de83b783.

  • Full P2 autoreview: scoped-clean.
  • Native macOS ARM64 build and all 12 Rust targets pass, with no skipped targets.
  • macOS accounting: 19/19 in default, interpreter, baseline, DFG and FTL modes; worker inspection 7/7; serializers 60/60; external N-API buffer accounting 1/1.
  • Corrected final fixtures on Linux x64: 19/19 in each of those five modes and 19/19 on Node 24.21.0.
  • Full corrected N-API suite on macOS: 235 pass, one existing todo, zero failures.
  • OpenClaw worker CPU consumer: 6/6 on Bun and 6/6 on Node 24.21.0.
  • Plain macOS selection: 20/21 rows, with exactly the same two socket-cancellation assertions as the unchanged baseline and zero new failures. Raw failures are retained.
  • Exact final-head Mac rebuild and the complete rerun passed, including the full N-API suite, all five accounting modes, plain selection with zero new regressions, and both worker-CPU consumers. Both exact-head fork CI lanes are green in run 37366689402. Darwin passed on attempt 2 after a hosted-runner capacity cancellation with no test steps. The CI merge tree equals the reviewed source tree.

On Darwin, global useJIT=false also disables SharedArrayBuffer. Both pre-change and candidate controls confirm this. Interpreter qualification disables the individual JS/Wasm compilation tiers instead, with effective options and shared-memory availability recorded; no case is skipped.

The first CI run exposed a transfer-fixture reference and two existing N-API fixture issues. The transfer receiver's async message parameter is now cleared immediately after assigning the intended global owner; GC count, stack depth, waits, deadlines and assertions are unchanged. On the same Linux binary, the original fails 17/100 runs; the corrected fixture passes 100/100 normally and 100/100 with concurrent GC disabled. All-tier and Node controls pass afterward.

The N-API conversion fixture now includes <cmath> for std::nextafter. Its text labels use synchronous writes to remain ordered with native printf under stdout backpressure. The output-order difference reproduces equally before this PR; after the fixture correction, pre-change Bun, candidate Bun, Node 24 and Node 26 produce byte-identical output under the same pressure.

The GitHub Actions runner-assignment incident delayed hosted lint jobs as well. Equivalent source lints (200 pass), JavaScript lint and all five TypeScript projects passed locally on the exact final head. Formatting, Clippy, Miri and lol-html tests passed in CI. The explicitly advisory mordant job repeats the unchanged preexisting resolver-style finding; source, baseline and workflow blobs match main.

Fixture lifetime and compilation controls

The original Blob/Response release fixture kept the output in the measuring async-call stack. The same retention occurs before this PR. A scalar-returning allocation helper now finishes that frame before release is measured. No release assertion, tolerance, retry count, or deadline was weakened. The corrected cases pass on Node 24.21.0 and the pre-change Bun, separating this fixture issue from the accounting change. By contrast, the pre-change Bun fails the typed-array external-growth and ArrayBuffer heap-exclusion probes; the corrected runtime and Node pass both.

The busy-worker fixture now isolates its identical loop in a small helper. Reporting functions and one empty worker lifecycle are warmed before the accounting baseline, preventing eager compilation of the reporter or worker-teardown setup from being counted as payload. Controlled replay reproduced the old measurement failures and resolves them with unchanged assertions, including eager FTL.

Heap-snapshot and LLDB evidence for the Blob fixture lifetime

These are matching excerpts from generateHeapSnapshotForDebugging() while the buffer was retained and the LLDB session inspecting the same process. The snapshot has no incoming edge or explicit root for the marked ArrayBuffer. LLDB finds its exact address in the live nativeOutput async-call frame, between the displayed frame pointers.

{
  "snapshot_sha256": "447620831bfe0c25aea4f94068b65eaa99e567cfd15eae272cfa945cd76f28f8",
  "snapshot_type": "GCDebugging",
  "node": {
    "id": 213,
    "class": "ArrayBuffer",
    "address": "0x52623f64b88"
  },
  "incoming_edges": [],
  "roots_entries": [],
  "debugger": {
    "target_address": "0x52623f64b88",
    "stack_address": "0x16fdfdca8",
    "frame_pointer_chain": [
      {
        "fp": "0x16fdfdc00",
        "symbol": "jsc_llint_begin",
        "callee": {
          "id": 1324,
          "class": "Function",
          "label": "nativeOutput"
        }
      },
      {
        "fp": "0x16fdfdd10",
        "symbol": "JSC::asyncFunctionGeneratorBodyCall(JSC::JSGlobalObject*, JSC::VM&, JSC::JSAsyncFunctionGenerator*, JSC::JSValue, JSC::JSGenerator::ResumeMode, JSC::MicrotaskCallCache*)",
        "callee": null
      }
    ]
  }
}

The complete local snapshot/debugger evidence archive has SHA-256 630cedd04abef1fd29f7728d851aeaba9129220f3a3a86976d1e3ea329f8dd2c.

@autofix-troubleshooter

Copy link
Copy Markdown

Hi! I'm the autofix logoautofix.ci troubleshooter bot.

It looks like you correctly set up a CI job that uses the autofix.ci GitHub Action, but the autofix.ci GitHub App has not been installed for this repository. This means that autofix.ci unfortunately does not have the permissions to fix this pull request. If you are the repository owner, please install the app and then restart the CI workflow! 😃

@steipete
steipete force-pushed the claude/w143-arraybuffer-accounting branch from 1e9176c to de83b78 Compare October 5, 2026 19:24
@steipete steipete changed the title fix(memory): account backing stores and inspect busy workers fix(runtime): account for ArrayBuffer ownership and external memory Oct 5, 2026
@steipete
steipete marked this pull request as ready for review October 5, 2026 19:24
@steipete
steipete merged commit 695dda4 into main Oct 5, 2026
12 of 25 checks passed
@steipete
steipete deleted the claude/w143-arraybuffer-accounting branch October 5, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants