fix(runtime): account for ArrayBuffer ownership and external memory

CHANGELOG.md

@@ -207,3 +207,4 @@
 - Build Windows ARM64 alongside x64, require Foundation Authenticode signatures before Windows release packaging, and qualify both architectures with the same native compatibility selection. Unsigned CI artifacts are explicitly test-only.
 
 - Pin immutable [OpenClaw WebKit `f1e1ca1156`](https://github.com/openclaw/WebKit/releases/tag/autobuild-f1e1ca1156c8cb3b468bec0e1989fbfa08899661) with all ten archive checksums, including Windows ARM64, cached repeated stack coordinates, and the ARM64 allocation-accounting arithmetic fix.
+- Account for typed-array and ArrayBuffer payloads before GC, preserve allocation-origin charges across worker transfers, separate external storage from JS heap statistics, and answer worker heap-statistics requests during JavaScript loops and atomic waits. Builds on oven-sh/bun#34406 and oven-sh/WebKit#303; thanks @robobun.

docs/runtime/nodejs-compat.mdx

@@ -228,6 +228,8 @@ Socket close notifications use internal scheduling and continue to run when user
 
 ### [`node:process`](https://nodejs.org/api/process.html)
 
+`process.memoryUsage().arrayBuffers` includes fixed backing storage and typed arrays before GC, including optimized allocations. Transferring a buffer moves its `external` payload charge to the receiving worker while its `arrayBuffers` charge stays with the allocating worker until storage is freed. Resizable buffers contribute their current payload to `external`; shared fixed backing storage contributes only to the allocator's `arrayBuffers`. Shared growable backing storage does not contribute to either field. Unshared WebAssembly memory contributes to `external` at allocation and growth, including before `.buffer` is accessed. External memory also includes native object overhead. Backing storage is excluded from `heapUsed` and `heapTotal`.
+
 `process.platform`, `arch`, `version`, `versions`, `pid`, and `release` expose Node-compatible data descriptors. `argv` and `execArgv` initialize lazily and support descriptor-based replacement. `ppid` and `title` retain their native behavior behind data descriptors. `exitCode` remains a non-configurable accessor.
 
 On POSIX, initializing `process.stdout` or `process.stderr` does not retain a duplicate pipe writer. Explicitly closing the underlying descriptor publishes EOF while the process remains alive. On Windows, the initialized pipe handle retains the writer until process exit, matching Node. On POSIX, socket and pipe stdio use nonblocking writes. `end()` and pipeline completion flush pending output and shut down the write side of socket stdio, so peers receive EOF before process exit. Actual pipes stay open until process exit or explicit descriptor closure. `destroy()` preserves fd 1/2; after POSIX socket or pipe finalization, later stream writes fail with `EPIPE`. On Windows, piped stdout/stderr reset their writable state after `end()` or pipeline completion and accept subsequent writes.
@@ -296,6 +298,8 @@ Valid `vm.Script` cached data is accepted when JSC runs without a JIT. Validatio
 
 `resourceLimits` reports effective heap, nursery, code-range, and stack settings. With the pinned OpenClaw WebKit, managed-heap exhaustion terminates only the affected worker with `ERR_WORKER_OUT_OF_MEMORY` and exit code 1. ArrayBuffer backing stores are excluded. `codeRangeSizeMb` is reporting-only because JSC shares its executable allocator. See [worker resource limits](/runtime/workers#node-worker-resource-limits) for defaults and limitations.
 
+`worker.getHeapStatistics()` takes an owning-thread snapshot while a worker is idle, executing JavaScript, or blocked in `Atomics.wait()`. The snapshot includes freshly allocated external storage. Inspection preserves the atomic wait and pending requests settle if the worker exits.
+
 File workers select `--require` and `--import` preloads from their environment independently of `execArgv`. Explicit `env` or `execArgv` options re-read the worker's `NODE_OPTIONS` and `BUN_OPTIONS` before applying the selected CLI arguments. `execArgv: []` removes CLI preloads while environment preloads still apply. Require preloads run before import preloads.
 
 `MessagePort.emit()` and `parentPort.emit()` preserve the emitted value for Node listeners. Web listeners receive `MessageEvent` objects for `message` and `messageerror`, and `CustomEvent` objects for other events.

src/js/node/v8.ts

@@ -10,6 +10,8 @@ const {
   stopGCProfiler,
   discardGCProfiler,
   getHeapUsage,
+  serialize: serializeNative,
+  bufferEnvelopeMagic: kBufferEnvelopeMagic,
   queryObjects: queryHeapObjects,
 } = $cpp("NodeV8.cpp", "Bun::createNodeV8Binding");
 
@@ -41,7 +43,6 @@ const FunctionPrototypeCall = Function.prototype.call;
 const uncurryThis = func => FunctionPrototypeCall.bind(func);
 const ObjectGetOwnPropertyDescriptor = Object.getOwnPropertyDescriptor;
 const Uint8ArrayCtor = Uint8Array;
-const BufferAllocUnsafe = Buffer.allocUnsafe;
 const TypedArrayProto = Object.getPrototypeOf(Uint8ArrayCtor.prototype);
 const TypedArrayPrototypeGetBuffer = uncurryThis(ObjectGetOwnPropertyDescriptor(TypedArrayProto, "buffer")!.get);
 const TypedArrayPrototypeGetByteOffset = uncurryThis(
@@ -50,7 +51,6 @@ const TypedArrayPrototypeGetByteOffset = uncurryThis(
 const TypedArrayPrototypeGetByteLength = uncurryThis(
   ObjectGetOwnPropertyDescriptor(TypedArrayProto, "byteLength")!.get,
 );
-const TypedArrayPrototypeSet = uncurryThis(TypedArrayProto.set);
 const DataViewPrototypeGetBuffer = uncurryThis(ObjectGetOwnPropertyDescriptor(DataView.prototype, "buffer")!.get);
 const DataViewPrototypeGetByteOffset = uncurryThis(
   ObjectGetOwnPropertyDescriptor(DataView.prototype, "byteOffset")!.get,
@@ -299,7 +299,6 @@ function getCppHeapStatistics(type = "detailed") {
 // Buffer-bearing payloads are framed as MAGIC + version + SSV([value, buffers]) so deserialize
 // can restore Buffer prototypes (see internal/serialization_buffers). Leading 0xFF cannot collide
 // with bare SSV output; Buffer-free payloads stay bare SSV so older readers keep working.
-const kBufferEnvelopeMagic = [0xff, 0x42, 0x55, 0x4e, 0x01]; // 0xFF "BUN" v1
 
 function hasBufferEnvelopeMagic(view) {
   // In-bounds integer-indexed reads on a typed array never consult the
@@ -347,14 +346,7 @@ function stopCoverage() {
 }
 function serialize(arg1) {
   const tagged = require("internal/serialization_buffers").tagBuffers(arg1);
-  if (tagged === null) {
-    return jsc.serialize(arg1, { binaryType: "nodebuffer" });
-  }
-  const payload = jsc.serialize(tagged, { binaryType: "nodebuffer" });
-  const framed = BufferAllocUnsafe(kBufferEnvelopeMagic.length + TypedArrayPrototypeGetByteLength(payload));
-  for (let i = 0; i < kBufferEnvelopeMagic.length; i++) framed[i] = kBufferEnvelopeMagic[i];
-  TypedArrayPrototypeSet(framed, payload, kBufferEnvelopeMagic.length);
-  return framed;
+  return serializeNative(tagged === null ? arg1 : tagged, tagged !== null);
 }
 
 function getDefaultHeapSnapshotPath() {

src/jsc/JSValue.rs

@@ -592,6 +592,7 @@ impl JSValue {
                 } else {
                     Some(MarkedArrayBuffer_deallocator)
                 },
+                crate::array_buffer::ArrayBufferAllocationMode::RuntimeOwned,
             )
         })
     }
@@ -619,6 +620,7 @@ impl JSValue {
                 } else {
                     Some(MarkedArrayBuffer_deallocator)
                 },
+                crate::array_buffer::ArrayBufferAllocationMode::RuntimeOwned,
             )
         })
     }
@@ -646,6 +648,7 @@ impl JSValue {
                 len,
                 ctx,
                 Some(free),
+                crate::array_buffer::ArrayBufferAllocationMode::RuntimeOwned,
             )
         })
     }
@@ -2005,6 +2008,7 @@ unsafe extern "C" {
         len: usize,
         ctx: *mut c_void,
         deallocator: Option<unsafe extern "C" fn(*mut c_void, *mut c_void)>,
+        allocation: crate::array_buffer::ArrayBufferAllocationMode,
     ) -> JSValue;
     safe fn JSBuffer__bufferFromLength(global: &JSGlobalObject, len: i64) -> JSValue;
     safe fn JSC__JSValue__dateInstanceFromNumber(global: &JSGlobalObject, n: f64) -> JSValue;

src/jsc/array_buffer.rs

@@ -13,6 +13,14 @@ bun_core::declare_scope!(ArrayBuffer, visible);
 /// when a zero-copy ArrayBuffer/typed array backing store is collected.
 pub type JSTypedArrayBytesDeallocator = Option<unsafe extern "C" fn(*mut c_void, *mut c_void)>;
 
+/// Matches JSC::ArrayBufferAllocationMode. Foreign storage does not acquire an allocator-origin charge.
+#[repr(u8)]
+#[derive(Clone, Copy)]
+pub enum ArrayBufferAllocationMode {
+    External = 0,
+    RuntimeOwned = 1,
+}
+
 // ──────────────────────────────────────────────────────────────────────────
 // ArrayBuffer
 // ──────────────────────────────────────────────────────────────────────────
@@ -115,6 +123,7 @@ unsafe extern "C" {
         len: usize,
         dealloc: JSTypedArrayBytesDeallocator,
         ctx: *mut c_void,
+        allocation: ArrayBufferAllocationMode,
     ) -> JSValue;
     fn Bun__makeTypedArrayWithBytesNoCopy(
         global: &JSGlobalObject,
@@ -123,6 +132,7 @@ unsafe extern "C" {
         len: usize,
         dealloc: JSTypedArrayBytesDeallocator,
         ctx: *mut c_void,
+        allocation: ArrayBufferAllocationMode,
     ) -> JSValue;
     fn Bun__createTypedArrayForCopy(
         global: *const JSGlobalObject,
@@ -459,6 +469,7 @@ impl ArrayBuffer {
                     // The deallocator ignores its ctx (mi_free needs no ctx). Any non-null
                     // sentinel would do; pass the data ptr itself.
                     self.ptr.cast(),
+                    ArrayBufferAllocationMode::RuntimeOwned,
                 )
             };
         }
@@ -472,6 +483,7 @@ impl ArrayBuffer {
                 self.byte_len,
                 Some(MarkedArrayBuffer_deallocator),
                 self.ptr.cast(),
+                ArrayBufferAllocationMode::RuntimeOwned,
             )
         }
     }
@@ -504,6 +516,7 @@ impl ArrayBuffer {
                         self.byte_len,
                         None,
                         ptr::null_mut(),
+                        ArrayBufferAllocationMode::External,
                     )
                 };
             }
@@ -517,6 +530,7 @@ impl ArrayBuffer {
                     self.byte_len,
                     None,
                     ptr::null_mut(),
+                    ArrayBufferAllocationMode::External,
                 )
             };
         }
@@ -542,6 +556,7 @@ impl ArrayBuffer {
         ctx: &JSGlobalObject,
         deallocator: *mut c_void,
         callback: JSTypedArrayBytesDeallocator,
+        allocation: ArrayBufferAllocationMode,
     ) -> JsResult<JSValue> {
         if !self.value.is_empty() {
             return Ok(self.value);
@@ -557,6 +572,7 @@ impl ArrayBuffer {
                     self.byte_len,
                     callback,
                     deallocator,
+                    allocation,
                 )
             };
         }
@@ -570,6 +586,7 @@ impl ArrayBuffer {
                 self.byte_len,
                 callback,
                 deallocator,
+                allocation,
             )
         }
     }
@@ -1017,12 +1034,20 @@ pub(crate) unsafe fn make_array_buffer_with_bytes_no_copy(
     len: usize,
     deallocator: JSTypedArrayBytesDeallocator,
     deallocator_context: *mut c_void,
+    allocation: ArrayBufferAllocationMode,
 ) -> JsResult<JSValue> {
     crate::host_fn::from_js_host_call(global, || {
         // SAFETY: forwarded verbatim; the caller upholds this function's
         // contract (`ptr` valid for `len` bytes until `deallocator` runs).
         unsafe {
-            Bun__makeArrayBufferWithBytesNoCopy(global, ptr, len, deallocator, deallocator_context)
+            Bun__makeArrayBufferWithBytesNoCopy(
+                global,
+                ptr,
+                len,
+                deallocator,
+                deallocator_context,
+                allocation,
+            )
         }
     })
 }
@@ -1042,6 +1067,7 @@ pub unsafe fn make_typed_array_with_bytes_no_copy(
     len: usize,
     deallocator: JSTypedArrayBytesDeallocator,
     deallocator_context: *mut c_void,
+    allocation: ArrayBufferAllocationMode,
 ) -> JsResult<JSValue> {
     crate::host_fn::from_js_host_call(global, || {
         // SAFETY: forwarded verbatim; the caller upholds this function's
@@ -1054,6 +1080,7 @@ pub unsafe fn make_typed_array_with_bytes_no_copy(
                 len,
                 deallocator,
                 deallocator_context,
+                allocation,
             )
         }
     })

src/jsc/bindings/BunClientData.cpp

@@ -63,7 +63,6 @@ JSVMClientData::JSVMClientData(VM& vm, RefPtr<JSC::SourceProvider> sourceProvide
     , CLIENT_ISO_SUBSPACE_INIT(m_domConstructorSpace)
     , CLIENT_ISO_SUBSPACE_INIT(m_domNamespaceObjectSpace)
     , m_clientSubspaces(makeUnique<ExtendedDOMClientIsoSubspaces>())
-    , m_heapUsage(vm.heap)
 {
 }
 

src/jsc/bindings/BunClientData.h

@@ -76,55 +76,13 @@ class DOMWrapperWorld;
 #include "NodeVMOptionNames.h"
 #include "NodeVMSourceOriginCache.h"
 #include "NodeVMCompilationCache.h"
-#include <JavaScriptCore/HeapObserver.h>
 namespace Zig {
 class GlobalObject;
 }
 
 namespace Bun {
 class StrongRootBlock;
 
-// Add allocations since the last collection to its survivor size. Heap::size()
-// counts mark bits and misses newly allocated cells and array backing storage.
-class HeapUsage final : public JSC::HeapObserver {
-    WTF_MAKE_NONCOPYABLE(HeapUsage);
-
-public:
-    explicit HeapUsage(JSC::Heap& heap)
-        : m_heap(heap)
-    {
-        m_heap.addObserver(this);
-    }
-
-    ~HeapUsage() final
-    {
-        m_heap.removeObserver(this);
-    }
-
-    size_t get() const
-    {
-        return m_sizeAfterLastCollection + (m_heap.totalBytesAllocated() - m_allocatedAtLastCollection);
-    }
-
-private:
-    void willGarbageCollect() final {}
-
-    // Heap::didFinishCollection() notifies observers after updateAllocationLimits()
-    // stored this collection's size, in the end phase of the collection, while
-    // the mutator is stopped. The mutator reads m_sizeAfterLastCollection once
-    // it resumes, the same way it reads JSC's own counters.
-    void didGarbageCollect(JSC::CollectionScope scope) final
-    {
-        m_sizeAfterLastCollection = scope == JSC::CollectionScope::Full
-            ? m_heap.sizeAfterLastFullCollection()
-            : m_heap.sizeAfterLastEdenCollection();
-        m_allocatedAtLastCollection = m_heap.totalBytesAllocated();
-    }
-
-    JSC::Heap& m_heap;
-    size_t m_sizeAfterLastCollection { 0 };
-    uint64_t m_allocatedAtLastCollection { 0 };
-};
 }
 
 namespace JSC {
@@ -249,9 +207,6 @@ class JSVMClientData : public JSC::VM::ClientData {
     Bun::NodeVMOptionNames& nodeVMOptionNames() { return m_nodeVMOptionNames; }
     Bun::NodeVMSourceOriginCache& nodeVMSourceOriginCache() { return m_nodeVMSourceOriginCache; }
 
-    // Survivor size plus allocations since the last collection; mutator thread only.
-    size_t heapUsage() const { return m_heapUsage.get(); }
-
     // The VM's default (first) Zig::GlobalObject: what defaultGlobalObject(JSC::VM&) returns on threads whose thread-local
     // default is not this VM's, e.g. the collector thread running a collection's end phase. gcProtect'ed for the VM's life.
     JSC::JSGlobalObject* defaultGlobalObject { nullptr };
@@ -351,8 +306,6 @@ class JSVMClientData : public JSC::VM::ClientData {
     Bun::NodeVMCompilationCache nodeVMCompilationCache;
 
 private:
-    Bun::HeapUsage m_heapUsage;
-
     SentinelLinkedList<JSVMClientDataClient, BasicRawSentinelNode<JSVMClientDataClient>> m_clients;
     bool m_isWorkerVM { false };
     bool m_isNodeWorkerVM { false };

src/jsc/bindings/BunProcess.cpp

@@ -4198,26 +4198,14 @@ JSC_DEFINE_HOST_FUNCTION(Process_functionMemoryUsage, (JSC::JSGlobalObject * glo
     //    arrayBuffers: 9386
     // }
 
-    size_t heapUsed = WebCore::clientData(vm)->heapUsage();
-    size_t heapTotal = std::max(vm.heap.capacity(), heapUsed);
+    size_t heapUsed = vm.heap.jsHeapSizeForReporting();
+    size_t heapTotal = vm.heap.jsHeapCapacityForReporting();
     result->putDirectOffset(vm, 0, JSC::jsNumber(current_rss));
     result->putDirectOffset(vm, 1, JSC::jsNumber(heapTotal));
 
     result->putDirectOffset(vm, 2, JSC::jsNumber(heapUsed));
 
-    result->putDirectOffset(vm, 3, JSC::jsNumber(vm.heap.extraMemorySize() + vm.heap.externalMemorySize()));
-
-    // JSC won't count this number until vm.heap.addReference() is called.
-    // That will only happen in cases like:
-    // - new ArrayBuffer()
-    // - new Uint8Array(42).buffer
-    // - fs.readFile(path, "utf-8") (sometimes)
-    // - ...
-    //
-    // But it won't happen in cases like:
-    // - new Uint8Array(42)
-    // - Buffer.alloc(42)
-    // - new Uint8Array(42).slice()
+    result->putDirectOffset(vm, 3, JSC::jsNumber(vm.heap.externalMemorySizeForReporting()));
     result->putDirectOffset(vm, 4, JSC::jsNumber(vm.heap.arrayBufferSize()));
 
     RELEASE_AND_RETURN(throwScope, JSC::JSValue::encode(result));

src/jsc/bindings/JSBuffer.cpp

@@ -414,7 +414,7 @@ bool Bun::rejectBytesNoCopyAboveArrayBufferLimit(JSC::JSGlobalObject* globalObje
     return true;
 }
 
-JSC::EncodedJSValue JSBuffer__bufferFromPointerAndLengthAndDeinit(JSC::JSGlobalObject* lexicalGlobalObject, char* ptr, size_t length, void* ctx, JSTypedArrayBytesDeallocator bytesDeallocator)
+JSC::EncodedJSValue JSBuffer__bufferFromPointerAndLengthAndDeinit(JSC::JSGlobalObject* lexicalGlobalObject, char* ptr, size_t length, void* ctx, JSTypedArrayBytesDeallocator bytesDeallocator, JSC::ArrayBufferAllocationMode allocationMode)
 {
     JSC::JSUint8Array* uint8Array = nullptr;
 
@@ -429,7 +429,8 @@ JSC::EncodedJSValue JSBuffer__bufferFromPointerAndLengthAndDeinit(JSC::JSGlobalO
 
         auto buffer = ArrayBuffer::createFromBytes({ reinterpret_cast<const uint8_t*>(ptr), length }, createSharedTask<void(void*)>([=](void* p) {
             bytesDeallocator(p, ctx);
-        }));
+        }),
+            allocationMode);
 
         uint8Array = JSC::JSUint8Array::create(lexicalGlobalObject, subclassStructure, WTF::move(buffer), 0, length);
     } else {
@@ -2641,7 +2642,8 @@ extern "C" JSC::EncodedJSValue JSBuffer__fromMmap(Zig::GlobalObject* globalObjec
 
     auto buffer = ArrayBuffer::createFromBytes({ static_cast<const uint8_t*>(ptr), length }, createSharedTask<void(void*)>([lengthAsContext](void* p) {
         unmapBufferBytes(p, lengthAsContext);
-    }));
+    }),
+        JSC::ArrayBufferAllocationMode::RuntimeOwned);
 
     auto* view = JSC::JSUint8Array::create(globalObject, structure, WTF::move(buffer), 0, length);
     RETURN_IF_EXCEPTION(scope, {});

src/jsc/bindings/JSBuffer.h

@@ -23,6 +23,7 @@
 #include "root.h"
 
 #include <JavaScriptCore/InternalFunction.h>
+#include <JavaScriptCore/ArrayBuffer.h>
 #include <JavaScriptCore/JSGlobalObject.h>
 #include <JavaScriptCore/ThrowScope.h>
 #include <wtf/NeverDestroyed.h>
@@ -31,7 +32,7 @@
 #include "headers-handwritten.h"
 
 extern "C" JSC::EncodedJSValue JSBuffer__bufferFromLength(JSC::JSGlobalObject* lexicalGlobalObject, int64_t length);
-extern "C" JSC::EncodedJSValue JSBuffer__bufferFromPointerAndLengthAndDeinit(JSC::JSGlobalObject* lexicalGlobalObject, char* ptr, size_t length, void* ctx, JSTypedArrayBytesDeallocator bytesDeallocator);
+extern "C" JSC::EncodedJSValue JSBuffer__bufferFromPointerAndLengthAndDeinit(JSC::JSGlobalObject* lexicalGlobalObject, char* ptr, size_t length, void* ctx, JSTypedArrayBytesDeallocator bytesDeallocator, JSC::ArrayBufferAllocationMode);
 extern "C" JSC::EncodedJSValue Bun__encoding__toString(const uint8_t* input, size_t len, JSC::JSGlobalObject* globalObject, Encoding encoding);
 extern "C" JSC::EncodedJSValue Bun__encoding__toStringUTF8(const uint8_t* input, size_t len, JSC::JSGlobalObject* globalObject);
 extern "C" bool Bun__Buffer_fill(EncodedSlice*, void*, size_t, WebCore::BufferEncodingType);