diff --git a/CHANGELOG.md b/CHANGELOG.md index c9a8d15d..8032a541 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -66,6 +66,32 @@ All notable changes to EigenScript are documented here. stated honestly in the gate header: its compile lines name sources via `$SOURCES` variables the recognizer cannot tell from a link line. +- **The sandbox memory budget now covers the zlib codecs (#292 + follow-up).** `inflate`/`deflate` (and their `zlib_*` duals) are on + the `sandbox_run` allowlist but charged nothing against `max_bytes`, + so an allowlisted call allocated straight past the cap that exists to + protect the host from untrusted code: a 9,732-byte compressed blob + decompressed to a 10M-element list (~800 MB of `Value`s) inside a + sandbox capped at 8 MiB — measured 853 MiB peak RSS, against 13 MiB + for the charged `zeros` control under the same budget. The `{ok:0}` + arrived only after the memory was already taken. Every other + allocator makes the caller *name* a size, which is what the charge + reads; a compressed blob names nothing and amplifies ~1000x, so the + "bounded by loop-iteration cap × per-op size" reasoning that covers + the remaining uncharged allocators does not hold for these. Both the + codec's own output buffer (charged as it grows) and the list built + from it are now charged. Behavior outside a budgeted `sandbox_run` is + unchanged — `sandbox_charge` is a no-op there. + +- **`sandbox_run` no longer blames a callable for a result it merely + could not scan.** The boundary scan is node/depth budgeted and fails + closed — correct — but every exhaustion was reported as `sandbox + result contains a callable`, so a large list of plain numbers was + attributed to a function that was never in the result. Budget + exhaustion now reports that it could not scan the result; the literal + claim is reserved for an actual sighting. Still refuses in both + cases. + - **Widget drawing is contained (#823).** `render` now wraps every widget's draw in a clip of its own rect intersected with its ancestors' — `canvas` `on_paint` included, so a paint callback can no diff --git a/docs/BUILTINS.md b/docs/BUILTINS.md index 30bb2f7e..d3aa8d3d 100644 --- a/docs/BUILTINS.md +++ b/docs/BUILTINS.md @@ -178,7 +178,7 @@ Compact typed arrays of doubles with O(1) indexed access. Iterable with |------|-----------|-------------| | `vm_run_bytecode` | `vm_run_bytecode of ` | Assemble a chunk from a descriptor and run it on the C VM, returning the result. Descriptor: `[abi, code, constants, functions?, param_count?, name?, local_names?]` — `abi` is the **bytecode ABI revision** the producer was built against (currently `1`; see below); `code` is a list of byte ints (opcodes + little-endian operands, 16-bit except `OP_LINE`'s, which is 32-bit since #630); `constants` is the pool; `functions` is a list of nested descriptors referenced by `OP_CLOSURE` (nested descriptors carry **no** `abi` element); `local_names` (slot order) sizes the call frame and names parameters. The minimal `[abi, code, constants]` form is a flat module chunk. The bridge for an EigenScript-written compiler: emit bytecode as data, execute it on the same VM (and JIT) the C compiler's output uses. Caller supplies a well-formed chunk ending in `OP_RETURN`. **A descriptor whose `abi` is missing or does not match the runtime raises (kind `value`) rather than executing** — #704: opcode numbers *and* operand widths are the bytecode ABI, and before the stamp a producer built against an older revision ran misaligned garbage at exit 0 with no error. Producers must hardcode the revision as a literal; a producer that reads the runtime's current value back would always agree and the check would be decoration. A non-list descriptor also raises (it silently returned `null` before #704). | | `record_history` | `record_history of flag` | Enable (nonzero) / disable (0) per-assignment history recording that `prev of x` and ` is x at ` temporal queries read (sets both value- and observer-state history). The C compiler auto-enables it when compiling a temporal query; a self-hosted compiler calls this. The flag must be a number — a non-numeric flag raises (it is not silently treated as disable). Returns the previous setting. | -| `sandbox_run` | `sandbox_run of [descriptor, max_iterations?, max_bytes?]` | Run a chunk (same ABI-stamped descriptor as `vm_run_bytecode`) under safety bounds. Does not throw: an ABI-revision mismatch comes back as `{ok: 0, error: {kind, message, line}}` with a message naming the revision, distinct from a malformed chunk's `invalid chunk descriptor`, so a grading ladder can tell "your producer is stale" from "your bytecode is wrong" without re-running. **Fail-closed**: only a pure-compute *allowlist* (math, bit, list/dict/string ops, buffers, json, regex, observer reads, parse/tokenize, `print`/`assert`) is visible — every other builtin (file/process/network/db, code-exec, threads, channels, terminal, `exit`, global-state mutators like `set_observer_thresholds`, and the whole extension surface) is shadowed by a blocked stub, so a new builtin is denied by default. The sandbox env is a **sealed root**, not a child of the host global env: the allowlist is copied in, so an outward assignment (`x is v`, `OP_SET_NAME`) has no outer binding to write through to and a name the host defines later is not reachable. `import` is gated at the opcode — it is not a name, so the allowlist never covered it. A **callable cannot cross the boundary**: a `fn` in the result closes over the sandbox env and would run in the host after the caps are restored, so it comes back as `{ok: 0, error: {kind: "sandbox", ...}}` instead. Loops are capped at `max_iterations` (default 1e6), so neither a runaway loop nor a single blocking call can hang the host. **Memory is capped at `max_bytes` (default 256 MiB)**: the size-controlled allocators (`zeros`/`fill`/`buffer`/`range`) charge a per-run budget, so a single huge allocation *or* an aggregate across a loop raises a caught error (→ `{ok:0}`) instead of an uncatchable out-of-memory `abort()`. Runtime errors are caught. Returns `{ok: 1/0, result: value}`. For validating untrusted/generated code. | +| `sandbox_run` | `sandbox_run of [descriptor, max_iterations?, max_bytes?]` | Run a chunk (same ABI-stamped descriptor as `vm_run_bytecode`) under safety bounds. Does not throw: an ABI-revision mismatch comes back as `{ok: 0, error: {kind, message, line}}` with a message naming the revision, distinct from a malformed chunk's `invalid chunk descriptor`, so a grading ladder can tell "your producer is stale" from "your bytecode is wrong" without re-running. **Fail-closed**: only a pure-compute *allowlist* (math, bit, list/dict/string ops, buffers, json, regex, observer reads, parse/tokenize, `print`/`assert`) is visible — every other builtin (file/process/network/db, code-exec, threads, channels, terminal, `exit`, global-state mutators like `set_observer_thresholds`, and the whole extension surface) is shadowed by a blocked stub, so a new builtin is denied by default. The sandbox env is a **sealed root**, not a child of the host global env: the allowlist is copied in, so an outward assignment (`x is v`, `OP_SET_NAME`) has no outer binding to write through to and a name the host defines later is not reachable. `import` is gated at the opcode — it is not a name, so the allowlist never covered it. A **callable cannot cross the boundary**: a `fn` in the result closes over the sandbox env and would run in the host after the caps are restored, so it comes back as `{ok: 0, error: {kind: "sandbox", ...}}` instead. That scan is node/depth budgeted and fails closed, so a result too large to scan is also refused — with a message saying so, distinct from the one naming an actual callable. Loops are capped at `max_iterations` (default 1e6), so neither a runaway loop nor a single blocking call can hang the host. **Memory is capped at `max_bytes` (default 256 MiB)**: the size-controlled allocators (`zeros`/`fill`/`buffer`/`range`/`concat`) and the zlib codecs (`inflate`/`deflate` and their `zlib_*` duals — both the codec buffer and the list of values built from it) charge a per-run budget, so a single huge allocation *or* an aggregate across a loop raises a caught error (→ `{ok:0}`) instead of an uncatchable out-of-memory `abort()`. The codecs matter here because every other allocator makes the caller *name* a size, which is what the charge reads, while a compressed blob names nothing and amplifies ~1000x. Runtime errors are caught. Returns `{ok: 1/0, result: value}`. For validating untrusted/generated code. | ### Bytes ↔ values diff --git a/examples/ui_canvas_events.eigs b/examples/ui_canvas_events.eigs index 2e0a1dc2..dedd715d 100644 --- a/examples/ui_canvas_events.eigs +++ b/examples/ui_canvas_events.eigs @@ -202,6 +202,5 @@ define on_key(ev) as: define on_tick(r) as: sb.text is status -ui._layout of [root, 0, 0] ui.app_loop of [root, on_key, on_tick] gfx_close of null diff --git a/src/builtins.c b/src/builtins.c index aeb83bed..2269da87 100644 --- a/src/builtins.c +++ b/src/builtins.c @@ -2884,17 +2884,30 @@ static int sandbox_name_allowed(const char *name) { * the fail-closed direction: an unwalkable result is refused, not trusted. */ #define SANDBOX_RESULT_MAX_DEPTH 32 #define SANDBOX_RESULT_MAX_NODES 100000 -static int sandbox_value_has_callable(Value *v, int depth, long *budget) { +/* Returns 1 if the result contains — or cannot be shown NOT to contain — a + * callable. Fails closed on an exhausted node/depth budget, but sets + * `*unverified` when that is why, because the two are different facts and the + * caller reports them differently: a 10M-element list of numbers is not "a + * callable crossing the boundary", it just outruns the scan. Blaming a + * callable there sends the caller hunting for a function that was never in + * the result. */ +static int sandbox_value_has_callable(Value *v, int depth, long *budget, + int *unverified) { if (!v) return 0; if (v->type == VAL_FN || v->type == VAL_BUILTIN) return 1; - if (depth > SANDBOX_RESULT_MAX_DEPTH || --(*budget) <= 0) return 1; + if (depth > SANDBOX_RESULT_MAX_DEPTH || --(*budget) <= 0) { + *unverified = 1; + return 1; + } if (v->type == VAL_LIST) { for (int i = 0; i < v->data.list.count; i++) - if (sandbox_value_has_callable(v->data.list.items[i], depth + 1, budget)) + if (sandbox_value_has_callable(v->data.list.items[i], depth + 1, + budget, unverified)) return 1; } else if (v->type == VAL_DICT) { for (int i = 0; i < v->data.dict.count; i++) - if (sandbox_value_has_callable(v->data.dict.vals[i], depth + 1, budget)) + if (sandbox_value_has_callable(v->data.dict.vals[i], depth + 1, + budget, unverified)) return 1; } return 0; @@ -3029,15 +3042,21 @@ Value* builtin_sandbox_run(Value *arg) { * Drop the result and report it as a sandbox denial — same {kind, message, * line} shape as any other refusal, so a grading ladder sees it. */ long scan_budget = SANDBOX_RESULT_MAX_NODES; - if (ok && result && sandbox_value_has_callable(result, 0, &scan_budget)) { + int scan_unverified = 0; + if (ok && result && + sandbox_value_has_callable(result, 0, &scan_budget, &scan_unverified)) { val_decref(result); result = NULL; ok = 0; Value *ev = make_dict(3); dict_set_owned(ev, "kind", make_str(err_kind_name(EK_SANDBOX))); dict_set_owned(ev, "message", - make_str("sandbox result contains a callable; " - "functions cannot cross the sandbox boundary")); + make_str(scan_unverified + ? "sandbox result too large to scan for " + "callables (exceeds the result node/depth " + "budget); refusing to cross the boundary" + : "sandbox result contains a callable; " + "functions cannot cross the sandbox boundary")); dict_set_owned(ev, "line", make_num(0)); dict_set_owned(out, "error", ev); } @@ -4821,6 +4840,15 @@ static int zlib_bytes_arg(Value *arg, const char *who, /* Wrap a finished byte buffer as the list-of-ints result value (the * read_bytes shape). Takes ownership of nothing; caller still frees. */ static Value *zlib_bytes_result(const unsigned char *buf, unsigned long n) { + /* #292: the result is `n` fresh number Values at sizeof(Value)+sizeof(Value*) + * each — ~80 bytes per decompressed BYTE. Charging only the codec's own + * output buffer would therefore miss 98% of the cost, so charge the list + * here too, with the same accounting range/zeros use. Without this an + * allowlisted `inflate` allocates straight past max_bytes: the budget + * bounds allocators the caller has to *name* a size for, and a compressed + * blob names nothing. */ + if (!sandbox_charge((size_t)n * (sizeof(Value) + sizeof(Value *)))) + return make_null(); Value *result = make_list((int)n); for (unsigned long i = 0; i < n; i++) list_append_owned(result, make_num((double)buf[i])); @@ -4845,6 +4873,15 @@ static Value *zlib_inflate_impl(const char *who, int window_bits, Value *arg) { } size_t cap = src_n * 3 + 64; if (cap > EIGS_INFLATE_MAX_OUT) cap = EIGS_INFLATE_MAX_OUT; + /* #292: charge the codec's own buffer as it grows, so a bomb is refused + * before the memory is touched rather than after. EIGS_INFLATE_MAX_OUT + * bounds this at 256 MiB, which is the *default* whole-run budget — a + * caller that lowered max_bytes must not be overrun by one call. */ + if (!sandbox_charge(cap)) { + inflateEnd(&zs); + free(src); + return make_null(); + } unsigned char *out = xmalloc(cap); int zrc = Z_OK; for (;;) { @@ -4858,6 +4895,12 @@ static Value *zlib_inflate_impl(const char *who, int window_bits, Value *arg) { if (cap >= EIGS_INFLATE_MAX_OUT) break; /* limit raise below */ size_t ncap = cap * 2; if (ncap > EIGS_INFLATE_MAX_OUT) ncap = EIGS_INFLATE_MAX_OUT; + if (!sandbox_charge(ncap - cap)) { /* #292: charge the delta */ + inflateEnd(&zs); + free(out); + free(src); + return make_null(); + } out = xrealloc(out, ncap); cap = ncap; } @@ -4915,6 +4958,14 @@ static Value *zlib_deflate_impl(const char *who, int window_bits, Value *arg) { return make_null(); } uLong bound = deflateBound(&zs, (uLong)src_n); + /* #292: deflate amplifies far less than inflate (bound ~= src_n), but the + * budget should account for every codec buffer, not just the dangerous + * one — an uncharged allocator is a gap whether or not it is exploitable. */ + if (!sandbox_charge((size_t)bound)) { + deflateEnd(&zs); + free(src); + return make_null(); + } unsigned char *out = xmalloc(bound > 0 ? bound : 1); size_t pos = 0; int zrc = Z_OK; diff --git a/src/vm.c b/src/vm.c index 93c3edd8..a17f2765 100644 --- a/src/vm.c +++ b/src/vm.c @@ -865,14 +865,26 @@ volatile int *g_vm_abort_flag = &g_vm_abort_never; * the allocation reach abort(). Cumulative over the run (snippets are short, so * cumulative ~= peak); generous default, overridable via sandbox_run's max_bytes * arg. No-op outside a budgeted sandbox (active=0), so normal runs pay nothing. - * Plain process globals: sandbox_run is synchronous and single-threaded. + * The three counters are per-THREAD (EigsThread fields reached through the + * g_sandbox_* macros), not the plain process globals this note used to claim: + * two threads — or two EigsStates — running sandbox_run concurrently keep + * separate budgets, and nothing here needs a lock. The save/restore around the + * run is what makes NESTING compose, not what makes it thread-safe. * * Charged at: zeros, fill, buffer, range, concat — the size-controlled list/ - * tensor allocators (the issue's vectors). NOT yet charged: zeros_like (mirrors - * an already-charged input), matmul output (per-call capped at 10M elems; its - * inputs are charged), and the text_builder growth path — all bounded by the - * loop-iteration cap × per-op size, not the byte budget. Extending the budget to - * them is a one-line sandbox_charge() at each; left out to match #292's scope. */ + * tensor allocators (the issue's vectors) — plus the zlib codecs (inflate/ + * deflate and their zlib_* duals): both the codec's own output buffer and the + * list of Values built from it. The codecs are the case that breaks the + * "bounded by loop-iteration cap × per-op size" reasoning the rest of this + * note relies on: every other allocator makes the caller NAME a size, which is + * what the charge reads, while a compressed blob names nothing and amplifies + * ~1000x. Uncharged, a 9 KB input decompressed to ~800 MB of Values inside an + * 8 MiB sandbox (measured 853 MiB peak RSS vs 13 MiB for the charged `zeros` + * control). NOT yet charged: zeros_like (mirrors an already-charged input), + * matmul output (per-call capped at 10M elems; its inputs are charged), and the + * text_builder growth path — those genuinely are bounded by the loop-iteration + * cap × per-op size. Extending the budget to them is a one-line + * sandbox_charge() at each; left out to match #292's scope. */ int sandbox_charge(size_t bytes) { if (!g_sandbox_active || g_sandbox_byte_max == 0) return 1; /* Overflow-safe: g_sandbox_bytes_used <= g_sandbox_byte_max is an invariant diff --git a/tests/run_all_tests.sh b/tests/run_all_tests.sh index 336c16b7..e67a4e15 100755 --- a/tests/run_all_tests.sh +++ b/tests/run_all_tests.sh @@ -2688,15 +2688,15 @@ ZLIB_PROBE_OUT=$(./eigenscript "$ZLIB_PROBE_FILE" 2>&1) rm -f "$ZLIB_PROBE_FILE" if ! echo "$ZLIB_PROBE_OUT" | grep -q "compiled without zlib support"; then - echo "[124] DEFLATE Codecs (#684, 22 checks)" + echo "[124] DEFLATE Codecs (#684, 24 checks)" INF_OUTPUT=$(./eigenscript ../tests/test_inflate.eigs 2>&1); INF_OUTPUT_RC=$? if rc_ok "$INF_OUTPUT_RC" "$INF_OUTPUT" && echo "$INF_OUTPUT" | grep -q "DEFLATE_ALL_PASS"; then - TOTAL=$((TOTAL + 22)) - PASS=$((PASS + 22)) - echo " PASS: all 22 inflate/deflate checks" + TOTAL=$((TOTAL + 24)) + PASS=$((PASS + 24)) + echo " PASS: all 24 inflate/deflate checks" else - TOTAL=$((TOTAL + 22)) - FAIL=$((FAIL + 22)) + TOTAL=$((TOTAL + 24)) + FAIL=$((FAIL + 24)) echo " FAIL: inflate/deflate tests" echo "$INF_OUTPUT" | grep -iE "assert|error|FAIL" | head -5 fi diff --git a/tests/test_inflate.eigs b/tests/test_inflate.eigs index 2e36c34f..e86144ca 100644 --- a/tests/test_inflate.eigs +++ b/tests/test_inflate.eigs @@ -126,6 +126,31 @@ catch e6: c22 is 1 check of ["D22 zlib_deflate null arg caught", c22, 1] +# D23/D24 (#292): inflate is on the sandbox allowlist, so its allocations must +# be charged against max_bytes. They were not — a ~9 KB bomb decompressed to a +# 10M-element list (~800 MB of Values) inside an 8 MiB sandbox, measured at +# 853 MiB peak RSS vs 13 MiB for the charged `zeros` control. The budget only +# ever bounded allocators the caller must NAME a size for; a compressed blob +# names nothing, so amplification walked straight past it. Both the codec's own +# output buffer and the list built from it are now charged. +SB_GET_NAME is 25 +SB_CONST is 0 +SB_CALL is 39 +SB_RETURN is 40 +SB_ABI is 1 + +sb_src is buffer of 2000000 +sb_blob is deflate of sb_src +sb_desc is [SB_ABI, [SB_GET_NAME,0,0, SB_CONST,1,0, SB_CALL,1,0, SB_RETURN], ["inflate", sb_blob]] + +# Refused under a budget far smaller than the decompressed result... +sb_small is sandbox_run of [sb_desc, 1000000, 65536] +check of ["D23 sandboxed inflate charged against max_bytes", sb_small["ok"], 0] + +# ...and the codecs are untouched outside a budgeted sandbox. +sb_round is inflate of sb_blob +check of ["D24 inflate unaffected outside the sandbox", len of sb_round, 2000000] + if fails == 0: print of "DEFLATE_ALL_PASS" if fails > 0: diff --git a/tests/test_sandbox_allow.eigs b/tests/test_sandbox_allow.eigs index fb81c994..5cecf42a 100644 --- a/tests/test_sandbox_allow.eigs +++ b/tests/test_sandbox_allow.eigs @@ -74,7 +74,21 @@ no_fn_escape is (r_fn["ok"] == 0) and ((type of r_fn["result"]) != "fn") r_imp is sandbox_run of [[ABI, [IMPORT,0,0, RETURN], ["math"], [], 0, "", []], 1000] no_import is (r_imp["ok"] == 0) and (r_imp["error"]["kind"] == "sandbox") -if escape_blocked and screen_blk and exit_blk and seed_blk and close_blk and rbb_blk and allow_ok and no_writethrough and no_fn_escape and no_import: +# (g) A RESULT TOO BIG TO SCAN IS NOT "A CALLABLE". The boundary scan is +# node/depth budgeted and fails CLOSED, which is right — but it used to report +# every budget exhaustion as "sandbox result contains a callable", so a large +# list of plain numbers was blamed on a function that was never in the result. +# Both branches are pinned here: the honest message for an unscannable result, +# and the literal claim only when a callable is genuinely present. +r_big is sandbox_run of [[ABI, [GET_NAME,0,0, CONST,1,0, CALL,1,0, RETURN], ["range", 200000]], 1000000, 268435456] +big_msg is r_big["error"]["message"] +big_unscannable is (r_big["ok"] == 0) and (r_big["error"]["kind"] == "sandbox") and ((index_of of [big_msg, "too large to scan"]) >= 0) + +r_fn is sandbox_run of [[ABI, [GET_NAME,0,0, RETURN], ["len"]], 1000] +fn_msg is r_fn["error"]["message"] +fn_named is (r_fn["ok"] == 0) and ((index_of of [fn_msg, "contains a callable"]) >= 0) + +if escape_blocked and screen_blk and exit_blk and seed_blk and close_blk and rbb_blk and allow_ok and no_writethrough and no_fn_escape and no_import and big_unscannable and fn_named: print of "SANDBOX_ALLOW_OK" else: print of "SANDBOX_ALLOW_FAIL"