Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,32 @@ All notable changes to EigenScript are documented here.
stated honestly in the gate header: its compile lines name sources via
`$SOURCES` variables the recognizer cannot tell from a link line.

- **The sandbox memory budget now covers the zlib codecs (#292
follow-up).** `inflate`/`deflate` (and their `zlib_*` duals) are on
the `sandbox_run` allowlist but charged nothing against `max_bytes`,
so an allowlisted call allocated straight past the cap that exists to
protect the host from untrusted code: a 9,732-byte compressed blob
decompressed to a 10M-element list (~800 MB of `Value`s) inside a
sandbox capped at 8 MiB — measured 853 MiB peak RSS, against 13 MiB
for the charged `zeros` control under the same budget. The `{ok:0}`
arrived only after the memory was already taken. Every other
allocator makes the caller *name* a size, which is what the charge
reads; a compressed blob names nothing and amplifies ~1000x, so the
"bounded by loop-iteration cap × per-op size" reasoning that covers
the remaining uncharged allocators does not hold for these. Both the
codec's own output buffer (charged as it grows) and the list built
from it are now charged. Behavior outside a budgeted `sandbox_run` is
unchanged — `sandbox_charge` is a no-op there.

- **`sandbox_run` no longer blames a callable for a result it merely
could not scan.** The boundary scan is node/depth budgeted and fails
closed — correct — but every exhaustion was reported as `sandbox
result contains a callable`, so a large list of plain numbers was
attributed to a function that was never in the result. Budget
exhaustion now reports that it could not scan the result; the literal
claim is reserved for an actual sighting. Still refuses in both
cases.

- **Widget drawing is contained (#823).** `render` now wraps every
widget's draw in a clip of its own rect intersected with its
ancestors' — `canvas` `on_paint` included, so a paint callback can no
Expand Down
2 changes: 1 addition & 1 deletion docs/BUILTINS.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ Compact typed arrays of doubles with O(1) indexed access. Iterable with
|------|-----------|-------------|
| `vm_run_bytecode` | `vm_run_bytecode of <descriptor>` | Assemble a chunk from a descriptor and run it on the C VM, returning the result. Descriptor: `[abi, code, constants, functions?, param_count?, name?, local_names?]` — `abi` is the **bytecode ABI revision** the producer was built against (currently `1`; see below); `code` is a list of byte ints (opcodes + little-endian operands, 16-bit except `OP_LINE`'s, which is 32-bit since #630); `constants` is the pool; `functions` is a list of nested descriptors referenced by `OP_CLOSURE` (nested descriptors carry **no** `abi` element); `local_names` (slot order) sizes the call frame and names parameters. The minimal `[abi, code, constants]` form is a flat module chunk. The bridge for an EigenScript-written compiler: emit bytecode as data, execute it on the same VM (and JIT) the C compiler's output uses. Caller supplies a well-formed chunk ending in `OP_RETURN`. **A descriptor whose `abi` is missing or does not match the runtime raises (kind `value`) rather than executing** — #704: opcode numbers *and* operand widths are the bytecode ABI, and before the stamp a producer built against an older revision ran misaligned garbage at exit 0 with no error. Producers must hardcode the revision as a literal; a producer that reads the runtime's current value back would always agree and the check would be decoration. A non-list descriptor also raises (it silently returned `null` before #704). |
| `record_history` | `record_history of flag` | Enable (nonzero) / disable (0) per-assignment history recording that `prev of x` and `<kw> is x at <line>` temporal queries read (sets both value- and observer-state history). The C compiler auto-enables it when compiling a temporal query; a self-hosted compiler calls this. The flag must be a number — a non-numeric flag raises (it is not silently treated as disable). Returns the previous setting. |
| `sandbox_run` | `sandbox_run of [descriptor, max_iterations?, max_bytes?]` | Run a chunk (same ABI-stamped descriptor as `vm_run_bytecode`) under safety bounds. Does not throw: an ABI-revision mismatch comes back as `{ok: 0, error: {kind, message, line}}` with a message naming the revision, distinct from a malformed chunk's `invalid chunk descriptor`, so a grading ladder can tell "your producer is stale" from "your bytecode is wrong" without re-running. **Fail-closed**: only a pure-compute *allowlist* (math, bit, list/dict/string ops, buffers, json, regex, observer reads, parse/tokenize, `print`/`assert`) is visible — every other builtin (file/process/network/db, code-exec, threads, channels, terminal, `exit`, global-state mutators like `set_observer_thresholds`, and the whole extension surface) is shadowed by a blocked stub, so a new builtin is denied by default. The sandbox env is a **sealed root**, not a child of the host global env: the allowlist is copied in, so an outward assignment (`x is v`, `OP_SET_NAME`) has no outer binding to write through to and a name the host defines later is not reachable. `import` is gated at the opcode — it is not a name, so the allowlist never covered it. A **callable cannot cross the boundary**: a `fn` in the result closes over the sandbox env and would run in the host after the caps are restored, so it comes back as `{ok: 0, error: {kind: "sandbox", ...}}` instead. Loops are capped at `max_iterations` (default 1e6), so neither a runaway loop nor a single blocking call can hang the host. **Memory is capped at `max_bytes` (default 256 MiB)**: the size-controlled allocators (`zeros`/`fill`/`buffer`/`range`) charge a per-run budget, so a single huge allocation *or* an aggregate across a loop raises a caught error (→ `{ok:0}`) instead of an uncatchable out-of-memory `abort()`. Runtime errors are caught. Returns `{ok: 1/0, result: value}`. For validating untrusted/generated code. |
| `sandbox_run` | `sandbox_run of [descriptor, max_iterations?, max_bytes?]` | Run a chunk (same ABI-stamped descriptor as `vm_run_bytecode`) under safety bounds. Does not throw: an ABI-revision mismatch comes back as `{ok: 0, error: {kind, message, line}}` with a message naming the revision, distinct from a malformed chunk's `invalid chunk descriptor`, so a grading ladder can tell "your producer is stale" from "your bytecode is wrong" without re-running. **Fail-closed**: only a pure-compute *allowlist* (math, bit, list/dict/string ops, buffers, json, regex, observer reads, parse/tokenize, `print`/`assert`) is visible — every other builtin (file/process/network/db, code-exec, threads, channels, terminal, `exit`, global-state mutators like `set_observer_thresholds`, and the whole extension surface) is shadowed by a blocked stub, so a new builtin is denied by default. The sandbox env is a **sealed root**, not a child of the host global env: the allowlist is copied in, so an outward assignment (`x is v`, `OP_SET_NAME`) has no outer binding to write through to and a name the host defines later is not reachable. `import` is gated at the opcode — it is not a name, so the allowlist never covered it. A **callable cannot cross the boundary**: a `fn` in the result closes over the sandbox env and would run in the host after the caps are restored, so it comes back as `{ok: 0, error: {kind: "sandbox", ...}}` instead. That scan is node/depth budgeted and fails closed, so a result too large to scan is also refused — with a message saying so, distinct from the one naming an actual callable. Loops are capped at `max_iterations` (default 1e6), so neither a runaway loop nor a single blocking call can hang the host. **Memory is capped at `max_bytes` (default 256 MiB)**: the size-controlled allocators (`zeros`/`fill`/`buffer`/`range`/`concat`) and the zlib codecs (`inflate`/`deflate` and their `zlib_*` duals — both the codec buffer and the list of values built from it) charge a per-run budget, so a single huge allocation *or* an aggregate across a loop raises a caught error (→ `{ok:0}`) instead of an uncatchable out-of-memory `abort()`. The codecs matter here because every other allocator makes the caller *name* a size, which is what the charge reads, while a compressed blob names nothing and amplifies ~1000x. Runtime errors are caught. Returns `{ok: 1/0, result: value}`. For validating untrusted/generated code. |

### Bytes ↔ values

Expand Down
1 change: 0 additions & 1 deletion examples/ui_canvas_events.eigs
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,5 @@ define on_key(ev) as:
define on_tick(r) as:
sb.text is status

ui._layout of [root, 0, 0]
ui.app_loop of [root, on_key, on_tick]
gfx_close of null
65 changes: 58 additions & 7 deletions src/builtins.c
Original file line number Diff line number Diff line change
Expand Up @@ -2884,17 +2884,30 @@ static int sandbox_name_allowed(const char *name) {
* the fail-closed direction: an unwalkable result is refused, not trusted. */
#define SANDBOX_RESULT_MAX_DEPTH 32
#define SANDBOX_RESULT_MAX_NODES 100000
static int sandbox_value_has_callable(Value *v, int depth, long *budget) {
/* Returns 1 if the result contains — or cannot be shown NOT to contain — a
* callable. Fails closed on an exhausted node/depth budget, but sets
* `*unverified` when that is why, because the two are different facts and the
* caller reports them differently: a 10M-element list of numbers is not "a
* callable crossing the boundary", it just outruns the scan. Blaming a
* callable there sends the caller hunting for a function that was never in
* the result. */
static int sandbox_value_has_callable(Value *v, int depth, long *budget,
int *unverified) {
if (!v) return 0;
if (v->type == VAL_FN || v->type == VAL_BUILTIN) return 1;
if (depth > SANDBOX_RESULT_MAX_DEPTH || --(*budget) <= 0) return 1;
if (depth > SANDBOX_RESULT_MAX_DEPTH || --(*budget) <= 0) {
*unverified = 1;
return 1;
}
if (v->type == VAL_LIST) {
for (int i = 0; i < v->data.list.count; i++)
if (sandbox_value_has_callable(v->data.list.items[i], depth + 1, budget))
if (sandbox_value_has_callable(v->data.list.items[i], depth + 1,
budget, unverified))
return 1;
} else if (v->type == VAL_DICT) {
for (int i = 0; i < v->data.dict.count; i++)
if (sandbox_value_has_callable(v->data.dict.vals[i], depth + 1, budget))
if (sandbox_value_has_callable(v->data.dict.vals[i], depth + 1,
budget, unverified))
return 1;
}
return 0;
Expand Down Expand Up @@ -3029,15 +3042,21 @@ Value* builtin_sandbox_run(Value *arg) {
* Drop the result and report it as a sandbox denial — same {kind, message,
* line} shape as any other refusal, so a grading ladder sees it. */
long scan_budget = SANDBOX_RESULT_MAX_NODES;
if (ok && result && sandbox_value_has_callable(result, 0, &scan_budget)) {
int scan_unverified = 0;
if (ok && result &&
sandbox_value_has_callable(result, 0, &scan_budget, &scan_unverified)) {
val_decref(result);
result = NULL;
ok = 0;
Value *ev = make_dict(3);
dict_set_owned(ev, "kind", make_str(err_kind_name(EK_SANDBOX)));
dict_set_owned(ev, "message",
make_str("sandbox result contains a callable; "
"functions cannot cross the sandbox boundary"));
make_str(scan_unverified
? "sandbox result too large to scan for "
"callables (exceeds the result node/depth "
"budget); refusing to cross the boundary"
: "sandbox result contains a callable; "
"functions cannot cross the sandbox boundary"));
dict_set_owned(ev, "line", make_num(0));
dict_set_owned(out, "error", ev);
}
Expand Down Expand Up @@ -4821,6 +4840,15 @@ static int zlib_bytes_arg(Value *arg, const char *who,
/* Wrap a finished byte buffer as the list-of-ints result value (the
* read_bytes shape). Takes ownership of nothing; caller still frees. */
static Value *zlib_bytes_result(const unsigned char *buf, unsigned long n) {
/* #292: the result is `n` fresh number Values at sizeof(Value)+sizeof(Value*)
* each — ~80 bytes per decompressed BYTE. Charging only the codec's own
* output buffer would therefore miss 98% of the cost, so charge the list
* here too, with the same accounting range/zeros use. Without this an
* allowlisted `inflate` allocates straight past max_bytes: the budget
* bounds allocators the caller has to *name* a size for, and a compressed
* blob names nothing. */
if (!sandbox_charge((size_t)n * (sizeof(Value) + sizeof(Value *))))
Comment on lines +4846 to +4850
return make_null();
Comment on lines +4850 to +4851
Value *result = make_list((int)n);
for (unsigned long i = 0; i < n; i++)
list_append_owned(result, make_num((double)buf[i]));
Expand All @@ -4845,6 +4873,15 @@ static Value *zlib_inflate_impl(const char *who, int window_bits, Value *arg) {
}
size_t cap = src_n * 3 + 64;
if (cap > EIGS_INFLATE_MAX_OUT) cap = EIGS_INFLATE_MAX_OUT;
/* #292: charge the codec's own buffer as it grows, so a bomb is refused
* before the memory is touched rather than after. EIGS_INFLATE_MAX_OUT
* bounds this at 256 MiB, which is the *default* whole-run budget — a
* caller that lowered max_bytes must not be overrun by one call. */
if (!sandbox_charge(cap)) {
inflateEnd(&zs);
free(src);
return make_null();
}
unsigned char *out = xmalloc(cap);
int zrc = Z_OK;
for (;;) {
Expand All @@ -4858,6 +4895,12 @@ static Value *zlib_inflate_impl(const char *who, int window_bits, Value *arg) {
if (cap >= EIGS_INFLATE_MAX_OUT) break; /* limit raise below */
size_t ncap = cap * 2;
if (ncap > EIGS_INFLATE_MAX_OUT) ncap = EIGS_INFLATE_MAX_OUT;
if (!sandbox_charge(ncap - cap)) { /* #292: charge the delta */
inflateEnd(&zs);
free(out);
free(src);
return make_null();
}
out = xrealloc(out, ncap);
cap = ncap;
}
Expand Down Expand Up @@ -4915,6 +4958,14 @@ static Value *zlib_deflate_impl(const char *who, int window_bits, Value *arg) {
return make_null();
}
uLong bound = deflateBound(&zs, (uLong)src_n);
/* #292: deflate amplifies far less than inflate (bound ~= src_n), but the
* budget should account for every codec buffer, not just the dangerous
* one — an uncharged allocator is a gap whether or not it is exploitable. */
if (!sandbox_charge((size_t)bound)) {
deflateEnd(&zs);
free(src);
return make_null();
}
unsigned char *out = xmalloc(bound > 0 ? bound : 1);
size_t pos = 0;
int zrc = Z_OK;
Expand Down
24 changes: 18 additions & 6 deletions src/vm.c
Original file line number Diff line number Diff line change
Expand Up @@ -865,14 +865,26 @@ volatile int *g_vm_abort_flag = &g_vm_abort_never;
* the allocation reach abort(). Cumulative over the run (snippets are short, so
* cumulative ~= peak); generous default, overridable via sandbox_run's max_bytes
* arg. No-op outside a budgeted sandbox (active=0), so normal runs pay nothing.
* Plain process globals: sandbox_run is synchronous and single-threaded.
* The three counters are per-THREAD (EigsThread fields reached through the
* g_sandbox_* macros), not the plain process globals this note used to claim:
* two threads — or two EigsStates — running sandbox_run concurrently keep
* separate budgets, and nothing here needs a lock. The save/restore around the
* run is what makes NESTING compose, not what makes it thread-safe.
*
* Charged at: zeros, fill, buffer, range, concat — the size-controlled list/
* tensor allocators (the issue's vectors). NOT yet charged: zeros_like (mirrors
* an already-charged input), matmul output (per-call capped at 10M elems; its
* inputs are charged), and the text_builder growth path — all bounded by the
* loop-iteration cap × per-op size, not the byte budget. Extending the budget to
* them is a one-line sandbox_charge() at each; left out to match #292's scope. */
* tensor allocators (the issue's vectors) — plus the zlib codecs (inflate/
* deflate and their zlib_* duals): both the codec's own output buffer and the
* list of Values built from it. The codecs are the case that breaks the
* "bounded by loop-iteration cap × per-op size" reasoning the rest of this
* note relies on: every other allocator makes the caller NAME a size, which is
* what the charge reads, while a compressed blob names nothing and amplifies
* ~1000x. Uncharged, a 9 KB input decompressed to ~800 MB of Values inside an
* 8 MiB sandbox (measured 853 MiB peak RSS vs 13 MiB for the charged `zeros`
* control). NOT yet charged: zeros_like (mirrors an already-charged input),
* matmul output (per-call capped at 10M elems; its inputs are charged), and the
* text_builder growth path — those genuinely are bounded by the loop-iteration
* cap × per-op size. Extending the budget to them is a one-line
* sandbox_charge() at each; left out to match #292's scope. */
int sandbox_charge(size_t bytes) {
if (!g_sandbox_active || g_sandbox_byte_max == 0) return 1;
/* Overflow-safe: g_sandbox_bytes_used <= g_sandbox_byte_max is an invariant
Expand Down
12 changes: 6 additions & 6 deletions tests/run_all_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2688,15 +2688,15 @@ ZLIB_PROBE_OUT=$(./eigenscript "$ZLIB_PROBE_FILE" 2>&1)
rm -f "$ZLIB_PROBE_FILE"

if ! echo "$ZLIB_PROBE_OUT" | grep -q "compiled without zlib support"; then
echo "[124] DEFLATE Codecs (#684, 22 checks)"
echo "[124] DEFLATE Codecs (#684, 24 checks)"
INF_OUTPUT=$(./eigenscript ../tests/test_inflate.eigs 2>&1); INF_OUTPUT_RC=$?
if rc_ok "$INF_OUTPUT_RC" "$INF_OUTPUT" && echo "$INF_OUTPUT" | grep -q "DEFLATE_ALL_PASS"; then
TOTAL=$((TOTAL + 22))
PASS=$((PASS + 22))
echo " PASS: all 22 inflate/deflate checks"
TOTAL=$((TOTAL + 24))
PASS=$((PASS + 24))
echo " PASS: all 24 inflate/deflate checks"
else
TOTAL=$((TOTAL + 22))
FAIL=$((FAIL + 22))
TOTAL=$((TOTAL + 24))
FAIL=$((FAIL + 24))
echo " FAIL: inflate/deflate tests"
echo "$INF_OUTPUT" | grep -iE "assert|error|FAIL" | head -5
fi
Expand Down
25 changes: 25 additions & 0 deletions tests/test_inflate.eigs
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,31 @@ catch e6:
c22 is 1
check of ["D22 zlib_deflate null arg caught", c22, 1]

# D23/D24 (#292): inflate is on the sandbox allowlist, so its allocations must
# be charged against max_bytes. They were not — a ~9 KB bomb decompressed to a
# 10M-element list (~800 MB of Values) inside an 8 MiB sandbox, measured at
# 853 MiB peak RSS vs 13 MiB for the charged `zeros` control. The budget only
# ever bounded allocators the caller must NAME a size for; a compressed blob
# names nothing, so amplification walked straight past it. Both the codec's own
# output buffer and the list built from it are now charged.
SB_GET_NAME is 25
SB_CONST is 0
SB_CALL is 39
SB_RETURN is 40
SB_ABI is 1

sb_src is buffer of 2000000
sb_blob is deflate of sb_src
sb_desc is [SB_ABI, [SB_GET_NAME,0,0, SB_CONST,1,0, SB_CALL,1,0, SB_RETURN], ["inflate", sb_blob]]

# Refused under a budget far smaller than the decompressed result...
sb_small is sandbox_run of [sb_desc, 1000000, 65536]
check of ["D23 sandboxed inflate charged against max_bytes", sb_small["ok"], 0]

# ...and the codecs are untouched outside a budgeted sandbox.
sb_round is inflate of sb_blob
check of ["D24 inflate unaffected outside the sandbox", len of sb_round, 2000000]

if fails == 0:
print of "DEFLATE_ALL_PASS"
if fails > 0:
Expand Down
Loading
Loading