observer: a predicate under unobserved: raises, and the block stops leaking its depth (#871)

CHANGELOG.md

@@ -160,6 +160,46 @@ All notable changes to EigenScript are documented here.
   wherever the extension is compiled in — previously the whole file was
   effectively inert without a server) plus DB18–DB26 against the live
   CI postgres service.
+- **An observer predicate inside `unobserved:` raises instead of
+  answering `false` forever (#871).** The block's depth is *dynamic* —
+  it covers every function called from inside it — so a caller adding a
+  performance annotation silently changed a callee's answer (a settle
+  loop returned `-1` instead of `22`), and a bare
+  `loop while not converged` inside one **never terminated**: the
+  predicate could not become true, and the stall backstop that would
+  have ended the loop is gated on the same depth, so both mechanisms
+  that could have saved it failed for the same reason at the same
+  moment. A predicate asked under an `unobserved:` block is being asked
+  a question the runtime structurally cannot answer, and now says so —
+  naming the predicate, the block, and the transitive scope. Everything
+  that does not interrogate the observer is untouched, so the block
+  remains the performance knob it is documented to be.
+  Deliberately *not* fixed by ungating the stall backstop instead: that
+  check reads a frozen trajectory as "quiet", so ungating it would exit
+  every legitimate `unobserved:` loop after 100 iterations — including
+  the accumulator loop README.md:189 measures at 2.7x. With the
+  predicate raising, the hang is unreachable. And `unobserved:` was left
+  dynamic rather than made lexical, because lexical scoping would
+  exclude callees, which is most of what a hot region does.
+
+### Fixed
+
+- **`unobserved:` leaked its depth on every exit edge but one, silently
+  killing the observer for the rest of the process (#871, found while
+  fixing it).** `g_unobserved_depth` is a runtime counter that only
+  `OP_UNOBSERVED_END` decrements, and the compiler emitted that opcode
+  on the fallthrough edge only. A `return`, `break`, or `continue` out
+  of the block — or **any error caught outside it** — left the depth
+  elevated permanently: from then on the observer recorded nothing, and
+  every `report` answered `equilibrium` about a value that was plainly
+  moving. Four independent silent deaths of the runtime's central
+  mechanism, none producing a diagnostic. Fixed the way #726 fixed the
+  identical disease in `g_try_depth`: `break`/`continue`/`return` now
+  emit the `OP_UNOBSERVED_END`s for every block they jump out of
+  (per-loop baselines for the first two), and a `try` handler records
+  the depth at registration and restores it when an error unwinds into
+  the catch. `tests/test_unobserved.eigs` pins all four edges plus
+  nesting, with a moving-value probe.
 
 - **chart renders 1.5× faster at high point counts (#828).** The series
   hot loop called `_chart_map` — a fresh 2-element list — per plotted
@@ -188,8 +228,6 @@ All notable changes to EigenScript are documented here.
   naming the file used and the file shadowed. Sweep of the repo and all
   15 consumer repos found zero imports whose resolution flips.
 
-### Fixed
-
 - **vm_run_bytecode/sandbox_run: an assembled chunk's temporal opcodes
   now turn history recording on themselves (#831).** `g_trace_hist` was
   set only by the bytecode compiler's source scan, so a descriptor

README.md

@@ -194,6 +194,24 @@ nothing; `--lint` flags a block with no plain-variable assignments as
 W020. Note the depth is global rather than lexical: a function *called*
 inside the block runs unobserved too.
 
+Because the depth is dynamic, an interrogation inside the block — or
+inside anything it calls — has no trajectory to classify. Rather than
+answer `false` forever, **an observer predicate raises inside an
+`unobserved:` block** (#871):
+
+```
+Error line 4: converged: the observer is off inside an 'unobserved:'
+block, so this predicate has no trajectory to classify — the block's
+depth is dynamic, so it also covers functions called from inside it
+```
+
+That is what keeps the annotation a *performance* knob: it cannot
+silently change an answer. Before it raised, wrapping a call in
+`unobserved:` made a settle loop return `-1` instead of `22`, and a bare
+`loop while not converged` inside one never terminated at all — the
+predicate could not become true, and the stall backstop that would have
+ended the loop is gated on the same depth.
+
 ### Tensor Math
 
 ```eigenscript

docs/SPEC.md

@@ -1277,7 +1277,11 @@ diverging
 ```
 
 `unobserved:` blocks (and `loop` bodies inside them) skip observer
-updates entirely — use them for hot numeric loops:
+updates entirely — use them for hot numeric loops. The depth is
+dynamic, so it covers functions called from inside the block; an
+observer predicate asked anywhere under one **raises**, because there is
+no trajectory for it to classify (a performance annotation must not
+change an answer):
 
 ```eigenscript
 total is 0

src/compiler.c

@@ -30,6 +30,9 @@ typedef struct {
     int  continue_target;
     int  scope_depth;
     int  has_fresh_env; /* 1 if loop emits OP_LOOP_ENV_FRESH per iteration (for-loops) */
+    int  unobs_depth_at_entry; /* #871: c->unobs_depth when the loop opened —
+                              * break/continue leave every `unobserved:` block
+                              * they jump out of, exactly as they leave `try`. */
     int  try_depth_at_entry; /* c->try_depth when the loop opened — break/continue
                               * must emit one OP_TRY_END per try block they jump
                               * out of, or the handler stays registered (#726) */
@@ -92,6 +95,11 @@ typedef struct Compiler {
                                      * normal name-call path honors the user
                                      * binding, and the builtin fallback is
                                      * semantically identical (fail-open). */
+    int               unobs_depth;  /* #871: lexical `unobserved:` nesting here.
+                                     * g_unobserved_depth is a runtime counter
+                                     * that only UNOBSERVED_END decrements, so
+                                     * every non-fallthrough exit must emit its
+                                     * own — same disease as #726's try_depth. */
     int               try_depth;    /* #726: lexical `try` nesting at this point in
                                      * THIS function's body (a nested AST_FUNC gets
                                      * its own Compiler, so it restarts at 0 —
@@ -193,6 +201,7 @@ static LoopCtx *loop_push(Compiler *c) {
     }
     LoopCtx *lp = xcalloc(1, sizeof(LoopCtx));
     lp->try_depth_at_entry = c->try_depth;
+    lp->unobs_depth_at_entry = c->unobs_depth;
     c->loops[c->loop_depth++] = lp;
     return lp;
 }
@@ -2256,6 +2265,13 @@ static void compile_node_inner(Compiler *c, ASTNode *node) {
              * (#726). Mirrors the loop-env cleanup below. */
             for (int t = c->try_depth; t > lp->try_depth_at_entry; t--)
                 emit(c, OP_TRY_END, node->line);
+            /* #871: and leave every `unobserved:` block being jumped out of.
+             * Without this the runtime depth stayed elevated for the rest of
+             * the PROCESS — the observer silently stopped recording, so every
+             * later `report` read `equilibrium` on a moving value and every
+             * predicate answered about a frozen trajectory. */
+            for (int u = c->unobs_depth; u > lp->unobs_depth_at_entry; u--)
+                emit(c, OP_UNOBSERVED_END, node->line);
             /* Clean up loop env before jumping out, but ONLY if the loop allocated
              * a per-iteration env. While-loops don't — emitting OP_LOOP_ENV_END
              * there would free the surrounding env (often the global one). */
@@ -2290,6 +2306,13 @@ static void compile_node_inner(Compiler *c, ASTNode *node) {
              * ran, so try_count climbed until it pinned at the cap. */
             for (int t = c->try_depth; t > lp->try_depth_at_entry; t--)
                 emit(c, OP_TRY_END, node->line);
+            /* #871: and leave every `unobserved:` block being jumped out of.
+             * Without this the runtime depth stayed elevated for the rest of
+             * the PROCESS — the observer silently stopped recording, so every
+             * later `report` read `equilibrium` on a moving value and every
+             * predicate answered about a frozen trajectory. */
+            for (int u = c->unobs_depth; u > lp->unobs_depth_at_entry; u--)
+                emit(c, OP_UNOBSERVED_END, node->line);
             /* End this iteration's env before jumping back, exactly as break
              * does below — the back-edge target sits BEFORE the per-iteration
              * OP_LOOP_ENV_FRESH, so without this the env is never torn down:
@@ -2484,6 +2507,10 @@ static void compile_node_inner(Compiler *c, ASTNode *node) {
          * every later uncaught error in the process (#726). */
         for (int t = c->try_depth; t > 0; t--)
             emit(c, OP_TRY_END, node->line);
+        /* #871: same for `unobserved:` — a `return` from inside one leaked the
+         * runtime depth permanently and killed the observer process-wide. */
+        for (int u = c->unobs_depth; u > 0; u--)
+            emit(c, OP_UNOBSERVED_END, node->line);
         emit(c, node->data.ret.expr ? OP_RETURN : OP_RETURN_NULL, node->line);
         break;
     }
@@ -2930,8 +2957,10 @@ static void compile_node_inner(Compiler *c, ASTNode *node) {
             name_set_free(&interrogated_here);
         }
         emit(c, OP_UNOBSERVED_BEGIN, node->line);
+        c->unobs_depth++;                                     /* #871 */
         /* Unobserved block body is stored as block.stmts */
         compile_block(c, node->data.block.stmts, node->data.block.count);
+        c->unobs_depth--;
         emit(c, OP_UNOBSERVED_END, node->line);
         break;
     }

src/eigenscript.c

@@ -945,6 +945,18 @@ int observer_slot_stable(const ObserverSlot *s) {
     return 1;
 }
 
+/* #871: the predicate vocabulary, in kind order. The parser derives a kind as
+ * `TOK_CONVERGED + k` (parser.c:842) and vm_slot_predicate switches on the same
+ * k, so this table is the one place the words live — lint's W016 reads it too,
+ * rather than keeping a second copy that could drift out of order. */
+static const char *EIGS_PREDICATE_NAMES[6] = {
+    "converged", "stable", "improving", "oscillating", "diverging", "equilibrium"
+};
+
+const char* eigs_predicate_name(unsigned kind) {
+    return kind < 6 ? EIGS_PREDICATE_NAMES[kind] : "predicate";
+}
+
 /* Slot mirror of builtin_report — same priority order and partial-window
  * fallback, reading the slot trajectory instead of a Value's. */
 /* The entropy-channel report — the classifier for non-numeric bindings, and

src/eigenscript.h

@@ -1220,6 +1220,8 @@ typedef enum {
     EK_USER,              /* `throw` — catch binds the thrown value, not a dict */
 } ErrKind;
 const char* err_kind_name(ErrKind k);
+/* #871: predicate word for a kind (parser/VM/lint share this table). */
+const char* eigs_predicate_name(unsigned kind);
 void rt_error(ErrKind kind, int line, const char *fmt, ...)
     __attribute__((format(printf, 3, 4)));
 char* read_file_util(const char *path, long *out_size);

src/lint.c

@@ -2039,16 +2039,13 @@ static void check_outer_mutation(ASTNode *ast, LintContext *ctx) {
  * #262 aliasing workaround, not a bare read). Sites that mean the bare read
  * deliberately carry `# lint: allow W016` (#399). */
 
-static const char *W016_PREDICATE_NAMES[] = {
-    "converged", "stable", "improving", "oscillating", "diverging", "equilibrium"
-};
 
 static void w016_scan(ASTNode *n, LintContext *ctx) {
     if (!n) return;
     switch (n->type) {
         case AST_PREDICATE: {
             int k = n->data.predicate.kind;
-            const char *nm = (k >= 0 && k < 6) ? W016_PREDICATE_NAMES[k] : "predicate";
+            const char *nm = (k >= 0) ? eigs_predicate_name((unsigned)k) : "predicate";
             lint_warn(ctx, n->line, "W016",
                 "bare '%s' reads the last-observed binding (an invisible "
                 "alias) — write '%s of <var>'", nm, nm);

src/vm.c

@@ -239,6 +239,37 @@ static inline void eigs_observe_safepoint(Env *e) {
 /* Classify an observer slot's trajectory by predicate kind (0..5), matching the
  * bare OP_PREDICATE dispatch. Shared by the named OP_PREDICATE_SLOT/NAME ops,
  * which read a SPECIFIC binding's slot instead of the global last-observed one. */
+/* #871: a predicate asked inside an `unobserved:` block cannot be answered.
+ *
+ * The block suppresses observer updates, and its depth is DYNAMIC — it covers
+ * every function called from inside it. So a caller adding a performance
+ * annotation silently changed a callee's answer (a settle loop returned -1
+ * instead of 22), and a bare `loop while not converged` inside one never
+ * terminated: the predicate could not become true, and the stall backstop that
+ * would have ended the loop is gated on the same depth.
+ *
+ * Returning `false` forever is the worst available answer — the predicate is
+ * being asked a question the runtime structurally cannot answer, so it says
+ * so. This is checked in the opcode handlers rather than inside
+ * vm_slot_predicate because a binding assigned inside the block has no `used`
+ * slot at all, so the classifier is never reached on exactly the path that
+ * hangs. Returns 1 when it raised.
+ *
+ * NOT fixed by ungating the stall backstop instead: that check treats a frozen
+ * trajectory as "quiet", so ungating it would exit every legitimate
+ * `unobserved:` loop after 100 iterations — including the accumulator loop
+ * README.md:189 measures at 2.7x. With the predicate raising, the hang is
+ * unreachable and the backstop's gate is no longer load-bearing here. */
+static int vm_pred_unobserved(uint16_t kind, int line) {
+    if (g_unobserved_depth == 0) return 0;
+    rt_error(EK_VALUE, line,
+             "%s: the observer is off inside an 'unobserved:' block, so this "
+             "predicate has no trajectory to classify — the block's depth is "
+             "dynamic, so it also covers functions called from inside it",
+             eigs_predicate_name(kind));
+    return 1;
+}
+
 static int vm_slot_predicate(const ObserverSlot *s, uint16_t kind) {
     switch (kind) {
     case 0: return observer_slot_converged(s);
@@ -2703,6 +2734,8 @@ static Value *vm_run_ex(EigsChunk *chunk, Env *env, Task *resume) {
                 frame->try_count--; \
                 uint8_t *_catch_ip = frame->try_handlers[frame->try_count].catch_ip; \
                 int _catch_bp = frame->try_handlers[frame->try_count].catch_bp; \
+                g_unobserved_depth = \
+                    frame->try_handlers[frame->try_count].unobs_depth;  /* #871 */ \
                 frame->is_try = (frame->try_count > 0); \
                 while (g_vm.sp > _catch_bp) val_decref(vm_pop()); \
                 vm_push(vm_take_error_value()); \
@@ -4571,6 +4604,7 @@ static Value *vm_run_ex(EigsChunk *chunk, Env *env, Task *resume) {
         g_try_depth++;
         frame->try_handlers[frame->try_count].catch_ip = ip + catch_offset;
         frame->try_handlers[frame->try_count].catch_bp = g_vm.sp;
+        frame->try_handlers[frame->try_count].unobs_depth = g_unobserved_depth;  /* #871 */
         frame->try_count++;
         frame->is_try = 1;
         DISPATCH();
@@ -5131,6 +5165,10 @@ static Value *vm_run_ex(EigsChunk *chunk, Env *env, Task *resume) {
          * only on the slot, an operand with no live slot has no trajectory →
          * the predicate is false. */
         uint16_t kind = read_u16(ip); ip += 2;
+        if (vm_pred_unobserved(kind, current_line)) {     /* #871 */
+            vm_push_slot(slot_null());
+            DISPATCH();
+        }
         int result = 0;
         const ObserverSlot *s =
             vm_slot_value_opaque(g_last_obs_slot_env, g_last_obs_slot_idx)
@@ -5151,6 +5189,10 @@ static Value *vm_run_ex(EigsChunk *chunk, Env *env, Task *resume) {
          * OP_PREDICATE reads. An unobserved/empty slot is false. */
         uint16_t kind = read_u16(ip); ip += 2;
         uint16_t slot = read_u16(ip); ip += 2;
+        if (vm_pred_unobserved(kind, current_line)) {     /* #871 */
+            vm_push_slot(slot_null());
+            DISPATCH();
+        }
         Env *e = frame->fn_env;
         int result = 0;
         const ObserverSlot *ps_l = env_obs_slot(e, (int)slot);
@@ -5167,6 +5209,10 @@ static Value *vm_run_ex(EigsChunk *chunk, Env *env, Task *resume) {
          * its slot (mirrors REPORT_NAME). Undefined name raises like GET_NAME. */
         uint16_t kind = read_u16(ip); ip += 2;
         uint16_t name_idx = read_u16(ip); ip += 2;
+        if (vm_pred_unobserved(kind, current_line)) {     /* #871 */
+            vm_push_slot(slot_null());
+            DISPATCH();
+        }
         const char *name = chunk->const_interns[name_idx];
         uint32_t h = chunk->const_hashes ? chunk->const_hashes[name_idx] : 0;
         if (h == 0) { h = env_hash_name(name); if (chunk->const_hashes) chunk->const_hashes[name_idx] = h; }

src/vm.h

@@ -461,7 +461,12 @@ typedef struct {
      * compiler rejects source that nests deeper than MAX_TRY_HANDLERS; the
      * VM re-checks because untrusted chunks (vm_run_bytecode / sandbox_run)
      * reach TRY_BEGIN without going through the compiler at all (#726). */
-    struct { uint8_t *catch_ip; int catch_bp; } try_handlers[MAX_TRY_HANDLERS];
+    /* #871: unobs_depth is g_unobserved_depth as it stood when this handler
+     * was registered. An error unwinding INTO the catch skips every
+     * OP_UNOBSERVED_END between the raise and here, so without restoring it
+     * the runtime depth stays elevated and the observer silently stops
+     * recording for the rest of the process. */
+    struct { uint8_t *catch_ip; int catch_bp; int unobs_depth; } try_handlers[MAX_TRY_HANDLERS];
     int        try_count;       /* number of active try handlers */
     /* Saved loop-stall globals (so a callee's loops don't inherit caller's
      * accumulated stall count / iteration count). Scoped per call frame. */