Skip to content

test(gateway): N107c the closed-schema property test and ratchets close their bypasses - #581

Draft
LamaSu wants to merge 6 commits into
fix/n107b-closed-schemafrom
fix/n107c-test-harness
Draft

LamaSu wants to merge 6 commits into
fix/n107b-closed-schemafrom
fix/n107c-test-harness

Conversation

@LamaSu

@LamaSu LamaSu commented Oct 4, 2026

Copy link
Copy Markdown
Owner

Board N107c, stacked on #538 (N107b). The steward's #6431 made the 6 test-harness MEDIUMs of #538's r3 tests pack a follow-up. Test files only.

r3 finding Fix
Dropped output could pass: render(undefined) became "", rejected telemetry was recorded as "", and a trace read could return an earlier trace The oracle fails on dropped or empty output unless the sink documents a refusal. Each trace probe proves its own span produced the record. (5ed07d8)
Markers invisible to the oracle (Date, Buffer, RegExp, symbol keys) Every generated kind must show its marker in render(make()) before it counts as a witness. (5ed07d8)
The SDK-time exclusion was unjustified while the timing ratchet had holes The timing ratchet follows every alias, computed key, element access and cast. SDK times keep only a number. (3900221, 13f40fc)
The pino prototype-name exclusion was not pinned; .child() sources were not ratcheted Each omitted binding key's real behavior is pinned, and .child() binding sources are scanned. (1ee5e22)
Raw logger messages passed the ratchet: info(raw, 1), computed {["msg"]: raw}, destructured methods The logger ratchet reads every message position and child binding, through casts. (1ee5e22, 13f40fc)
Producer span times passed the timing ratchet: {["startTime"]: x}, span["end"](x), aliased methods Closed (3900221, 13f40fc).

Reproduced at b5718ecb:

  • the logger ratchet probe missed lines 15-31;
  • the timing ratchet probe missed lines 18-29;
  • 14 generated kinds were invisible to the oracle;
  • a dropped trace read returned an earlier trace.

Mutations at 13f40fce: 12 production mutations, each caught here and not caught at 0debe698.

Verify at 13f40fce: tsc 0; 219 of 219 files (3803 tests) on a rerun. The first run's one failure was the completion-real-tier load flake, which passes alone.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Sbd5dpwvmRsJqdff9zvNW6

LamaSu added 4 commits October 3, 2026 20:18
… dropped output (N107c)

This commit only changes tests. It answers the tests pack
(rm-n107b-538-r3-tests-b5718ecb, SHIP-WITH-FIXES).

The verdict said dropped output can pass:
- render(undefined) printed "<nothing>";
- a rejected telemetry emit was recorded as the text "";
- a trace read returned the newest trace, whichever attempt wrote it.
At b5718ec, a read after an attempt that started no span returned an
earlier trace.

Now:
- violations() counts undefined, an empty text and "<nothing>" as no
  output. Each sink must document a refusal by its exact label, with a
  reason, and a documented refusal that does write is a violation too.
- closeValue's five refusals (a bigint, symbol, function, Buffer or typed
  array left out by closeField) are the only ones; every other position
  writes.
- Each trace probe stores its own declared witness (n107b.attempt). A read
  counts only when getRecentTraces, GET /api/traces, getTrace and
  GET /api/traces/:traceId each show that witness, so an attempt that
  started no span reads as no output.

The verdict also said some markers were invisible to the oracle: a Date, a
Buffer, a RegExp and symbol-keyed objects. At b5718ec, 14 generated kinds
were invisible to render(make()). render() now exposes:
- a Date's milliseconds, seconds and ISO time;
- a RegExp's source, a Buffer's text, a typed array's items and a URL;
- a boxed value;
- an error's name, message and stack;
- every non-enumerable and symbol-keyed own property, and getters' values;
- the class name, and what an object's own toString, valueOf and toJSON
  return.
A self-check requires every kind's marker to be visible in render(make())
before it counts as a witness. Only boolean and null carry none.

The pino prototype-name exclusion is now pinned as observed (pino 9.14).
Nine Object.prototype member names throw a TypeError for every value,
constructor throws for a symbol or a null-prototype value, and the rest are
closed. Nothing leaks.

With the stronger oracle, every sink still leaks 0 positions.
Written by implementer-mike.

agent: pcc-readmodels (c255d7dc)
…ild binding (N107c)

This commit only changes tests. It answers the tests pack
(rm-n107b-538-r3-tests-b5718ecb, SHIP-WITH-FIXES): the logger ratchet
could be bypassed, and it did not scan .child(). At b5718ec its probe
missed:
- app.log.info(raw, 1): with two arguments, only the second was checked;
- { ["msg"]: raw } and { "msg": raw }, and a msg key from a constant;
- a destructured, aliased or bound level method, and log["error"](raw);
- a child logger's bindings from a variable or a spread, and a binding
  key named like an Object.prototype member.

The scanner now:
- finds a log call by its resolved signature (pino's LogFn or the
  gateway's DeclaredLogFn), however the function was reached: a property,
  an element access, an alias, a destructured or a bound method;
- reads the message where pino reads it: the first argument unless that is
  a merging object (an object type), else the second. It must be declared.
  A spread argument is refused;
- reads a merging object's msg field from its type, so a computed key that
  names a literal or a constant counts, and an index signature (keys the
  scan cannot name) is refused;
- requires a child logger's bindings to be an object literal with readable
  keys, no spread and no Object.prototype member name, and its options to
  carry no msgPrefix.

The probe adds those forms, plus an index-signature merging object, a
spread argument and a message prefix: 19 lines are caught, and the
declared forms pass. The gateway scan must find more than 80 log calls, so
an empty scan cannot pass. It finds no violation.
Written by implementer-mike.

agent: pcc-readmodels (c255d7dc)
…keep only a number (N107c)

This commit only changes tests. It answers the tests pack
(rm-n107b-538-r3-tests-b5718ecb, SHIP-WITH-FIXES). The timing ratchet
could be bypassed, and the property test's SDK-time exclusion was not
justified while it could. At b5718ec the ratchet's probe missed:
- a computed time key ({ ["startTime"]: t }, or one from a constant);
- span["end"](t), a bound end, and a destructured Sentry function;
- an event's and an exception's time;
- a computed key the scan cannot read.

The scanner now:
- finds a Sentry or OpenTelemetry call by its resolved signature's
  declaration and names it by that declaration, however the function was
  reached;
- reads keys from their type, so a computed key naming a literal counts
  and an unreadable one is refused;
- treats addEvent's third argument (or a time as its second) and
  recordException's second as times;
- follows an API function used as a value. If it is bound, its pre-bound
  arguments are checked. If it is called through call or apply, those
  arguments are checked. If it is passed into a function of this program,
  each use of that parameter must be a direct call, which is checked as a
  call of the function passed. Any other use, such as a variable whose
  annotation drops the API's type, is refused.

The probe adds call, apply, a pre-bound time, an event time as the second
argument, a type annotation that loses the API, a time passed through a
parameter, and a function the ratchet cannot follow. Allowed aliases pass.
The gateway scan must find more than 30 API calls and must follow Sentry's
span API into startDeclaredSpan six times. It finds no violation.

With the ratchet closed, the property test checks the SDK-time positions
with the numeric marker too. Fed a marker directly, an SDK time keeps a
finite number and nothing else: only the four number kinds keep the
numeric marker, and that exclusion is pinned exactly.
Written by implementer-mike.

agent: pcc-readmodels (c255d7dc)
…ions (N107c)

This commit only changes tests. It follows up the tests pack
(rm-n107b-538-r3-tests-b5718ecb) on the two ratchets: a type assertion was
still a way around each of them. Both read an argument's type, so a cast
replaced what the scan saw:
- `raw as unknown as Declared` passed as a declared message;
- `{ msg: raw } as object` hid the msg field, and the same cast hid
  prototype-named child bindings;
- `{ startTime: t } as { name: string }` hid a time option;
- `t as unknown as Record<string, string>` hid an event time.

Both scanners now take each argument's type assertions, parentheses,
satisfies and non-null marks off before reading it, so they see the value
as it is. The probes add those forms. The logger probe catches 22 lines
and the timing probe 23, and the gateway scans still find no violation.
Written by implementer-mike.

agent: pcc-readmodels (c255d7dc)
LamaSu added 2 commits October 4, 2026 00:31
… stream proves each attempt

astra r1 (rm-n107c-581-r1-13f40fce, SHIP-WITH-FIXES), two MEDIUMs. Both
were reproduced at 13f40fc before any change.

1. Logger calls through Function.prototype.call/apply bypassed the
   ratchet. With app.log.info.call(app.log, raw) and
   app.log.info.apply(app.log, [raw]) added to the probe, the scan
   flagged neither line. The visitor now treats call, apply and
   Reflect.apply on a level method (a LogFn-typed value, or
   <logger>.<level>) as a log call. It reads pino's arguments from them,
   and it refuses an apply whose argument list is not an array literal.
   The probe pins call, apply, an unreadable apply, Reflect.apply, and
   call on an element access with a raw msg field. Declared forms of
   call and apply stay unflagged.

2. The stream snapshot could drop every trace and still pass, because it
   was read once per 20 attempts and checked only for its
   "event: connected" line. With the handler mutated to write no trace,
   the property test passed 19/19.
   - Each attempt now reads the stream after the four ordinary paths.
   - The stream must show the attempt's witness whenever the record
     lies in its 20-trace window.
   - Only an attempt that poisons the trace id itself may fall outside
     the window, and the test names each one: 5 of 7355, all traceId
     variants.
   - The same mutation now fails the test.

agent: pcc-readmodels (c255d7dc)
…into N107c

agent: pcc-readmodels (c255d7dc)

This branch had an error being deployed

1 failed deployment
trusted-checks — 5ee9fee6 Deployed Oct 4, 2026 by LamaSu via post-verdicts #66
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant