P3: incremental parser chunk-invariance

[phenomenon0]

Stage 4 — P3 incremental / streaming chunk-invariance

The streaming IncrementalParser emitted different event streams depending on how the input was chunked, so a consumer reacting to events could observe a different document purely because of byte-arrival timing.

Bugs fixed (incremental.go)

• Close-token stall. In stateAfterValue, a closing }/]/) only bounced to the parent state and returned 0, which Feed treated as EventNeedMore — so {a=1} stalled at } and never emitted EndObject.
• Implicit-separator stall. A whitespace-separated next item likewise returned 0 after a state change → stall ({a:1 b:2} stopped at b).
• Early emission on partial tokens. t/f/true/false/null and numbers were emitted as soon as a prefix was seen: feeding true byte-by-byte produced Bool(true) then bare "rue"; 30 produced 3 then 0.
• Path accumulation. Object keys were never popped between sibling fields ({a:1 b:2} → paths .a, .a.b…), and nested list indices collapsed.
• Sum close ) unhandled. No state consumed ), so EventEndSum was never emitted and sum frames never popped.

Rework

• process() treats a state / stack / path change as progress, so zero-width transitions (closing a container, an implicit separator) never stall; genuine no-progress = need-more.
• Extensible tokens (number, identifier/keyword, ref, bare key) only emit once terminated by a non-token byte or at end-of-input. A new atEnd flag, set by End(), flushes the final pending token. (Keyword/bool/null detection moved into the unified parseIdentifier.)
• closeContainer() consumes the matching close token, unwinds the path to the container's entry depth (tracked per frame as pathLen), pops the frame, and emits the correct End event — including EventEndSum for ).
• updatePathIndex() is now frame-relative, fixing sibling and nested-list paths; object keys are popped on each field transition.

Tests (incremental_chunk_test.go)

• The deliverable: events(feed_all(x)) == events(feed_one_byte_at_a_time(x)) == events(split_at_every_single_byte(x)), over scalars, refs, containers, nested structs/lists, sums, multi-byte ∅/UTF-8, whitespace, and malformed input. (EventNeedMore is excluded from the comparison — its frequency is inherently chunk-dependent.)
• TestIncrementalParserFinishesContainers — directly asserts the stall is gone ({a=1}, [1 2 3], Tag(5), etc. finish with their End event and no error).
• TestIncrementalSiblingPathsDoNotAccumulate — {a:1 b:2 c:3} yields key paths .a .b .c.

All pre-existing incremental_test.go tests still pass.

Verification

ulimit -v 6000000; go test -C go ./glyph/ -skip 'Industrial|Cliff|Perf|Savings|Benchmark' -count=1 → PASS. go vet clean. Heavy memory-bomb tests skipped locally for memory safety — CI runs them.

Note

EventNeedMore is deliberately excluded from the invariant (a byte-by-byte feed naturally raises far more need-more signals than a single feed). The structural event sequence is what must be — and now is — identical.

Do not merge — for maintainer review.