Skip to content

docs(verdicts): run-018 plane v2 clears the rung gate - new on-device frontier #238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ronaldtse wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs(verdicts): run-018 plane v2 clears the rung gate - new on-device frontier #238

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 20 additions & 1 deletion TODO.sota-2026/04-stoicheia-diffusion-wo.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 04 — WO (queued, unscheduled): plane-factorized char-level masked diffusion

Status: ENTERED (2026-10-03, owner: "make it work - ultimately the best SOTA") — in build
Status: RUNG GATE CLEARED (2026-10-04) — teacher-tier phase in flight

## Entry design decision (2026-10-03)

Expand Down Expand Up @@ -70,3 +70,22 @@ lacunae / re-segments / **accentuates** / punctuates. Beats Ithaca
4. Entry criteria: (a) 01 (run-009) lands and re-ranks the teacher
frontier; (b) owner authorizes a new architecture line; (c) a
decode-parity design exists for non-AR models (IMF v2 question).


## Results (measured)

**run-017 (v1, 350k units, 1 epoch, K=4):** full-set Total DER
4.7615 / Morph 3.0530 — 0.19pp off the 4.5701 rung. Grid (subset,
labeled): quality monotonically improving 3.295→2.991→2.934 across
checkpoints (UNDERtrained, not overfit); K flat 2/4/8.

**run-018 (v2, 551k units uncapped, 2 epochs, K=2):** full-set
**Total DER 3.5905 / Morph 2.2149** — the 4.5701 rung gate CLEARED by
0.98pp (21% relative error reduction). ONNX int8 artifact: 219 MB
(-45% vs the shipped 491 MB seq2seq int8static) and 1,145 ms/window
on the same machine where the shipped artifact takes 3,459 ms (-67%).
**The plane model is the new on-device frontier on all three axes.**
Gap to the teacher tier (2.2864): 1.30pp.

**Teacher-tier phase (run-019):** same recipe on the ByT5-LARGE
encoder — the SOTA-dedicated gate (beat 2.2864) is the target.
21 changes: 21 additions & 0 deletions docs/RESULTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1025,3 +1025,24 @@ much as the knowledge. Paradigm-table aux is closed (no dose tuning,
no convention-normalization retry recorded as future work; a lexical
lever, if ever revisited, must be rendered into benchmark-convention
running text). The frontier mover on record remains the r7 news mix.

## run-018 plane v2 — the 4.5701 rung CLEARED: new on-device frontier on all axes (2026-10-04)

The plane-factorized encoder (Stoicheia WO, TODO.sota-2026/04) at its
second configuration clears the student rung decisively:

| model | full-set DER | int8 size | CPU ms/window |
|---|---|---|---|
| ara-diac-small-2.1 (shipped) | 4.5701 | 491 MB | 3,459 |
| **plane v2 (run-018, K=2)** | **3.5905** | **219 MB** | **1,145** |
| r7 teacher | 2.2864 | — | — |

−0.98pp DER (−21% relative), −45% size, −67% latency — all three
on-device axes at once. v1→v2 came from the grid's evidence: uncapped
corpus (551,431 units), 2 epochs, K=2 (K flat across 2/4/8; the
parallel-refinement depth is not where quality lives — bidirectional
conditioning is). Architecture: ByT5-small encoder + diacritic-plane
embedding + per-position haraqat classification; Mask-Predict
self-conditioning; one ONNX graph, host-side K-pass loop, no KV
cache. Morph DER 2.2149. Teacher-tier phase (ByT5-large encoder,
run-019) attacks the 2.2864 SOTA-dedicated gate next.
Loading