From c959b62deb188972d017793b0e06a5846e563c53 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sat, 3 Oct 2026 19:28:29 +0800 Subject: [PATCH] docs(todos): Stoicheia WO entered - encoder-only plane predictor design --- TODO.sota-2026/04-stoicheia-diffusion-wo.md | 32 ++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/TODO.sota-2026/04-stoicheia-diffusion-wo.md b/TODO.sota-2026/04-stoicheia-diffusion-wo.md index 7fccd4b..20b9c4b 100644 --- a/TODO.sota-2026/04-stoicheia-diffusion-wo.md +++ b/TODO.sota-2026/04-stoicheia-diffusion-wo.md @@ -1,6 +1,36 @@ # 04 — WO (queued, unscheduled): plane-factorized char-level masked diffusion -Status: QUEUED (2026-10-01) — entry criteria at the bottom +Status: ENTERED (2026-10-03, owner: "make it work - ultimately the best SOTA") — in build + +## Entry design decision (2026-10-03) + +Diacritization is a LENGTH-PRESERVING PLANE PROJECTION: the letters +plane is the given skeleton; the haraqat plane is a per-letter +classification. So the model is an ENCODER-ONLY plane predictor, not a +decoder: + +- Init: **ByT5-small encoder** (pretrained backbone — the + from-scratch-collapse law; byte+3 table reused). +- Input: byte embeddings of the SKELETON + a diacritic-plane + embedding per position (MASK at pass 1; pass k>1 carries pass k-1 + predictions) — Mask-Predict self-conditioning, trained at random + corruption levels; inference with K passes of FULLY PARALLEL + prediction. +- Head: per-position classification over the corpus haraqat-combo + inventory (label = canonical combining-mark combo following each + letter; non-letter positions labeled none). +- Why this can beat the seq2seq rungs: the AR students decode + left-to-right; iʿrāb (33% of the residual) depends on sentence + structure AHEAD. Bidirectional conditioning attacks exactly the + residual's largest component. Encoder-only + K parallel passes is + also a CPU-latency win vs byte-by-byte KV decode. +- Corpus: benchmark-convention RUNNING TEXT ONLY (r5-units + + arabic-combined + news mix) — the r9 convention-drift lesson + applied by design; NO paradigm tables. +- Gates (pre-registered): windowed zero-skip SadeedDiac-25 — rung + gate beat 4.5701 (student tier, on-device class); SOTA-dedicated + gate beat 2.2864 (teacher tier); CPU latency benchmark vs + ara-diac-small-int8static-2.1. Literature basis: Stoicheia (arXiv 2608.07249) — 405M character-level masked-diffusion encoder for Ancient Greek; input factors into five aligned, independently maskable planes (letters, boundaries,