Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 31 additions & 1 deletion TODO.sota-2026/04-stoicheia-diffusion-wo.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,36 @@
# 04 — WO (queued, unscheduled): plane-factorized char-level masked diffusion

Status: QUEUED (2026-10-01) — entry criteria at the bottom
Status: ENTERED (2026-10-03, owner: "make it work - ultimately the best SOTA") — in build

## Entry design decision (2026-10-03)

Diacritization is a LENGTH-PRESERVING PLANE PROJECTION: the letters
plane is the given skeleton; the haraqat plane is a per-letter
classification. So the model is an ENCODER-ONLY plane predictor, not a
decoder:

- Init: **ByT5-small encoder** (pretrained backbone — the
from-scratch-collapse law; byte+3 table reused).
- Input: byte embeddings of the SKELETON + a diacritic-plane
embedding per position (MASK at pass 1; pass k>1 carries pass k-1
predictions) — Mask-Predict self-conditioning, trained at random
corruption levels; inference with K passes of FULLY PARALLEL
prediction.
- Head: per-position classification over the corpus haraqat-combo
inventory (label = canonical combining-mark combo following each
letter; non-letter positions labeled none).
- Why this can beat the seq2seq rungs: the AR students decode
left-to-right; iʿrāb (33% of the residual) depends on sentence
structure AHEAD. Bidirectional conditioning attacks exactly the
residual's largest component. Encoder-only + K parallel passes is
also a CPU-latency win vs byte-by-byte KV decode.
- Corpus: benchmark-convention RUNNING TEXT ONLY (r5-units +
arabic-combined + news mix) — the r9 convention-drift lesson
applied by design; NO paradigm tables.
- Gates (pre-registered): windowed zero-skip SadeedDiac-25 — rung
gate beat 4.5701 (student tier, on-device class); SOTA-dedicated
gate beat 2.2864 (teacher tier); CPU latency benchmark vs
ara-diac-small-int8static-2.1.
Literature basis: Stoicheia (arXiv 2608.07249) — 405M character-level
masked-diffusion encoder for Ancient Greek; input factors into five
aligned, independently maskable planes (letters, boundaries,
Expand Down
Loading