Skip to content

feat(ride): displacement arm - ridge-projected teacher+base hidden regression (run-016) - #233

Merged
ronaldtse merged 5 commits into
mainfrom
feat/ride-displacement-arm
Oct 1, 2026
Merged

ronaldtse merged 5 commits into
mainfrom
feat/ride-displacement-arm

Conversation

@ronaldtse

Copy link
Copy Markdown
Contributor

The RIDE displacement arm (TODO.sota-2026/05 in this repo), built and launched after the direction-transferability probe passed decisively (max-layer cos 0.9513, domain-general L0-L8).

Mechanism: regress the student's encoder hidden states toward ridge-projected displaced teacher targets h_t + lam*(h_t - h_b), with the frozen r7 teacher and r6 base both in the training loop. The per-layer ridge map is fit at arm start from the teacher's hidden space to the student's INITIAL hidden space on the same inputs - by linearity the displacement survives the projection. This is the first student-side lever with a mechanism novel to the ledger (representation-space extrapolation; all eight prior levers were loss/data/optimizer-side) and a measured transfer premise, per the closure rule in TODO 03.

  • src/gpu/ride.py - displaced_targets, masked_mse, closed-form fit_ridge (TDD, 12 tests, tests/test_ride.py)
  • modal_distill.py - ride_aux spec wiring: frozen base alongside teacher (byte-table parity asserted at startup), ridge prefit on the first fit_batches batches, sub-batched frozen passes (memory), beta auto-calibrated so the MSE term is 10% of CE at start (pre-registered), ride.pt persisted with checkpoints for resume
  • distill_specs.yaml - ara-diac-small-2-1-ride (run-016): the 2.1 recipe verbatim (r7 labels, Muon, 6 epochs, seed 42) + ride_aux layers 0-8, lam 1.0. Gate: adopt iff >= 0.3pp better than 4.5701 (E4-style bar). In flight.

Ronald Tse added 5 commits October 1, 2026 22:10
…gression

Probe passed (direction domain-general L0-L8, max cos 0.9513):
gpu/ride.py (TDD, 12 tests), ride_aux wiring in distill_sequence
(frozen r6 base alongside r7 teacher, per-layer ridge map fit at
start on student-init geometry, sub-batched frozen passes, beta
auto-calibrated to 10% of CE at start, ride.pt checkpointed),
spec ara-diac-small-2-1-ride (run-016, gate vs 4.5701, adopt >= 0.3pp)
@ronaldtse
ronaldtse force-pushed the feat/ride-displacement-arm branch from 99df38d to 949d003 Compare October 1, 2026 14:10
@ronaldtse
ronaldtse merged commit 79c8962 into main Oct 1, 2026
7 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant