feat(MODEL-MM-GLM53-FLASH): W9c-3 device compose forward in glm5_next_device.cpp - #3175
Open
localai-org-maint-bot wants to merge 3 commits into
Open
localai-org-maint-bot wants to merge 3 commits into
localai-org-maint-bot wants to merge 3 commits into
Conversation
mudler
force-pushed
the
row/MODEL-MM-GLM53-FLASH-W9C3
branch
from
September 13, 2026 09:33
d4a16bd to
adf3cff
Compare
… and the remaining eight arms Three of eleven compute arms are on the device after W9c-3a (expert GEMM), W9c-2 (KDA recurrence, MoE router topk), and W9c-3b (KV binding). The other eight still run on the interposed CPU queue. This spec defines the compose wave that grows glm5_next_device.cpp from a 16-line stub to a full device forward in the kimi_linear_device.cpp single-queue shape, moving six arms to the device (RMSNorm, embedding gather, lm_head, k-pool indexer, MoE combine, dense+shared MLPs) and leaving two as host-fallback islands (MLA attention, owned by W9c-1; mHC sites, owned by O34). The per-arm device op map includes provider availability on both CUDA and ROCm, and the k-pool wiring closes O36. Issue #3174 imported as ISSUE-GH-3174. O43 updated to reflect the current state (three of eleven arms on device) and to reference #3174. The ## Now section and the wave briefing updated to mark W9c-3 as spec'd. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki]
…_device.cpp Grow the 16-line stub into the full device-resident compose forward following the kimi_linear_device.cpp single-queue pattern. Four arms move to the device: embedding (vt::Embedding), RMSNorm at all sites (vt::RmsNorm), MoE combine (through the queue passed to MoeForward), and lm_head (vt::MatmulBT). The KDA recurrence and MoE routed experts pass the Dev through their existing device arms from W9c-2 and W9c-3a. Three arms stay as host-fallback islands: mHC sites (no vt:: provider for kDeepseekV4Mhc), DSA/MLA attention (monolithic host function), and dense+shared MLP (deepseek_v4::ClampedSwiGLU has no vt:: device op). The spec planned to move the dense MLP to the device; that arm is deferred until a vt::ClampedSwiGLU op exists. Routing: VT_GLM5_NEXT_DEVICE=1 delegates Glm5NextHostForward to Glm5NextDeviceForward, a superset of VT_GLM5_NEXT_DEVICE_EXPERTS. Documented in docs/ENVIRONMENT.md. On a CPU queue the vt:: kernels use float32 accumulation where the host reference uses double, so the output agrees within a float-vs-double envelope. The test compares device vs host with 1.0 max_abs tolerance and greedy-token agreement: 5740 assertions across 3 cases, 33/33 total. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki]
mudler
force-pushed
the
row/MODEL-MM-GLM53-FLASH-W9C3
branch
from
September 15, 2026 00:29
adf3cff to
7572f0b
Compare
localai-org-maint-bot
changed the title
… to O55 The implementation discovered that DenseMlpForward's ExpertGate calls deepseek_v4::ClampedSwiGLU, which has no vt:: device op. vt::MoeSiluMul is plain silu(gate)*up without clamping, so it is not a substitute. The dense+shared MLP stays as a host-fallback island until a vt::ClampedSwiGLU op is added; O55 owns that gap. The arm table (line 3487) and scope description (lines 3491-3495, 3561-3569) are updated: arm 9 is HOST ISLAND (ClampedSwiGLU), not MOVES. The ## Now section is updated: seven of eleven arms are on the device (up from three), four are host islands, and the CPU test passes (5740 assertions, 33/33 total). The GPU gate on dgx:gpu0 and strix:gpu0 is owed. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The W9c-3 compose wave: the spec and the device-resident forward for
GLM-5.3-Flash, growing
glm5_next_device.cppfrom a 16-line stub to a fulldevice forward in the
kimi_linear_device.cppsingle-queue shape.What this PR does
.agents/specs/glm5-next-flash.mdglm5_next_device.cpp: the full device-resident compose forwardvt::Embedding), RMSNorm(
vt::RmsNorm), MoE combine (viaMoeForwardwithdev), andlm_head(
vt::MatmulBT)Devthrough their existingdevice arms from W9c-2 and W9c-3a
vt::provider forkDeepseekV4Mhc), DSA/MLA attention (monolithic host function), anddense+shared MLP (
deepseek_v4::ClampedSwiGLUhas novt::device op)VT_GLM5_NEXT_DEVICE=1delegatesGlm5NextHostForwardtoGlm5NextDeviceForward, a superset ofVT_GLM5_NEXT_DEVICE_EXPERTSVT_GLM5_NEXT_DEVICEindocs/ENVIRONMENT.mdagreement (5740 assertions across 3 cases, 33/33 total)
The dense+shared MLP was planned to move to the device but is deferred until a
vt::ClampedSwiGLUop exists; it stays as a host-fallback island.On a CPU queue the
vt::kernels use float32 accumulation where the hostreference uses double, so the output agrees within a float-vs-double envelope
rather than byte-exact. On a GPU the device kernels match upstream PyTorch's
float32 numerics.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:regolo/glm5.2 [maki]