Skip to content

feat(glm53): add text runtime and measured performance paths - #40

Merged
ro99 merged 16 commits into
mainfrom
feat/glm53f-text-support
Aug 28, 2026
Merged

feat(glm53): add text runtime and measured performance paths#40
ro99 merged 16 commits into
mainfrom
feat/glm53f-text-support

Conversation

@ro99

@ro99 ro99 commented Aug 28, 2026

Copy link
Copy Markdown
Owner

Summary

  • add text-only GLM-5.3-Flash support to chat and the OpenAI-compatible server
  • implement checkpoint validation, hybrid KDA/MLA execution, W8A8 FP8 kernels, fused KDA and mHC transitions
  • add dynamic host-FP8 expert execution, topology-aware pipeline/TP2 placement, and page-level expert reuse
  • add MTP verification, continuous iteration scheduling, paged sequence state, and prefix caching
  • reject vision inputs and preserve exact text behavior without hybrid cache quantization

Measured result

On the available dual RTX 3090 CUDA pair (--devices 0,1):

  • load: 1.65 s
  • prefill: 0.80 tok/s on the 13-token chat prompt
  • decode: 0.90 tok/s greedy

The mixed RTX 3090 / RTX 5060 Ti pair measured 0.91 tok/s decode. Nsight attribution reduced routed-expert prefill H2D traffic from 70.34 GB to 14.04 GB.

Validation

  • cmake --build build-cuda -j $(nproc)
  • build-cuda/strata-tests
  • 331 tests passed, 38 skipped
  • deterministic text matched between the measured device configurations

Deliberate defaults

  • MTP is implemented and available through STRATA_GLM53_MTP=1, but remains opt-in until acceptance amortizes verification for the workload.
  • absorbed resident MLA remains experimental and opt-in until its BF16 boundary equivalence gate is closed.
  • vision and hybrid cache quantization are out of scope.

@ro99
ro99 merged commit 981f4a4 into main Aug 28, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant