Skip to content

docs(models): record full glm-5.3 checkpoint screening - #406

Draft
earlvanze wants to merge 3 commits into
FlashML-org:mainfrom
earlvanze:feat/glm53-full-upstream
Draft

docs(models): record full glm-5.3 checkpoint screening#406
earlvanze wants to merge 3 commits into
FlashML-org:mainfrom
earlvanze:feat/glm53-full-upstream

Conversation

@earlvanze

Copy link
Copy Markdown

Scope

Record the full LibertAIDAI/GLM-5.3-NVFP4 checkpoint on the existing glm_moe_dsa path, preserving GLM-5.2 aliases, with config regression coverage and measured Vast screening evidence. This is full GLM-5.3, not Flash.

Measured evidence

One RTX PRO 6000 Blackwell Max-Q 96 GB, driver 595.84, Ubuntu 24.04.3, 36.48 effective CPU cores and a 1.87 TiB container memory limit. Runtime commit 55120d9. The report includes the exact inspected launch command and immutable links to all raw request results.

Initial 32-token direct-worker screen: c8 3.937 aggregate output tok/s; c16 offered against eight running slots 6.297. All 24 requests truncated during reasoning; zero final coding answers. Neither result qualifies the requested c8 target. A separate arithmetic smoke returned the correct answer but exposed a reasoning-parser formatting issue.

Verification and remaining work

Locally ran server tests plus the new model-config tests: 541 passed. That run included a separately prepared, uncommitted parser fix; this PR does not include that fix and does not claim patched-runtime validation.

Draft: longer repeated benchmarks, final-answer checks, actual serverless routing and idle/cached-restart lifecycle remain pending. No production-readiness or per-user 15 tok/s claim. No performance-engine change or A/B speedup claim.

AI-assisted under operator direction; preliminary evidence is intentionally retained, including failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant