fix(models): serve qwen4_exp FTW checkpoints with PLE streamed from source - #405
Open
Eng-Ahmd wants to merge 1 commit into
Open
fix(models): serve qwen4_exp FTW checkpoints with PLE streamed from source#405Eng-Ahmd wants to merge 1 commit into
Eng-Ahmd wants to merge 1 commit into
Conversation
…ource The FTW converter stores dense weights and expert banks only; the PLE n-gram table is skipped by iter_weights and no safetensors index is copied. Resolving PLE rows from an FTW dir therefore found zero shards. Fall back to the conversion source recorded in freetoken_weight.json, which covers both the disk and pinned PLE paths.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem: serving a converted FTW checkpoint of Qwen3.8-Flash-Next-NVFP4 fails during load_host_tables with ValueError: PLE shard indices are not contiguous 0..N-1: []. The conversion itself succeeds.
Root cause: convert_checkpoint stores dense weights and expert banks only. qwen4_exp iter_weights deliberately skips ngram_embedding.* (the table loads via load_ple_table), and the safetensors index is intentionally not copied. So _ple_table_files finds no index and no *.safetensors in the FTW dir and resolves zero files, for both the disk (resolve_row_source) and pinned (load_ple_table) paths.
Change (python/freetoken/models/qwen4_exp/weight.py, _ple_table_files only): when the folder is an FTW checkpoint, read source_model_path from freetoken_weight.json and resolve the PLE shards from the conversion source. Missing source raises an actionable error instead of the empty-list message.
Verification: _ple_table_files on the nvidia safetensors dir still resolves the single model-fp8-mtp-ple.safetensors file; a synthetic FTW dir pointing at it resolves identically; a dangling source raises the new error; plain non-index dirs keep the legacy fallback. tests/models/qwen4_exp/test_config.py: 8 passed.
Scope: qwen4_exp PLE resolution only. No Laguna changes.