Add Qwen3.8-27B VL-text loader and WISE padding/thinking fixes - #715
Open
Zhang-zu-hao wants to merge 3 commits into
Open
Add Qwen3.8-27B VL-text loader and WISE padding/thinking fixes#715Zhang-zu-hao wants to merge 3 commits into
Zhang-zu-hao wants to merge 3 commits into
Conversation
Detect Qwen3_5ForConditionalGeneration via AutoConfig before the qwen2/qwen3 CausalLM branch so Qwen3.5-9B is unchanged, and document WISE/FT/GRACE plus ROME scan scaffolding.
… edit example. Right-pad label masks no longer count trailing pads; edit-time train() enables transformers 5.8 gradient checkpointing; optional yaml fields keep old configs loading.
Strip local paths, keep Hub-free loader detection, restore WISE train/eval with try/finally, and record teacher-forcing vs free-generation metrics.
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Qwen3_5ForConditionalGeneration, VL shell + 64-layer hybrid LM). This is not pure-text Qwen3.5-9B (model.layers.{}); the edit path ismodel.language_model.layers.{}.mlp.down_proj.enable_thinking=False, and run the edit loop inmodel.train()so transformers 5.8 gradient checkpointing actually runs.hparams/WISE/qwen3.8-27b.yaml(padding_side: right,enable_thinking: false, layer 53). FT/GRACE yamls are structural only and untested. ROME is.exampleonly.