Fix ORT GenAI export metadata - #377
Conversation
Copy tokenizer files directly for local model directories and emit Qwen3-VL-specific GenAI metadata, including model type and vision/video fields. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Xiaoyu <xiaoyuzhang@microsoft.com>
Performance Comparison
|
|
The author of this PR, xiaoyu-work, is not an activated member of this organization on Codecov. |
There was a problem hiding this comment.
Pull request overview
This PR updates the ORT GenAI auto-export integration to generate more accurate export metadata for Qwen3-VL and to better handle tokenizer artifacts when the HuggingFace “model id” is actually a local directory path.
Changes:
- Extend
GenaiConfigGenerator.with_vision()to optionally emit Qwen3-VL vision metadata (tokens_per_second,patch_size,window_size). - Update ORT GenAI model-type mapping so Qwen3-VL exports emit
qwen3_vlinstead of reusingqwen2_5_vl. - Copy tokenizer files directly from disk when
hf_model_idpoints to a local directory, and add unit tests for this behavior and the new Qwen3-VL fields.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/mobius/integrations/ort_genai/genai_config.py |
Adds optional Qwen3-VL vision metadata fields to the generated vision section. |
src/mobius/integrations/ort_genai/auto_export.py |
Adjusts model-type mapping, writes Qwen3-VL vision/video metadata, and copies tokenizer files from local hf_model_id directories. |
src/mobius/integrations/ort_genai/auto_export_test.py |
Adds/updates unit tests for local tokenizer copying and Qwen3-VL genai_config fields. |
…PI change In transformers 5.14, DynamicCache.layers[N].recurrent_states changed from returning a tensor directly to returning a dict keyed by layer index. Handle both forms so the test works with both old and new transformers versions.
|
@copilot resolve the merge conflicts in this pull request |
|
|
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
- auto_export.py: add deepseek_v4 mapping from main; keep qwen3_vl->qwen3_vl; merge MTP config block from main alongside local tokenizer copy logic - integration_test.py: keep transformers >=5.14 recurrent_states comment from our branch
已解决合并冲突（commit
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (2)
src/mobius/integrations/ort_genai/auto_export.py:1141
- Unresolved merge-conflict marker line (>>>>>>> origin/main) remains here; it must be removed or the module won't import.
result[tf] = os.path.join(directory, tf)
src/mobius/integrations/ort_genai/auto_export.py:1110
- Unresolved merge-conflict markers are present before the MTP config block, which will break parsing. Keep the tokenizer-copy comment and remove the conflict markers/selector line.
json.dump(
{
"model": {"filename": "mtp/model.onnx"},
"inputs": [
Summary
Validation