feat: Add automatic perf runtime selection - #1308
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Qiong Wu (qiowu) (DingmaomaoBJTU)
left a comment
There was a problem hiding this comment.
Auto runtime selection works for ordinary paths, but normalized local paths are not propagated to the GenAI dispatch path, causing valid bundle inputs such as ~/bundle to be misrouted after detection.
| except Exception as e: | ||
| raise click.ClickException(f"Failed to resolve Hub-hosted ONNX path {model!r}: {e}") from e | ||
| model = hf_model | ||
| runtime = _resolve_runtime(runtime, model) |
There was a problem hiding this comment.
Auto detection uses the normalized model path here, but _run_genai_runtime later reads the original value from ctx.params['model']. For example, normalize_model_arg expands ~/bundle, so this selects winml-genai, then the runner sees the unexpanded ~/bundle, Path(...).is_dir() is false, and it attempts to auto-build it as a model ID. Please pass the normalized model into the GenAI runner or synchronize ctx.params['model'] after normalization, and add a CLI-level regression test.
Summary
autoas the defaultperf --runtimechoicewinml-genaifor model folders containinggenai_config.jsonwinmlfor all other model inputs while preserving explicit runtime choices