Excuse me. I'm trying to run the example of Qwen 3.5. I managed to export the .pte model. But when I try to run the native Runner:
python -m executorch.examples.models.llama.runner.native \
--model qwen3_5_0_8b \
--pte qwen3_5_0_8b_fp32.pte \
--tokenizer /path/to/tokenizer.json \
--tokenizer_config /path/to/tokenizer_config.json \
--prompt "<|im_start|>user\nHello<|im_end|>\n<|im_start|>assistant\n" \
--params examples/models/qwen3_5/config/0_8b_config.json \
--max_len 128 \
-kv \
--temperature 0.3
I met the error below:
Traceback (most recent call last):
File "/.../executorch/extension/llm/custom_ops/custom_ops.py", line 26, in
op = torch.ops.llama.sdpa_with_kv_cache.default
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/.../miniforge3/envs/execut_main_new/lib/python3.12/site-packages/torch/_ops.py", line 1379, in getattr
raise AttributeError(
AttributeError: '_OpNamespace' 'llama' object has no attribute 'sdpa_with_kv_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/.../executorch/examples/models/llama/runner/native.py", line 26, in
from executorch.extension.llm.custom_ops import custom_ops # noqa # usort: skip
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.../executorch/extension/llm/custom_ops/custom_ops.py", line 44, in
assert len(libs) == 1, f"Expected 1 library but got {len(libs)}"
^^^^^^^^^^^^^^
AssertionError: Expected 1 library but got 0
cc @larryliu0820 @mergennachin @cccclai @helunwencser @jackzhxng @digantdesai
Excuse me. I'm trying to run the example of Qwen 3.5. I managed to export the
.ptemodel. But when I try to run the native Runner:I met the error below:
cc @larryliu0820 @mergennachin @cccclai @helunwencser @jackzhxng @digantdesai