Smolvla ONNX LP review

content/learning-paths/cross-platform/smolvla-onnx-conversion/_index.md

@@ -1,29 +1,71 @@
 ---
-title: Export and Quantize SmolVLA for ONNX Runtime on Arm
+title: Export and quantize SmolVLA for ONNX Runtime on Arm
 
-draft: true
-cascade:
-    draft: true
-
 description: Export SmolVLA from PyTorch to ONNX, quantize linear weights to INT4 with TorchAO, and compare action accuracy and ONNX Runtime latency on an Arm CPU.
 
 minutes_to_complete: 60
 
 who_is_this_for: This is an advanced topic for machine learning developers who want to export and quantize a vision-language-action model for ONNX Runtime on Arm.
+
 learning_objectives:
   - Export SmolVLA from PyTorch as an ONNX model
   - Run and validate the FP32 ONNX model with ONNX Runtime on an Arm CPU
   - Quantize eligible linear weights to INT4 and store them in a packed ONNX model
   - Run the FP32 and INT4 models with identical inputs and compare their action outputs and ONNX Runtime latency
 
 prerequisites:
-  - An Arm Linux system, such as an Arm cloud instance or Radxa Orion O6
+  - An Arm Linux system, such as an Arm cloud instance or Radxa Orion O6, with Python 3.12 installed and at least 50 GB of free storage
   - Familiarity with Python, PyTorch, and Linux command-line tools
 
+# START generated_summary_faq
+generated_summary_faq:
+  template_version: summary-faq-v3
+  generated_at: '2026-08-26T22:01:23Z'
+  generator: ai
+  ai_assisted: true
+  ai_review_required: true
+  model: gpt-5
+  prompt_template: summary-faq-v3
+  source_hash: 928438780614fcd9081745b0f052058751923431630919351e91acf15e4a76ab
+  summary_generated_at: '2026-08-26T22:01:23Z'
+  summary_source_hash: 928438780614fcd9081745b0f052058751923431630919351e91acf15e4a76ab
+  faq_generated_at: '2026-08-26T22:01:23Z'
+  faq_source_hash: 928438780614fcd9081745b0f052058751923431630919351e91acf15e4a76ab
+  summary: >-
+    You'll export SmolVLA from PyTorch to FP32 ONNX and validate it with ONNX Runtime on an Arm Linux CPU.
+    First, you'll set up the pinned model, source, and Python environment. Then, you'll export the
+    model and compare its output with PyTorch. Finally, you'll quantize eligible linear weights to
+    packed INT4 with TorchAO and compare FP32 and INT4 action outputs and latency using identical inputs.
+  faqs:
+  - question: What do I need to check on my Arm machine before I run the setup scripts?
+    answer: >-
+      Confirm you are on an Arm Linux CPU with Python 3.12 and at least 50 GB of free storage.
+  - question: Where do the scripts place the exported and quantized models?
+    answer: >-
+      The FP32 export is saved at `work/onnx/fp32/model.onnx`. The TorchAO converter writes the
+      packed INT4 model to `work/onnx/int4/smolvla-int4.onnx`.
+  - question: What output confirms that the ONNX export matches PyTorch?
+    answer: >-
+      The exporter ends with `PASS: ONNX Runtime matches PyTorch within atol=0.001 and rtol=0.001`.
+      You can also inspect `work/onnx/fp32/validation.json` to confirm the provider, output shape,
+      and validation result.
+  - question: What exactly gets quantized, and how can I verify it?
+    answer: >-
+      TorchAO applies weight-only INT4 quantization to eligible constant linear weights and replaces
+      supported `MatMul` and `Gemm` operations with packed `com.microsoft::MatMulNBits` operations.
+      The converter reports how many eligible linear operations it converted.
+  - question: How should I compare FP32 and INT4 runs on Arm?
+    answer: >-
+      Run both models with the deterministic reference batch created during export. The comparison
+      writes an action-output figure and JSON report with the latency and overall output error.
+# END generated_summary_faq
+
 author: Tirui Wu
-generate_summary_faq: true
+
+generate_summary_faq: false
 rerun_summary: false
 rerun_faqs: false
+
 skilllevels: Advanced
 subjects: ML
 armips:

content/learning-paths/cross-platform/smolvla-onnx-conversion/convert-and-validate-onnx.md

@@ -15,10 +15,12 @@ actions.
 
 Image source: [SmolVLA paper](https://arxiv.org/pdf/2506.01844).
 
+Now that you understand the model inputs and outputs, export the SmolVLA policy to ONNX and validate it with ONNX Runtime.
+
 ## Export SmolVLA to ONNX
 
 The checkpoint includes the SmolVLA policy and the LeRobot processors used
-before and after inference. The exporter writes the policy to an ONNX graph;
+before and after inference. The exporter writes the policy to an ONNX graph, and
 the processors remain outside it.
 
 Run the exporter:
@@ -30,8 +32,10 @@ python scripts/export_onnx.py \
   --reference-dir work/onnx/fp32/reference
 ```
 
-The export may take several minutes while it loads the checkpoint, traces the
+{{% notice Note %}}
+The export might take several minutes while it loads the checkpoint, traces the
 model graph, and validates the output with ONNX Runtime.
+{{% /notice %}}
 
 The exporter creates a fixed-shape model and a deterministic reference batch.
 The batch includes an explicit flow-matching noise tensor, so the PyTorch and
@@ -86,6 +90,7 @@ of `[1, 50, 7]`, and passes validation.
 
 ## What you've accomplished and what's next
 
-You have exported SmolVLA to FP32 ONNX and validated its action output
-with ONNX Runtime on an Arm CPU. Next, you will quantize the eligible linear
-weights to packed INT4 and run the resulting model through the same interface.
+You've exported SmolVLA to FP32 ONNX and validated its action output
+with ONNX Runtime on an Arm CPU.
+
+Next, you'll quantize the eligible linear weights to packed INT4 and run the resulting model through the same interface.

content/learning-paths/cross-platform/smolvla-onnx-conversion/quantize-and-run-int4-onnx.md

@@ -5,7 +5,7 @@ weight: 4
 layout: learningpathall
 ---
 
-## INT4 quantization scope
+## Scope of INT4 quantization
 
 TorchAO quantizes eligible constant linear weights across the exported SmolVLA
 model. The converter replaces supported ONNX `MatMul` and `Gemm` operations
@@ -15,6 +15,8 @@ This is weight-only quantization, not an entirely INT4 graph.
 
 On supported Arm CPUs, ONNX Runtime can use optimized kernels such as KleidiAI.
 
+With the quantization scope established, convert the eligible linear weights to a packed INT4 ONNX model.
+
 ## Create the packed INT4 model
 
 Run the TorchAO converter:
@@ -25,8 +27,10 @@ python scripts/quantize_onnx_torchao.py \
   --output work/onnx/int4/smolvla-int4.onnx
 ```
 
-The quantization processes each eligible weight individually and may take
+{{% notice Note %}}
+The quantization processes each eligible weight individually and might take
 several minutes on an embedded system.
+{{% /notice %}}
 
 The command creates a packed ONNX file and reports how many eligible linear
 operations were converted. Dynamic or unsupported matrix multiplications
@@ -45,19 +49,14 @@ python scripts/compare_onnx_outputs.py \
 ```
 
 The script runs both models with ONNX Runtime `CPUExecutionProvider` and
-creates:
+creates `work/comparison/smolvla-action-comparison.png` and `work/comparison/smolvla-action-comparison.json`.
 
-```text
-work/comparison/smolvla-action-comparison.png
-work/comparison/smolvla-action-comparison.json
-```
+## Interpret the FP32 and INT4 comparison
 
 The figure compares all seven normalized output channels and median latency.
 The JSON file records the latency and overall output error.
 
-## Review the Orion O6 result
-
-![Seven plots compare FP32 and TorchAO INT4 normalized SmolVLA outputs across all 50 predicted steps for each of seven channels. A latency panel compares median ONNX Runtime latency on a Radxa Orion O6.#center](smolvla-action-comparison.png "SmolVLA Action Comparison")
+![Seven plots compare FP32 and TorchAO INT4 normalized SmolVLA outputs across all 50 predicted steps for each of seven channels. A latency panel compares median ONNX Runtime latency on a Radxa Orion O6.#center](smolvla-action-comparison.png "SmolVLA action comparison")
 
 On the Radxa Orion O6, INT4 reduced median ONNX Runtime latency from 3.33 seconds to 2.06
 seconds, a 1.61x speedup. The normalized outputs had an MAE of 0.153.
@@ -67,9 +66,9 @@ bandwidth available.
 
 ## What you've accomplished
 
-You have converted eligible SmolVLA linear weights to packed INT4 in an ONNX
+You've converted eligible SmolVLA linear weights to packed INT4 in an ONNX
 model, run FP32 and INT4 with identical inputs on an Arm CPU, and compared all
 seven normalized output channels and ONNX Runtime latency.
 
-From here you can integrate the ONNX model into a robotics pipeline, experiment
+From here, you can integrate the ONNX model into a robotics pipeline, experiment
 with different quantization group sizes, or benchmark on other Arm platforms.

content/learning-paths/cross-platform/smolvla-onnx-conversion/set-up-environment.md

@@ -1,11 +1,11 @@
 ---
 title: Set up the SmolVLA environment
-description: Create a Python environment and download the pinned SmolVLA model and LeRobot source.
+description: Set up an Arm Linux environment with the pinned SmolVLA model and LeRobot source for ONNX export and validation.
 weight: 2
 layout: learningpathall
 ---
 
-## Get the scripts
+## Download the scripts
 
 Download the scripts for this Learning Path by copying and pasting the following commands into your terminal:
 
@@ -21,21 +21,23 @@ cd ..
 
 ## Check the system requirements
 
-This Learning Path runs the exported models on an Arm Linux CPU. Review the
-processor, Python version, and available space on the filesystem where you will
-keep the project:
+You'll run the exported models on an Arm Linux CPU.
+
+Review the processor, Python version, and available space on the system you'll use to run the project:
 
 ```bash
 lscpu
 python3 --version
 df -h .
 ```
-
-Use Python 3.12 and confirm at least 50 GB of free storage. Install Git and
-Python 3.12 if they are not already available on your system.
+Install Git and Python 3.12 if they aren't already available on your system.
 
 If `python3 -m venv` fails, install the venv module for your distribution. On
-Ubuntu or Debian, run `sudo apt install python3.12-venv`.
+Ubuntu or Debian, run the following command:
+
+```bash
+sudo apt install python3.12-venv
+```
 
 ## Create the environment
 
@@ -46,14 +48,19 @@ bash scripts/setup.sh
 ```
 
 The script downloads model weights, clones the LeRobot source, and installs
-PyTorch and other Python dependencies. It may take 30 minutes or more depending
-on your network speed.
+PyTorch and other Python dependencies.
+
+{{% notice Note %}}
+The setup might take 30 minutes or more, depending on your network speed.
+{{% /notice %}}
 
-The script creates `work/venv`, checks out the pinned LeRobot source, installs
-the conversion and runtime dependencies, and downloads the SmolVLA policy and
-its SmolVLM2 dependency. It records the installed Python packages in
-`work/environment.freeze.txt` and the source and model revisions in
-`work/revisions.json`.
+The script:
+
+- Creates `work/venv`
+- Checks out the pinned LeRobot source
+- Installs the conversion and runtime dependencies
+- Downloads the SmolVLA policy and its SmolVLM2 dependency
+- Records the installed Python packages in `work/environment.freeze.txt` and the source and model revisions in `work/revisions.json`
 
 Activate the virtual environment so you can use `python` directly in later
 commands:
@@ -78,6 +85,7 @@ PASS: public policy, base model, LeRobot source, and environment are ready
 
 ## What you've accomplished and what's next
 
-You have prepared an Arm Linux environment with the pinned SmolVLA checkpoint,
-source, and Python dependencies. Next, you will export SmolVLA as an FP32 ONNX
-model and validate it with ONNX Runtime.
+You've prepared an Arm Linux environment with the pinned SmolVLA checkpoint,
+source, and Python dependencies.
+
+Next, you'll export SmolVLA as an FP32 ONNX model and validate it with ONNX Runtime.