Align GenAI perf metrics schema

docs/commands/perf.md

@@ -36,7 +36,7 @@ $ winml perf [options]
 | `--use-cache/--no-use-cache` | | flag | `true` | Reuse persistent model build artifacts. `--no-use-cache` performs a fresh build in a temporary folder and discards it after benchmarking. |
 | `--rebuild/--no-rebuild` | | flag | `false` | Force model rebuild even if a cached artifact already exists. |
 | `--module` | | `TEXT` | — | PyTorch module class name for per-module benchmarking (e.g., `BertAttention`). Builds and times each matching instance separately. See [Load and export](../concepts/load-and-export.md). |
-| `--monitor/--no-monitor` | | flag | `false` | Show a live NPU/CPU utilization chart while the benchmark runs and include hardware metrics in the JSON report. |
+| `--monitor/--no-monitor` | | flag | `false` | Show a live NPU/CPU utilization chart while the benchmark runs and include hardware metrics in the JSON report. With `--runtime winml-genai`, the monitor wraps the genai load + generation benchmark. |
 | `--op-tracing` | | `basic\|detail` | — | Enable operator-level profiling. QNN detail tracing requires an EPContext model; a raw ONNX input is detected and compiled automatically with the required profiling options. |
 | `--compile` / `--no-compile` | | flag | `false` | Compile the model to EPContext binaries during build. QNN detail op-tracing enables this automatically for a raw ONNX input unless `--no-compile` or `--skip-build` was explicitly specified. For `--runtime winml-genai` on the NPU, `--compile` pre-compiles each QNN stage (in an isolated subprocess) before generation. |
 | `--compile-timeout` | | `INTEGER` | `300` | *(winml-genai)* Max seconds to compile each EPContext stage before falling back to the original ONNX. Requires `--compile`. |
@@ -48,6 +48,26 @@ $ winml perf [options]
 
 `winml perf` loads the model through `WinMLAutoModel` — accepting both HuggingFace IDs and local ONNX files — then generates random input tensors from the model's I/O configuration. It runs the specified number of warm-up iterations (excluded from statistics) followed by the timed iterations, collecting per-sample latency. The final report includes mean, min, max, P50, P90, P95, P99, standard deviation, and throughput in samples per second. When `--monitor` is active, a hardware polling loop runs in parallel and records NPU / GPU utilization, CPU usage, and device memory alongside the timing data.
 
+Both runtime reports include `schema_version: 2` and a `benchmark_info.runtime` discriminator (`winml` or `winml-genai`). Shared metadata such as `model_id`, `running_model_path`, `device`, `ep`, `iterations`, `warmup`, and `timestamp` uses the same field names where the concepts overlap; GenAI also keeps `bundle_dir` because the runnable artifact is a bundle directory.
+
+When `--memory` is enabled, both `winml` and `winml-genai` reports use the same `memory` field names for shared concepts: RSS baseline, after-compile/load, after-inference, peak, model-load delta, inference/generation delta, and total delta; VRAM local/shared baseline, after-compile/load, after-inference, peak, model-load delta, inference/generation delta, and total delta.
+
+With `--runtime winml-genai`, `winml perf` benchmarks the onnxruntime-genai decoder pipeline rather than a single `session.run()`. The JSON report uses a phase-based schema: `load` contains startup spans, `requests` contains one warmup or timed generation sample per request, `aggregate` summarizes timed requests only, `memory` contains optional RAM/VRAM deltas, and `hw_monitor` contains optional monitor output. The optional `memory` and `hw_monitor` top-level names match the classic `winml` perf report; GenAI keeps `load`/`requests`/`aggregate` instead of classic `latency_ms`/`throughput` because generation has distinct prompt, first-token, and decode phases.
+
+### GenAI metric definitions
+
+| Core metric | JSON field(s) | Definition |
+|---|---|---|
+| Model Load Time | `load.session_load_duration_ms`, `load.native_load_duration_ms` | `session_load_duration_ms` is the outer `GenaiSession.load()` wall-clock span. `native_load_duration_ms` is the onnxruntime-genai `og.Config` + `og.Model` + `og.Tokenizer` span. |
+| Weight Upload Time | `load.weight_upload_duration_ms`, `load.weight_upload_estimate_duration_ms` | Exact upload telemetry is `null` today because onnxruntime-genai does not expose it. The estimate is `model_create_duration_ms` and is labeled by `weight_upload_estimate_source`. |
+| Cold Start Time | `aggregate.cold_start_ttft_duration_ms`, `aggregate.cold_start_total_duration_ms` | Load plus the first request's TTFT, or load plus the first request's total request duration. |
+| Warm Start Time / Latency | `aggregate.request_duration_ms` | Timed-request full duration after warmups: template + tokenization + generator creation + model compute + sequence fetch + detokenization. |
+| TTFT | `requests[].model_ttft_duration_ms`, `requests[].request_ttft_duration_ms`, `aggregate.*ttft*` | Model TTFT is prefill + first-token compute. Request TTFT also includes template, tokenization, and generator creation. |
+| Prefill TPS | `requests[].prefill_tokens_per_second`, `aggregate.prefill_tokens_per_second` | Prompt tokens divided by `prefill_duration_ms`. |
+| Decode TPS | `requests[].steady_state_decode_tokens_per_second`, `aggregate.steady_state_decode_tokens_per_second` | Tokens after the first divided by the sum of per-token decode durations after the first. |
+| RAM Usage | `memory.rss_*` | Classic-compatible RSS fields such as `rss_baseline_mb`, `rss_after_compile_mb`, `rss_after_inference_mb`, `rss_model_load_delta_mb`, `rss_inference_delta_mb`, and `rss_total_delta_mb`. `rss_checkpoint_peak_mb` is the maximum of the sampled checkpoints, not a continuously sampled peak. Requires `--memory`. |
+| VRAM Usage | `memory.vram_*` | Adapter memory fields are emitted only when the effective GenAI route proves a specific accelerator adapter. Fields include baseline, after-compile, after-inference, load/inference/total deltas, and `vram_*_checkpoint_peak_mb` checkpoint maxima. Requires `--memory`. |
+
 ## Examples
 
 Basic benchmark on the best available device:

docs/samples/qwen3-genai-bundle.md

@@ -113,8 +113,11 @@ winml perf -m out/qwen3-bundle --runtime winml-genai --device npu --compile \
 `--device npu` selects the QNN execution provider: `winml perf` registers the WinML
 QNN EP and the bundle's `context` and `iterator` stages run on the NPU HTP, while the
 CPU companions handle the embedding lookup and vocab projection. The command reports
-time-to-first-token (prefill) and decode throughput, and writes a results JSON under
-`~/.cache/winml/perf/`.
+canonical GenAI phases: session/native load, best-effort weight-upload estimate,
+cold-start TTFT/total, request/model TTFT, prefill throughput, steady-state decode
+throughput, full request latency, optional RAM/VRAM deltas, and a results JSON under
+`~/.cache/winml/perf/`. Exact weight-upload telemetry is currently `null` because
+onnxruntime-genai does not expose it; the estimate is labeled in JSON.
 
 !!! tip "One command from a model id (auto-build)"
     `winml perf --runtime winml-genai` also accepts a HuggingFace **model id** directly.