Skip to content

User/mishrak/add support for extended stats metrics#3

Open
Kritika1128 wants to merge 3 commits into
rel-1.23.2-winml-standalonefrom
user/mishrak/add-support-for-extended-stats-metrics
Open

User/mishrak/add support for extended stats metrics#3
Kritika1128 wants to merge 3 commits into
rel-1.23.2-winml-standalonefrom
user/mishrak/add-support-for-extended-stats-metrics

Conversation

@Kritika1128

@Kritika1128 Kritika1128 commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

Add Extended Statistics Support to ORT Perf Test Tool

Summary
Add support for a new -es command-line flag to the ORT Perf test tool that enables extended latency statistics output i.e., P75 latency, mean latency, and standard deviation. The -es flag requires -s to be enabled first — it extends the existing statistics output.
Additionally, updated README documentation to reflect the new -es flag and corrected the -s flag description to accurately list the percentiles it reports (P50, P90, P95, P99, P999). Also fixed a typo in the sample output (se -> sec).

Points to note

  • -es is optional and depends on -s: Extended stats are not computed or shown unless both flags are enabled, keeping the default output unchanged.
  • Extended stats positioned for readability: P75 is inserted between P50 and P90 (chronological order), and mean/standard deviation appear after P999 as summary metrics.

Files

  • Modified: onnxruntime/test/perftest/README.md, onnxruntime/test/perftest/command_args_parser.cc, onnxruntime/test/perftest/performance_runner.cc, onnxruntime/test/perftest/performance_runner.h, onnxruntime/test/perftest/test_configuration.h

Validation Results

  • Only -s flag is specified -> Outputs only existing statistics output
image
  • Both -s and -es flag is specified ->Outputs both existing and extended statistics output
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant