From ea0358611254a68148f470640a61e3387fa7e869 Mon Sep 17 00:00:00 2001 From: ColumbusLabs <287001685+ColumbusLabs@users.noreply.github.com> Date: Fri, 24 Jul 2026 22:58:33 -0400 Subject: [PATCH] docs: explain benchmark summary statistics --- README.md | 12 ++++++++++++ doc/hyperfine.1 | 12 ++++++++++++ 2 files changed, 24 insertions(+) diff --git a/README.md b/README.md index ff6020064..77f18db66 100644 --- a/README.md +++ b/README.md @@ -44,6 +44,18 @@ If you want to compare the runtimes of different programs, you can pass multiple hyperfine 'hexdump file' 'xxd file' ``` +### Interpreting benchmark results + +The summary for each command combines measurements from all timed benchmark runs. +`Time (mean ± σ)` reports the mean elapsed (wall-clock) time and the corrected +sample [standard deviation](https://en.wikipedia.org/wiki/Standard_deviation) `σ` +of those times. A smaller `σ` means that the measured runtimes were more tightly +clustered around the mean. + +`User` and `System` report the mean CPU time per run spent in user mode and +kernel/system mode, respectively. They are CPU-time components and do not need +to add up to the elapsed time. + ### Warmup runs and preparation commands For programs that perform a lot of disk I/O, the benchmarking results can be heavily influenced diff --git a/doc/hyperfine.1 b/doc/hyperfine.1 index 449b470f2..e6c944bbb 100644 --- a/doc/hyperfine.1 +++ b/doc/hyperfine.1 @@ -81,6 +81,18 @@ A command\-line benchmarking tool which includes: .RS * Parameterized benchmarks (e.g. vary the number of threads) .RE +.SH BENCHMARK OUTPUT +The summary for each command combines measurements from all timed benchmark +runs. +.PP +\fBTime (mean \(+- sigma)\fP reports the mean elapsed (wall\-clock) time and +the corrected sample standard deviation, sigma, of those times. A smaller +standard deviation means that the measured runtimes were more tightly +clustered around the mean. +.PP +\fBUser\fP and \fBSystem\fP report the mean CPU time per run spent in user +mode and kernel/system mode, respectively. They are CPU\-time components and +do not need to add up to the elapsed time. .SH OPTIONS .HP \fB\-w\fR, \fB\-\-warmup\fR \fINUM\fP