Skip to content

report.md

Latest commit

 

History

History
133 lines (94 loc) · 3.35 KB

report.md

File metadata and controls

133 lines (94 loc) · 3.35 KB
Raw

(cli-report)=

report - Historical Analysis

Analyze benchmark results over time, generate platform rankings, and detect performance regressions. Results are stored in a local SQLite database at ~/.benchbox/results.db.

Basic Syntax

benchbox report <subcommand> [OPTIONS]

Subcommands

rankings - Platform Rankings

Generate platform rankings for a benchmark based on historical results.

benchbox report rankings --benchmark <name> --scale-factor <sf> [OPTIONS]

Required:

  • --benchmark, -b TEXT: Benchmark name (e.g., TPC-H, TPC-DS)
  • --scale-factor, -s FLOAT: Scale factor to rank

Optional:

  • --metric, -m [geometric_mean|power_at_size|cost_efficiency]: Metric to rank by (default: geometric_mean)
  • --min-samples INT: Minimum samples required to include a platform (default: 1)
  • --lookback-days INT: Consider results from the last N days (default: 90)
  • --db-path PATH: Path to result database
benchbox report rankings --benchmark TPC-H --scale-factor 1
benchbox report rankings -b TPC-DS -s 10 --metric power_at_size

trends - Performance Trends

Show performance trends for a platform over time, highlighting regressions.

benchbox report trends --platform <name> --benchmark <name> --scale-factor <sf> [OPTIONS]

Required:

  • --platform, -p TEXT: Platform name
  • --benchmark, -b TEXT: Benchmark name
  • --scale-factor, -s FLOAT: Scale factor

Optional:

  • --periods INT: Number of periods to analyze (default: 6)
  • --period-days INT: Days per period (default: 30)
  • --db-path PATH: Path to result database
benchbox report trends --platform DuckDB --benchmark TPC-H --scale-factor 1

regressions - Detect Regressions

Scan all platforms and benchmarks for significant performance slowdowns.

benchbox report regressions [OPTIONS]

Optional:

  • --threshold FLOAT: Regression threshold percentage (default: 10.0)
  • --lookback-days INT: Comparison period in days (default: 30)
  • --db-path PATH: Path to result database
benchbox report regressions
benchbox report regressions --threshold 15 --lookback-days 14

import - Import Results

Import benchmark result files into the historical database.

benchbox report import <directory> [OPTIONS]

Required:

  • DIRECTORY: Path to directory containing result JSON files

Optional:

  • --pattern TEXT: Glob pattern for result files (default: **/*.json)
  • --db-path PATH: Path to result database
benchbox report import benchmark_runs/results/
benchbox report import ./results --pattern "*.json"

stats - Database Statistics

Show summary statistics for the result database.

benchbox report stats [OPTIONS]

Optional:

  • --db-path PATH: Path to result database

list - List Results

List stored benchmark results with optional filters.

benchbox report list [OPTIONS]

Optional:

  • --platform, -p TEXT: Filter by platform
  • --benchmark, -b TEXT: Filter by benchmark
  • --limit INT: Maximum results to show (default: 20)
  • --db-path PATH: Path to result database
benchbox report list
benchbox report list --platform DuckDB --limit 10

Related

  • results - View and export recent benchmark results
  • visualize - Generate charts from results