Skip to content

Separate the comparison engine from the reporter#90

Merged
KaykCaputo merged 1 commit into
KaykCaputo:masterfrom
AnshBajpai05:refactor/pure-compare-display-split
Jul 25, 2026
Merged

Separate the comparison engine from the reporter#90
KaykCaputo merged 1 commit into
KaykCaputo:masterfrom
AnshBajpai05:refactor/pure-compare-display-split

Conversation

@AnshBajpai05

Copy link
Copy Markdown
Contributor

Second slice from #80 — the one you said you were most excited about. Pure refactor: no new flags, no behavior change, identical output.

What changes

compare_traces() used to compute and print at the same time, which meant any new output format would have to re-implement the comparison logic (or re-parse printed text). Now:

  • compare.py is pure — no print, no side effects. It returns a richer ComparisonData:

    ComparisonData(
        deltas,             # every function, with a status
        regressions,        # unchanged: List[RegressionData]
        improvements,       # functions that got faster past the threshold
        new_functions,      # names only
        removed_functions,
        threshold,          # what it was compared against
        has_regression,     # unchanged
    )

    Each FunctionDelta carries name, status, old_time, new_time, percent. Statuses are module-level constants: regression / improvement / stable / new / removed / no_signal / no_baseline — the states the old code expressed only as printed strings.

  • display.py is new and owns all rich printing: print_comparison(comparison, only_regressions=False). The show_only_regressions argument moved from the comparison function to the display function, which is where filtering actually belongs.

  • cli.py now does compare_traces(...)print_comparison(...). Both call sites (--compare and baseline compare) updated.

Why this ordering matters

This is the enabler for the Markdown reporter (next PR): with comparison results as data, a reporter is ~100 lines consuming ComparisonData, and the same is true for anything after it — JUnit, the HTML comparison column, whatever comes later. None of them need to know how a regression is decided.

RegressionData and has_regression keep their shape, so any external consumer of the current API keeps working.

Verification

  • Existing behavior tests were split by concern: test_compare.py now asserts on returned data (including a test that comparison produces no output at all), and the printing assertions moved to a new test_display.py — same expectations, same strings, just pointed at the display layer.
  • Added a format-fidelity test that pins the header, the arrow line, and the (+N.NN%) formatting.
  • Manually diffed real --compare output before and after the refactor: identical, down to blank lines and colors.
  • Full suite green (113 passing locally; the one failure on my machine is the pre-existing test_symlinked_user_code, which needs Windows symlink privileges and passes on CI).

🤖 Generated with Claude Code

compare_traces() computed and printed at once, so any new output
format had to re-implement comparison logic. It is now pure and
returns a richer ComparisonData: per-function deltas with explicit
statuses (regression / improvement / stable / new / removed /
no_signal / no_baseline), plus improvements, new/removed function
lists and the threshold used.

All rich printing moves to a new display.py (print_comparison),
which also takes over the only-regressions filtering — filtering is
a display concern. cli.py wires compare -> display at both call
sites.

No new flags, no behavior change: RegressionData and has_regression
keep their shape and the terminal output is byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@KaykCaputo KaykCaputo left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, just as i imagined

@KaykCaputo
KaykCaputo merged commit f41d6ca into KaykCaputo:master Jul 25, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants