Skip to content

expose latencies with the speedup in OnnxDiscrepancyCheck#2534

Open
xadupre wants to merge 5 commits into
mainfrom
xadupre/lat
Open

expose latencies with the speedup in OnnxDiscrepancyCheck#2534
xadupre wants to merge 5 commits into
mainfrom
xadupre/lat

Conversation

@xadupre

@xadupre xadupre commented Jun 22, 2026

Copy link
Copy Markdown
Member

Describe your changes

Speedup is not enough as a metric, latency must be added to understand which side is changing.

@xadupre

xadupre commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

@copilot extend the existing unit test to cover the changes

Copilot AI commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

@copilot extend the existing unit test to cover the changes

Done in 72c5d4e: I extended the existing discrepancy-check unit tests to cover the new latency-and-speedup return path from _measure_speedup.

@xadupre xadupre requested a review from Copilot June 22, 2026 12:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.

This PR enhances OnnxDiscrepancyCheck speed measurements by exposing the underlying per-iteration latencies (PyTorch and ONNX) in addition to the speedup ratio, making it easier to interpret which side changed.

Changes:

  • Updates _measure_speedup to return (pytorch_latency_s, onnx_latency_s, speedup) instead of only speedup.
  • Stores pytorch_latency_s, onnx_latency_s, and speedup into the results dict when timing is measured.
  • Adds a unit test verifying the new _measure_speedup return tuple.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
olive/passes/onnx/discrepancy_check.py Returns and records both latencies alongside speedup in results.
test/passes/onnx/test_discrepancy_check.py Adds coverage for the new tuple return contract of _measure_speedup.

Comment thread olive/passes/onnx/discrepancy_check.py
Comment thread olive/passes/onnx/discrepancy_check.py
Comment thread test/passes/onnx/test_discrepancy_check.py
@xadupre xadupre marked this pull request as ready for review June 24, 2026 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants