Repository navigation
[Performance #727] Add local-only, parity-gated per-workload scorecards - #1473
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Partial implementation of #727, not a closeout of the epic. Replaces the empty
tools/perf_scorecard/main.flowstub with a classifier for existing localbench_harnessJSON results.It enforces verified semantic parity, timing scope, minimum repetition count and dispersion bounds before scoring. Speed wins need a >=1.05x advantage; equal-performance memory wins use a >=5% lower RSS threshold. Flow/native tax is separately retained rather than averaged away. Missing CPython baselines, noisy data, unresolved parity and empty reports never become benchmark wins. A strict mode returns nonzero for any loss, tax or unscored row.
Includes synthetic local tool tests and documentation:
tests/scripts/perf_scorecard.flow,docs/project/performance-scorecards.md.Validation
Static structure inspection: delimiters and string literal pairing passed for both Flow files. Tool execution and end-to-end compiler tests not run (local checkout/toolchain unavailable in this session). This PR is intentionally draft until local gates run.
Run locally:
As required by #727, no hosted workflows/CI were inspected or relied upon. No performance speedup is claimed. Other #727 tasks (and #736/#766/#679/#665/#666) are not implemented by this PR.