Skip to content

fix(common): order a pre-release version when importing a report - #345

Merged
danielplohmann merged 1 commit into
masterfrom
fix/prerelease-version-gate
Sep 14, 2026
Merged

danielplohmann merged 1 commit into
masterfrom
fix/prerelease-version-gate

Conversation

@danielplohmann

Copy link
Copy Markdown
Owner

Found while setting up the first rehearsal of #343's release workflow, which needs a v4.8.0rc1 tag and therefore a pre-release string in __version__.

The defect

SmdaFunction.fromDict decides whether a report's stored pic_hash can be trusted by comparing its version against the per-architecture escape-version constants. The comparison is gated on a regex:

if version and re.fullmatch(r"v?\d+(\.\d+)*", version):

No PEP 440 pre-release matches it:

4.7.0      matches: True
4.8.0rc1   matches: False
4.8.0a1    matches: False

A non-match falls to the else branch — the one commented "if we don't have valid version information, always recalculate". So a report written by the newest build in existence is handled as one too old to trust.

Measured on the bundled cutwail_xored report, importing the same dict twice with only smda_version changed:

smda_version getPicHash calls on import
4.8.0 0
4.8.0rc1 33 of 33 functions

The hashes come out identical, so nothing was ever wrong in the output — this is cost and classification, not corruption. The cost is proportional to function count, which is why it matters where reports are imported in bulk rather than one at a time.

It is not hypothetical now that #343 has landed: RELEASING.md prescribes pre-release tags with a matching __version__ across the ecosystem, so any rc ever shipped produces reports that every later import re-hashes.

The fix

A report version becomes a comparable key of (release numbers, 1 for a release / 0 for a pre-release), so 4.8.0rc1 sorts directly below 4.8.0 rather than nowhere.

A pre-release of the version that changed an escaper still recomputes. 4.3.5rc1 precedes 4.3.5, and a candidate cut before the escaper change merged would otherwise be trusted on stored hashes it did not produce. Recomputing is the safe direction and costs only that one release's candidates.

The tuple shape is what keeps every already-ordered string ordering exactly as before, including the truncated form: 4.3 against a [4, 3, 5] gate still compares by the list prefix rule and still recomputes. Appending the flag to a flat list would have silently reversed that one.

A version this cannot order — 4.4.4-dev, MCRIT4IDA, 4..5 — is unchanged and still takes the recalculate-everything branch, which test_imported_report_with_a_suffixed_version_recalculates_instead_of_raising already pins.

Tests

Three cases in tests/testCommonModels.py, beside the existing escape-version ones, each over a1/b2/rc1:

  • a pre-release above the gate keeps the stored pic_hash;
  • a pre-release of the gate version recomputes;
  • the ordering table, which pins the shapes that already worked — 4.3.5, v4.3.5, 4.3.4, 4.3, 4.3.6, 5 — plus the five strings that must stay unorderable.

Each verified to fail under a mutation of the line it covers, and only that one:

mutation what fails
pre-release group dropped, i.e. master's regex the "above the gate keeps its hash" case, all three suffixes
pre-release flagged as a release the "of the gate recomputes" case, all three suffixes
flag dropped, comparing release numbers only the same case

Verification

  • python -m pytest tests/ → 2,171 passed, 1 skipped, 2,618 subtests
  • ruff check . and ruff format --check . → clean
  • ty → 276 diagnostics, which is this tree's own count for master to the number
  • After the fix, 4.8.0rc1 imports with 0 recalculations and 4.4.4-dev still with 33, hashes identical in both

No bundled fixture baseline moves: no report in the tree carries a pre-release version, so the only behaviour that changes is for versions nothing has produced yet.

🤖 Generated with Claude Code

fromDict gated its escape-version comparisons on v?\d+(\.\d+)*, which no
PEP 440 pre-release matches, so a report written by 4.8.0rc1 took the
"no valid version information" branch and had every pic_hash and nesting
depth recomputed on import. A pre-release now sorts directly below its
own release, so a candidate for the release that changed an escaper is
still recomputed while every later one is trusted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

📊 SMDA Performance Evaluation Benchmark Results

Generated on: 2026-09-14 12:52:25

Summary

Metric Result
Correctness ✅ PASS — 0 / 155 common file(s) differ
Determinism ✅ PASS — base 3 run(s), PR 3 run(s)
Verdict inconclusive — within run-to-run noise (±8.1%)
Median paired speedup -0.36% (95% CI [-1.48%, +1.28%])
Timing noise band ±8.1%

Per-side Timing Context (best of 3/3 runs per file)

Side Files Functions Median best time/file (s) Sum of best times (s) Throughput estimate (func/s)
base 155 173496 0.4815 202.85 ~855
pr 155 173496 0.5000 201.91 ~859

These context rows sum each file's best observed time across repeated runs; they are normalized comparison estimates, not single-run CI wall-clock times.

Paired per-file timing (positive speedup = PR faster):

Statistic Value
Files compared 155
Median paired speedup -0.36% (95% CI [-1.48%, +1.28%])
Mean speedup -5.48% (95% CI [-13.41%, +0.18%])
Std dev / IQR 44.69% / 10.97%
Wilcoxon signed-rank p 0.7908 (n=155)

ℹ️ base and PR are timed in the same job on the same runner, with their passes interleaved base/PR/PR/base/base/PR so that drift over the job cancels rather than landing on whichever side ran last. The noise band above is therefore run-to-run variation on one machine, which is what it is able to measure; a difference inside it is reported as inconclusive. Correctness and determinism are not timing-based and are unaffected either way.

Determinism (self-check across repeated runs)

Side Runs Files Deterministic Median timing CV
base 3 155 ✅ 8.1%
pr 3 155 ✅ 6.7%
Pairwise run matrix (individual run medians, diagnostic)

Diagnostic only: each row compares one raw PR run against one raw base run. The headline verdict above uses paired per-file best-of-runs timings.

Comparison Run PR Files Base Files Common Function Set Matches Med PR (s) Med Base (s) Med Diff (s) Speedup %
pr_0 vs base_0 155 155 155 155/155 0.6897s 0.7200s -0.0303s +4.21%
pr_0 vs base_1 155 155 155 155/155 0.6897s 0.6335s +0.0562s -8.87%
pr_0 vs base_2 155 155 155 155/155 0.6897s 0.6806s +0.0091s -1.33%
pr_1 vs base_0 155 155 155 155/155 0.7215s 0.7200s +0.0015s -0.21%
pr_1 vs base_1 155 155 155 155/155 0.7215s 0.6335s +0.0880s -13.90%
pr_1 vs base_2 155 155 155 155/155 0.7215s 0.6806s +0.0409s -6.01%
pr_2 vs base_0 155 155 155 155/155 0.6400s 0.7200s -0.0800s +11.11%
pr_2 vs base_1 155 155 155 155/155 0.6400s 0.6335s +0.0065s -1.03%
pr_2 vs base_2 155 155 155 155/155 0.6400s 0.6806s -0.0406s +5.97%

@danielplohmann
danielplohmann merged commit 7065e5f into master Sep 14, 2026
47 checks passed
@danielplohmann
danielplohmann deleted the fix/prerelease-version-gate branch September 14, 2026 12:57
@danielplohmann danielplohmann added this to the v4.8.0 milestone Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant