ci: stabilize ratchet digests across the Python matrix

[yisding]

Why the Dependabot PRs are failing

Two unrelated causes, both ours rather than the bumps':

#856 (cryptography 49→50) — Validate Quick (3.14) only. The reviewed manifests in tests/ratchets/ pin an ast_hash per classified source site, and CI verifies the same manifests on 3.11 and 3.14. Those digests came from ast.dump, whose output is not stable across that range:

• 3.12 added PEP 695 type_params to FunctionDef/AsyncFunctionDef/ClassDef
• 3.13 started omitting fields equal to their default, so args=[] / keywords=[] vanish

So every hash for the affected nodes shifted on 3.14 and five ratchets failed with a wall of "removed or structurally changed" sites that no commit caused. This is not specific to Dependabot — Validate Quick (3.14) has been red on main since #808 and fails any PR that touches nothing related.

#855 (aiohttp in /perf/framework_environments/pipecat) — nearly every job. Those competitor environments are deliberately frozen: uv lock --exclude-newer records the cutoff in the lock's [options] table so the benchmark resolves the same transitive snapshot each run. Dependabot regenerated the lock without the flag, dropping the table. The cutoff was read at import with ["options"], so it raised KeyError: 'options' during collection — the whole perf module errored, 0 of its tests ran, and the lane reported an opaque missing key. The literal-pinned cutoff assertion that exists to catch exactly this drift never got to run.

What this changes

tests/ratchets/_ast_digest.py pins one canonical serialization instead of inheriting the stdlib's. It reproduces 3.11's ast.dump(annotate_fields=True, include_attributes=False) byte for byte — verified over 1,379,718 nodes across src/ and tests/ — so baselines reviewed against that output stay valid and no manifest changes were needed. The digest is now identical on 3.11, 3.12, 3.13, and 3.14. test_ast_digest.py pins the canonical form and digest so a revert to ast.dump fails loudly on whichever interpreter regresses.

I enumerated every AST field difference across 3.11–3.14 programmatically rather than assuming: type_params on those three node types is the only one.

perf/bench_framework_latency.py reads the cutoff leniently and resolves it at use. A stripped lock now fails as three focused tests naming the lock and the exact re-pin command, while the module's other 13 tests still run:

BEFORE: KeyError: 'options' — collection interrupted, 0 tests run
AFTER:  13 passed, 3 failed — "perf/framework_environments/pipecat/uv.lock has no
        [options] exclude-newer ... re-pin it with `uv lock --project
        perf/framework_environments/pipecat --exclude-newer <cutoff>`"

Verification

Full credential-free suite, run with each interpreter:

	3.11	3.14
before	8472 passed	5 failed (ratchets)
after	8472 passed	8472 passed

pre-commit run --all-files (incl. ruff, import-linter, zizmor), mypy src/easycat, and ruff format --check all pass. The KeyError path was reproduced by stripping [options] from the real pipecat lock, then restored.

Follow-ups for the two open PRs

• build(deps): bump cryptography from 49.0.0 to 50.0.0 #856 should go green once this lands.
• build(deps): bump aiohttp from 3.14.1 to 3.14.3 in /perf/framework_environments/pipecat #855 should be closed, not merged. It unpins a deliberately frozen benchmark snapshot; after this change it fails with a message saying so instead of a KeyError. The same drift landed once before in build(deps): bump pipecat-ai from 1.0.0 to 1.4.0 in /perf/framework_environments/pipecat #591 and needed manual repair (perf: repair locked benchmark worker, perf: stabilize framework locks).
• Dependabot keeps proposing bumps to perf/framework_environments/* despite directory: "/". I did not touch dependabot.yml: GitHub documents glob support for directories but documents no negation/exclusion syntax, so any exclusion I added would be unverifiable in CI and might silently do nothing. Worth a follow-up that can be validated against a live Dependabot run.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Benchmark configuration now handles missing lock metadata gracefully.
  • Missing lock cutoff information produces a clear error with instructions for regenerating the lock data.
  • Worker setup is prevented when required lock metadata is unavailable.

• Refactor

  • AST-based consistency checks now use a shared, interpreter-stable fingerprinting method, improving reliability across Python versions.

• Tests

  • Added coverage for missing lock metadata and stable AST serialization and fingerprint behavior.

[coderabbitai]

Warning

Review limit reached

@yisding, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 51 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 8c031955-46d3-41ce-aa61-f6b9be898586

📥 Commits

Reviewing files that changed from the base of the PR and between 0afec15 and 93a7c9d.

📒 Files selected for processing (1)

• tests/perf/test_framework_latency_benchmark.py

📝 Walkthrough

Walkthrough

The benchmark now validates missing lock cutoff metadata and shares competitor definitions. Ratchet inventories now use centralized, interpreter-stable AST serialization and digest helpers.

Changes

Benchmark lock validation

Layer / File(s)	Summary
Lock cutoff contract perf/bench_framework_latency.py, tests/perf/test_framework_latency_benchmark.py	Lock cutoff loading accepts missing values. Validation raises a descriptive re-pinning error. Tests cover missing metadata and rejected worker specifications.
Validated benchmark operations perf/bench_framework_latency.py	Dependency pin loading, worker commands, and lock metadata generation use COMPETITORS and require_lock_exclude_newer.

Shared AST digest ratchet

Layer / File(s)	Summary
Canonical AST digest helpers tests/ratchets/_ast_digest.py, tests/ratchets/test_ast_digest.py	Canonical AST serialization omits version-dependent and unset optional fields. Tests verify stable dumps, expected digests, and structural differences.
Inventory digest migration tests/ratchets/_*inventory.py	Ratchet inventories replace local hashing and raw AST dumps with shared helpers.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

• yisding/easycat#468: Updates the same benchmark module and tests.
• yisding/easycat#469: Extends the benchmark module introduced by this related PR.

Suggested reviewers: charliecreates

Poem

A rabbit checks each lock with care,
And finds the cutoff hiding there.
ASTs now march in ordered rows,
Their hashes keep their shape as code grows.
Shared helpers guide the way.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the primary change: stabilizing ratchet digests across supported Python versions.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch worktree-ci-python-version-stability

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@perf/bench_framework_latency.py`:
- Line 50: Remove synchronous lock-file I/O from perf/bench_framework_latency.py
by replacing import-time cache construction and Path.read_text() usage with
explicit asynchronous lock initialization, then make every lock-dependent caller
await that initialization. In tests/perf/test_framework_latency_benchmark.py at
line 149, convert the test to async and await the fixture file write.

In `@tests/perf/test_framework_latency_benchmark.py`:
- Around line 155-170: The test covers require_lock_exclude_newer and
worker_specs under the missing cutoff condition, but does not verify the
validation in _lock_metadata itself. Under the same monkeypatch setup
(LOCK_EXCLUDE_NEWER_BY_FRAMEWORK with "pipecat" set to None), add an assertion
that _lock_metadata raises RuntimeError when invoked. This ensures the
report-generation validation in _lock_metadata is tested and prevents future
refactoring from removing it undetected.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 38bed6ae-00ed-4822-9d18-d9cb024ffefd

📥 Commits

Reviewing files that changed from the base of the PR and between e2d6b90 and 0afec15.

📒 Files selected for processing (10)

• perf/bench_framework_latency.py
• tests/perf/test_framework_latency_benchmark.py
• tests/ratchets/_ast_digest.py
• tests/ratchets/_pause_generation_inventory.py
• tests/ratchets/_source_inventory.py
• tests/ratchets/_teardown_budget_inventory.py
• tests/ratchets/_turn_commit_inventory.py
• tests/ratchets/_turn_lifecycle_inventory.py
• tests/ratchets/_turn_predicate_inventory.py
• tests/ratchets/test_ast_digest.py