Skip to content

python-math #35: feat(delphi): certify battery speedup — engine-scoped py cache + parallel entries (Phase 0) - #2664

Draft
jucor wants to merge 1 commit into
spr/edge/efb914d7from
spr/edge/31c19ea5
Draft

python-math #35: feat(delphi): certify battery speedup — engine-scoped py cache + parallel entries (Phase 0)#2664
jucor wants to merge 1 commit into
spr/edge/efb914d7from
spr/edge/31c19ea5

Conversation

@jucor

@jucor jucor commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Why

GOAL_CUTOVER_READY.md Phase 0 (Julien ruling 2026-07-27): the certification battery re-ran the full ~36-minute Python re-replay after ANY polismath edit, blocking every code change.

What

  • The Python recording cache key is now the ENGINE-scoped tree hash: pure-harness files (certify.py, poller_equiv.py, prodclone.py, shard_bench.py, poller/**) are excluded from the hash — none is reachable from the replay subprocess import graph (scripts/replay_driver.py -> driver/schedule/real_data/store/stepcompare/types -> engine). Manifest key renamed py_tree_sha256 -> engine_tree_sha256; the resulting one-time invalidation of all 20 Python recordings doubled as the timed A/B run.
  • run_battery(workers=N) fans the per-entry heavy work (driver subprocesses + hash-first compare) across threads; the ledger fold stays strictly serial in battery order, so report + ledger are bit-identical to workers=1 (pinned by test). Step-verdict cache writes are atomic (tmp + os.replace). CLI --workers defaults to 6.

Measured A/B on the 10-core host

  • First pass: 19m17s wall vs ~36m serial (the long pole: pakistan:uniform8 alone is ~18m).
  • Cached pass: 22s.
  • Harness-only edit: 2m9s with ZERO re-replays (previously: the full 36-minute re-replay).

Testing

TDD: 7 RED -> GREEN; replay_harness 464 passed / 5 skipped (baseline 456/5 plus exactly the 8 new tests). Battery on this tree: 20/20 MATCH x2 (post-invalidation re-record + cached pass). The Phase 1 engine_mode inventory (18 branch sites classified DELETE/PARK/KEEP) is journaled.

commit-id:31c19ea5


Stack:


⚠️ Part of a stack created by spr. Do not merge manually using the UI - doing so may have unexpected results.

This was referenced Jul 27, 2026
This was referenced Jul 27, 2026
@jucor
jucor force-pushed the spr/edge/efb914d7 branch from a25906e to ee94641 Compare July 28, 2026 00:11
@jucor
jucor force-pushed the spr/edge/31c19ea5 branch from bc79e36 to 7193da4 Compare July 28, 2026 00:11
@jucor jucor changed the title feat(delphi): certify battery speedup — engine-scoped py cache + parallel entries (Phase 0) python-math #35: feat(delphi): certify battery speedup — engine-scoped py cache + parallel entries (Phase 0) Jul 28, 2026
…d py cache + parallel entries (Phase 0)

## Why

`GOAL_CUTOVER_READY.md` Phase 0 (Julien ruling 2026-07-27): the certification battery re-ran the full ~36-minute Python re-replay after ANY `polismath` edit, blocking every code change.

## What

- The Python recording cache key is now the ENGINE-scoped tree hash: pure-harness files (`certify.py`, `poller_equiv.py`, `prodclone.py`, `shard_bench.py`, `poller/**`) are excluded from the hash — none is reachable from the replay subprocess import graph (`scripts/replay_driver.py` -> driver/schedule/real_data/store/stepcompare/types -> engine). Manifest key renamed `py_tree_sha256` -> `engine_tree_sha256`; the resulting one-time invalidation of all 20 Python recordings doubled as the timed A/B run.
- `run_battery(workers=N)` fans the per-entry heavy work (driver subprocesses + hash-first compare) across threads; the ledger fold stays strictly serial in battery order, so report + ledger are bit-identical to `workers=1` (pinned by test). Step-verdict cache writes are atomic (tmp + `os.replace`). CLI `--workers` defaults to 6.

## Measured A/B on the 10-core host

- First pass: 19m17s wall vs ~36m serial (the long pole: `pakistan:uniform8` alone is ~18m).
- Cached pass: 22s.
- Harness-only edit: 2m9s with ZERO re-replays (previously: the full 36-minute re-replay).

## Testing

TDD: 7 RED -> GREEN; replay_harness 464 passed / 5 skipped (baseline 456/5 plus exactly the 8 new tests). Battery on this tree: 20/20 MATCH x2 (post-invalidation re-record + cached pass). The Phase 1 `engine_mode` inventory (18 branch sites classified DELETE/PARK/KEEP) is journaled.

commit-id:31c19ea5
@jucor
jucor force-pushed the spr/edge/31c19ea5 branch from 7193da4 to 5f7b47f Compare July 28, 2026 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant