Repository navigation
Conversation
diagnose() walks every module stream to count record kinds, then called module_procs(), which re-reads and re-slices each of those same streams to extract the procedures it counts. Publics were worse: the symbol-record stream is already read for the kind counts and the truncation check, and public_symbols() read it a second time and then ordered the result through the publics hash -- an ordering the len() around it discards. Both counts now come from bytes the function is already holding. The totals are unchanged on all seven fixtures. Not a measurable speedup, and worth saying so rather than leaving the next reader to assume one: diagnose() is dominated by the record walks themselves -- count_malformed_records alone runs twice per module, once for the general count and once for S_INLINESITE -- against which the duplicated reads removed here do not register.
…anielplohmann#59 and the dependabot bumps Fill in the fuzz and timing results, note the follow-up branches, and summarise the review in the log. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017jQKALSgFN7VecWTqzWSmD
|
Superseded by #61, which landed the same ground with measurements behind it. Suggest closing. This PR's contribution is removing In #61 neither call exists. The framing here was right, and honest about its limits - "not a measurable speedup, and worth saying so rather than leaving the next reader to assume one". #61 measured the same walks at 3-8x once they were done once instead of three and four times. Nothing is lost by closing: the behaviour is in #61, and this branch is still on the pre-release-workflow base and conflicts with both #62 and #53 in |
…ery listing, output unchanged Eleven commits on top of #60. Output snapshots are byte-identical to the fixes branch on all 7 fixtures, 439 corpus files and node.pdb; every commit passed the suite, `ruff`, `ty` and a 2000-input fuzz pass. Full tables and dead ends in `docs/audit/perf.md`. Wall seconds, best of N, 4-core Xeon, CPython 3.11: | file | op | before | after | speedup | |---|---|---:|---:|---:| | sqlite x64 (3 MB) | all listings | 1.19 | 0.30 | 4.0x | | | diagnose | 0.55 | 0.07 | 8.1x | | python314.pdb (21 MB, MSVC PGO) | functions | 1.77 | 0.27 | 6.6x | | | diagnose | 6.69 | 0.88 | 7.6x | | ntkrnlmp.pdb (8.5 MB, stripped) | diagnose | 1.36 | 0.22 | 6.3x | | node.pdb (355 MB) | functions | 21.6 | 3.5 | 6.2x | | | all listings | 193 | 61 | 3.2x | | xul.pdb (1.93 GB) | diagnose | 954 (11.4 GB RSS) | 152 (7.0 GB) | 6.3x | What changed: - `iter_records`: one `unpack_from` per header and a `kinds=` filter, so an extractor after one kind pays a slice and a `RawRecord` only for the records it wants. This loop was 73% of all time. - `Reader.u16/u32` via `Struct.unpack_from` at the cursor; a fixed-portion `Struct` per named record kind, with one `find` for the name. - `codeview.survey_records`: `diagnose()` and `functions()` walk each module stream once instead of three and four times, and `diagnose()` never builds the inline listing. This also subsumes #52, which is closed. - `msf._read_blocks` reads contiguous block runs as single slices. - Inline-site annotations walked with an integer cursor; the IPI counted positionally without a `RawRecord` per record; `Line`, `LineEntry`, `RawRecord`, `InlineFunction` slotted; per-file name and per-segment base hoisted out of the line loop. Public API unchanged. Rebased onto main after #60 squash-landed; the rebased tree is byte-identical to the pre-rebase branch, so the squash carried #60 verbatim. 627 passed, ruff clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Summary
diagnose()walks every module stream to count record kinds, then calledmodule_procs(), which re-read and re-sliced each of those same streams toextract the procedures it counts. Publics were worse: the symbol-record stream
is already read for the kind counts and the truncation check, and
public_symbols()read it a second time and then ordered the result throughthe publics hash — an ordering the
len()around it discards.Both counts now come from bytes the function is already holding. The totals
are unchanged on all seven fixtures.
Not a measurable speedup, and worth saying so rather than leaving the next
reader to assume one:
diagnose()is dominated by the record walksthemselves —
count_malformed_recordsalone runs twice per module, once forthe general count and once for
S_INLINESITE— against which the duplicatedreads removed here do not register.
Testing
pytest: 588 passedruff checkandty check: cleantools/fuzz.py --iterations 2000 --seed 0: no exception escaped