Repository navigation
harden parse boundary, C13 diagnostics, and DBI/GSI error handling - #53
Conversation
…as diagnose() already promises
…s stream Review follow-up on the audit fixes. PublicsStream.parse now raises plain PdbError rather than MsfError -- the publics stream is not an MSF container, and the exception should not claim to be one. The DBI substream bound checks carry the reason a header that lies about the bounds raises outright while in-substream damage degrades gracefully, pinned by a test that cuts a DBI stream short. The C13 re-decode in diagnose() carries its measured cost (18ms of 180ms on the 3 MB sqlite fixture) and why a header-only twin of the block walk was rejected.
… padding as padding Two of the checks this branch added were sharper than the contract allows. A substream size past the end of the DBI stream raised MsfError out of PDB.open(), which turned a file missing the last eight bytes of its debug header -- every function recoverable -- into one that would not open, with no diagnostic to say why; the negative size still raises, because the next offset then goes backwards and every later slice aliases bytes that were never substream data, but the overrun is now read as far as the stream goes, recorded in DbiStream.substream_overrun, and reported by diagnose() as Diagnostics.dbi_overrun with a warning naming the substream. And a C13 region ending in a few zero bytes -- a producer rounding it up -- was reported as lines going missing when none did; fewer than eight bytes cannot be a header, and only bytes with something in them are evidence of a cut. The inline-site coordinate change is not observable on a well-formed stream, and its test now says so and adds the one input where the two spaces disagree, a damaged End two bytes past the site. The CLI's diagnose gets a test for the C13 warning it prints. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017jQKALSgFN7VecWTqzWSmD
…anielplohmann#59 and the dependabot bumps Fill in the fuzz and timing results, note the follow-up branches, and summarise the review in the log. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017jQKALSgFN7VecWTqzWSmD
P3-5 holds up, and it should not block the stackChecked this while sequencing the queue against #60/#61/#62, because the stack rewrites the same code. Two separate answers. The fix is correct
So the compensation has to match the strip: 4 when a signature was removed, 0 when it was not. A conditional strip paired with an unconditional Confirmed against the 46-file bag (39 readable, 23478 module symbol streams). For every proc record I tested both readings of 757682 procs, no ambiguous cases, no misses. The docstring is right. But the branch it guards never fires on real inputEvery module stream in the corpus is C13-signed. And when the unstripped path is taken, the body it returns begins with a non-C13 signature word that That reads as a fuzzer-facing robustness fix, which fits: Sequencing
Nothing here is superseded — Suggested order: #60 → #61 → #62 → this. Details and the conflict inventory in the plan on #62. Adjacent, not covered by this PRA C7 ( |
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md # purepdb/c13.py # purepdb/pdb.py # tests/test_diagnose.py # tests/test_inline.py
# Conflicts: # AGENTS.md # purepdb/pdb.py
… unknown Fixes #64. `PDB.module_symbol_bytes` stripped the CodeView signature only when it was `CV_SIGNATURE_C13`. A C7 (`0`) or C11 (`1`) signature stayed on the front of the buffer, and `codeview.iter_records` read that word as a record header - for C11, length 1 and kind 0 - so every record after it was walked at the wrong alignment, with no error and no warning. - All three signatures are recognised and stripped, as `llvm-pdbutil`'s module stream reader does. - A first word that is none of the three yields no symbols: the layout after an unknown signature is not known, and reading the word as a record is what produced the silent misparse. `Diagnostics.unrecognised_signatures` counts those modules by the value found, and `diagnose()` warns with it. - A stream too short to hold a signature is unchanged: its bytes go to the record walk, which reports them as a truncation. Two tests, both verified to fail under a mutation of the line they cover: `test_every_codeview_signature_is_stripped` (C7 and C11 fail when only C13 is recognised) and `test_an_unrecognised_signature_is_not_parsed_as_a_record`, whose unknown word is chosen so that, read as a header, it is a 16-byte `S_GPROC32`. Every golden fixture is C13-signed, so no golden count moves. This also settles #53's P3-5. The strip is no longer conditional on any path that returns records - a body is either the stream minus exactly four bytes, or empty - so the fixed `+ CV_SIGNATURE_SIZE` compensation already matches it. The one path returning an unstripped non-empty body is a stream shorter than four bytes, which cannot hold a record header and so yields no offsets at all. Merged with main carrying #60, #61 and #62; 636 passed, ruff and ty clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Status, since we crossed in flight: #60 ( Your 19:20 push picked up the stack but not #65, which landed a couple of minutes after it, so the P3-5 is now a no-op, as you predictedYour call on #62 was right, and it holds on the merged code. After #65, So the resolution is to drop P3-5 rather than port it: delete For what it is worth, the corpus said the same before #65 landed: across 23478 module streams every one is C13-signed, so the branch P3-5 guarded never fired on real input. It was correct and unreachable; #65 made it unnecessary. What is left conflicting
Happy to do the rebase myself if you would rather hand it over, but you are clearly mid-flight on it, so I will keep off the branch unless you say otherwise. |
With the C7/C11 signature change on main, every path that returns module records has stripped exactly four bytes, so the conditional signature offset has nothing left to correct. _module_symbol_bytes_and_offset and its test are removed, and the inline-site helpers keep main's fixed CV_SIGNATURE_SIZE. The rest of the branch is unchanged.
|
Merged main in and dropped P3-5 like you said: |
Dev-group pin only; `ruff` governs the `lint and type check` CI job and `make lint`. No `.pre-commit-config.yaml` in this repository, so there is no `ruff-pre-commit` rev that has to move with it. Verified against this main rather than the one the PR was opened on: 0.16.6 reports `All checks passed!` over the whole tree, including the roughly 4400 lines that landed in #60, #61, #62, #65 and #53 after this PR was raised. `target-version` stays `py311`.
Dev-group pin only; `ty` governs the type half of the `lint and type check` CI job and `make lint`. Dependabot rebased this onto the ruff bump and re-resolved the target from 0.0.79 to 0.0.80 in the process, so it was re-verified at the version actually pinned here: 0.0.80 reports `All checks passed!` over the whole tree, including the roughly 4400 lines that landed in #60, #61, #62, #65 and #53 after this PR was raised. `ty` is still 0.0.x, so it is the pin in this group most likely to surface a new diagnostic on a bump.
Both version strings to 0.6.0, `## [Unreleased]` renamed to `## [0.6.0] - 2026-09-15` with a fresh empty one above it, and the compare links moved on. No source changes. First release cut through `publish-release.yml`, which has existed since #59 without a tag ever exercising it. 23 entries from five merged PRs plus the release machinery and three dependency bumps: the corpus-audit fixes (#60), the single-pass record walk (#61), the mapped `PDB.open` and streamed inline-site placement (#62), the C7/C11 signature handling (#65) and the parse-boundary hardening (#53). Minor rather than patch: the `Fixed` entry for the fused `ChangeCodeLengthAndCodeOffset` annotation says **This moves addresses** -- the second and later ranges of an inline site built from that opcode now start earlier than 0.5.0 reported, on every rust-lld and clang output. The first range, and every single-range site, are unchanged. `CHANGELOG.md`'s preamble reserves that for a breaking change and requires it to be stated. Verified before tagging: `release_guard.py --tag v0.6.0` extracts the section and reports prerelease=false, and refuses v0.7.0 naming both version strings; 645 passed, ruff and ty clean on the new pins; `python -m build` plus `twine check --strict` pass on both artefacts; and the wheel, installed into a clean venv and imported with `python -P`, reports 0.6.0 from site-packages with `py.typed` present and runs the entry point. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This PR addresses the findings from the correctness audit across purepdb:
Summary of Changes
PDB.named_streams()withself.msf.is_valid_stream(STREAM_PDB_INFO)and caughtMsfErrorto return{}as documented, ensuringdiagnose(),string_table(), andlines()produce empty results and clear diagnostic explanations rather than raising an unhandled exception.ValueErrorwith plainPdbErrorinPublicsStream.parse()when header or address map is truncated, preserving the invariant thatPdbErroris the only exception crossing public boundaries. PlainPdbErrorrather thanMsfError: the publics stream is not an MSF container.DbiStream.parse()that each substream size is non-negative and does not overrun the stream buffer, raisingMsfErrorwith descriptive context if corrupted.C13Truncationdataclass inpurepdb.c13(exported in__all__), collected truncations initer_subsectionsandparse_lines, surfacedDiagnostics.c13_truncations, and added an explanatory warning toDiagnostics.warnings.pdb.publics_stream()toexercise()intools/fuzz.py.CV_SIGNATURE_SIZEalready matches and the conditional helper had nothing left to correct.Reader.aligndead code inpurepdb/reader.py, and added docstrings toContributionMap.__len__andSectionContribution.containsinpurepdb/dbi.py.CHANGELOG.mdunder## [Unreleased].Design decisions (deliberate, and written down in the code)
DbiStream.parse()raiseMsfErrorfor a negative size (which would slice backwards, aliasing bytes that were never substream data) and for a size past the stream end (the header's arithmetic disagreeing with the container). The graceful paths —module_list_stopped_at,codeview.Truncation— still cover damage inside substreams whose bounds the header describes correctly. The rationale lives in a comment at_check_substream, and a test cuts a valid DBI stream short to pin the choice.diagnose()re-decodes everyDEBUG_S_LINESpayload and discards the entries. Measured at 18 ms of the 180 msdiagnose()costs on the 3 MB sqlite fixture. Deliberate, for the same reasoncount_malformed_recordsruns twice per module:parse_linesis the only implementation of the block walk, and a header-only twin kept beside it would drift.lines()cannot be reused because it is a generator a caller may never consume, and it resolves file names this loop does not want. The reason and the number live in a comment at the call site.Skipped Findings
Gate Verification
make test PYTHON=.venv/bin/pythontests/test_groundtruth.pyremain unchanged.make lint PYTHON=.venv/bin/pythonmake fuzz PYTHON=.venv/bin/python