Skip to content

strip a C7 or C11 module-stream signature, and read nothing after one unknown - #65

Merged
danielplohmann merged 2 commits into
danielplohmann:mainfrom
r0ny123:module-stream-signature
Sep 15, 2026
Merged

danielplohmann merged 2 commits into
danielplohmann:mainfrom
r0ny123:module-stream-signature

Conversation

@r0ny123

@r0ny123 r0ny123 commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Fixes #64.

What changes

PDB.module_symbol_bytes stripped the CodeView signature only when it was CV_SIGNATURE_C13. A C7 (0) or C11 (1) signature stayed on the front of the buffer, and codeview.iter_records read that word as a record header — for C11, length 1 and kind 0 — so every record after it was walked at the wrong alignment, with no error and no warning.

  • All three signatures are recognised and stripped. llvm-pdbutil's module stream reader likewise consumes the first four bytes as the signature before reading records.
  • A first word that is none of the three yields no symbols: the layout after an unknown signature is not known, and reading the word as a record is what produced the silent misparse. Diagnostics.unrecognised_signatures counts those modules by the value found, and diagnose() warns with it, following the existing "read it, but say what was odd" pattern.
  • A stream too short to hold a signature is unchanged: its bytes go to the record walk, which reports them as a truncation.

Relationship to #53

#53's P3-5 makes the + CV_SIGNATURE_SIZE offset compensation conditional to match the conditional strip. After this change the strip is no longer conditional on any path that returns records — a body is either the stream minus exactly four bytes, or empty — so the fixed compensation already matches it. The two touch the same lines in pdb.py and will conflict; whichever lands second needs a small rebase, and P3-5 may reduce to a no-op on top of this.

Tests

  • test_every_codeview_signature_is_stripped — C7, C11 and C13 each read the same procedure with no warnings. C7 and C11 fail on main.
  • test_an_unrecognised_signature_is_not_parsed_as_a_record — the unknown word is chosen so that, read as a header, it is a 16-byte S_GPROC32; nothing is listed, nothing is counted as malformed, and the warning names 0x11100010.
  • tests/_synth.py's module_sym_stream takes the signature as a parameter, defaulting to C13.

make test (605 passed), make lint and make fuzz are clean, plus a 5000-input fuzz run at seed 7. Every golden fixture is C13-signed, so no golden count moves.

… unknown

module_symbol_bytes stripped the 4-byte CodeView signature only when it
was C13. A C7 (0) or C11 (1) signature stayed on the front of the
records, where iter_records read it as a record header and walked every
record after it at the wrong alignment, silently.

All three signatures are now stripped. A first word that is none of
them yields no symbols, since the layout after it is not known, and
Diagnostics.unrecognised_signatures counts such modules by the value
found, which a diagnose() warning names.

Because a returned body now always had exactly four bytes stripped, the
fixed + CV_SIGNATURE_SIZE applied to record offsets matches the strip
on every path that yields records.

Fixes danielplohmann#64
…ture

# Conflicts:
#	CHANGELOG.md
#	purepdb/pdb.py
#	tests/test_diagnose.py
@danielplohmann
danielplohmann merged commit 5ad39e3 into danielplohmann:main Sep 15, 2026
11 checks passed
danielplohmann pushed a commit that referenced this pull request Sep 15, 2026
Findings from the correctness audit, each one a case where a damaged or unusual
file produced an unhandled exception or a silent gap rather than a `PdbError`
and a diagnostic.

- **A missing PDB Info stream** no longer escapes. `PDB.named_streams()` is
  guarded by `is_valid_stream` and catches `MsfError`, returning `{}` as
  documented, so `diagnose()`, `string_table()` and `lines()` produce empty
  results with a clear explanation instead of raising.
- **`PublicsStream.parse`** raises plain `PdbError` rather than `ValueError`
  when the header or address map is truncated, preserving the invariant that
  `PdbError` is the only exception crossing a public boundary. Plain `PdbError`
  and not `MsfError`: the publics stream is not an MSF container.
- **DBI substream sizes are bounded.** A negative size would slice backwards,
  aliasing bytes that were never substream data; a size overrunning the buffer
  is a header that lies about its own layout. Both raise `MsfError` with
  context, while damage *inside* a substream still degrades rather than raising.
- **C13 line-info truncations are reported.** `C13Truncation` records them,
  `iter_subsections` and `parse_lines` collect them, and
  `Diagnostics.c13_truncations` plus a warning surface them.
- **The fuzzer covers `publics_stream()`**, which was the one public listing it
  never exercised.
- Dead `Reader.align` removed; docstrings added to `ContributionMap.__len__`
  and `SectionContribution.contains`.

P3-5 (the conditional signature offset) is **not** in this PR. #65 made the
strip unconditional on every path that returns records, so the fixed
`+ CV_SIGNATURE_SIZE` already matches and the conditional helper had nothing
left to correct; `tests/test_inline.py` is identical to main as a result.

Last of the audit queue, after #60, #61, #62 and #65. 645 passed, ruff and ty
clean, fuzz clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
danielplohmann pushed a commit that referenced this pull request Sep 15, 2026
Dev-group pin only; `ruff` governs the `lint and type check` CI job and
`make lint`. No `.pre-commit-config.yaml` in this repository, so there is no
`ruff-pre-commit` rev that has to move with it.

Verified against this main rather than the one the PR was opened on: 0.16.6
reports `All checks passed!` over the whole tree, including the roughly 4400
lines that landed in #60, #61, #62, #65 and #53 after this PR was raised.
`target-version` stays `py311`.
danielplohmann pushed a commit that referenced this pull request Sep 15, 2026
Dev-group pin only; `ty` governs the type half of the `lint and type check` CI
job and `make lint`.

Dependabot rebased this onto the ruff bump and re-resolved the target from
0.0.79 to 0.0.80 in the process, so it was re-verified at the version actually
pinned here: 0.0.80 reports `All checks passed!` over the whole tree, including
the roughly 4400 lines that landed in #60, #61, #62, #65 and #53 after this PR
was raised.

`ty` is still 0.0.x, so it is the pin in this group most likely to surface a new
diagnostic on a bump.
danielplohmann added a commit that referenced this pull request Sep 15, 2026
Both version strings to 0.6.0, `## [Unreleased]` renamed to
`## [0.6.0] - 2026-09-15` with a fresh empty one above it, and the compare links
moved on. No source changes.

First release cut through `publish-release.yml`, which has existed since #59
without a tag ever exercising it.

23 entries from five merged PRs plus the release machinery and three dependency
bumps: the corpus-audit fixes (#60), the single-pass record walk (#61), the
mapped `PDB.open` and streamed inline-site placement (#62), the C7/C11 signature
handling (#65) and the parse-boundary hardening (#53).

Minor rather than patch: the `Fixed` entry for the fused
`ChangeCodeLengthAndCodeOffset` annotation says **This moves addresses** -- the
second and later ranges of an inline site built from that opcode now start
earlier than 0.5.0 reported, on every rust-lld and clang output. The first
range, and every single-range site, are unchanged. `CHANGELOG.md`'s preamble
reserves that for a breaking change and requires it to be stated.

Verified before tagging: `release_guard.py --tag v0.6.0` extracts the section
and reports prerelease=false, and refuses v0.7.0 naming both version strings;
645 passed, ruff and ty clean on the new pins; `python -m build` plus
`twine check --strict` pass on both artefacts; and the wheel, installed into a
clean venv and imported with `python -P`, reports 0.6.0 from site-packages with
`py.typed` present and runs the entry point.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@r0ny123
r0ny123 deleted the module-stream-signature branch September 30, 2026 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A C7 or C11 module-stream signature is neither stripped nor recognised

2 participants