Skip to content

feat(common): let the exception directory refuse an interior candidate too - #329

Merged
danielplohmann merged 4 commits into
danielplohmann:masterfrom
r0ny123:feat/pdata-interior-analysis-gate
Sep 13, 2026
Merged

danielplohmann merged 4 commits into
danielplohmann:masterfrom
r0ny123:feat/pdata-interior-analysis-gate

Conversation

@r0ny123

@r0ny123 r0ny123 commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Closes #328. Stacked on #327 — it extends the helper that PR introduces, so it must merge after it. The diff shown against master will include #327's commit until then; this PR's own commit is 2efa621.

Third of the three rules named in the #324 sweep, and the last one with anything left to reach. #327 widened the .eh_frame arm to a candidate from any source; this does the same for the PE exception directory, under the guards that arm established.

The rule

The same two conditions, unchanged: the RUNTIME_FUNCTION record's own function has to be recovered, and its recovered extent has to surround the address. The two evidence sources stay format-disjoint exactly as the gap scan's comment already notes — _pdata_ranges is only ever filled from a PE exception directory and the .eh_frame ranges decode nothing unless lief reports an ELF — so no address is ever arbitrated between them.

One deliberate difference from the gap scan. There, a fragment record is evidence on its own: it says the extent is part of some other function, and the scan needs nothing more. Here it is not taken as that. A fragment's own start is not the function covering the address, so it fails the recovered-owner test and declines rather than refusing. At a point where the better evidence — an actual recovered owner — is available, using the weaker form would be the wrong reading. No fragment record appeared among the interior false positives in the sample either way.

Measured

feat/fde-interior-analysis-gate (#327) → this branch, so the figures isolate this arm.

corpus n TP ΔFP bit-identical cells
Built C/C++ MinGW PE x64 120 unchanged −47 109 / 120
Built Rust, windows-gnu-x64 2 unchanged −2 —
Built C/C++ ELF (gcc, clang) 140 unchanged 0 140 / 140
Built C/C++ AArch64 ELF 72 unchanged 0 72 / 72
Built Go 23 unchanged 0 23 / 23
ARM64 Mach-O 11 unchanged 0 11 / 11
Rust, ELF and 32-bit PE cells 22 unchanged 0 22 / 22
Malpedia dumps 57 unchanged 0 57 / 57

No corpus loses a true positive, and only PE x64 cells differ anywhere — which is the control this arm has to pass, the mirror of #327's ELF-only one.

The part worth reading before merging

On memory dumps this evidence does not exist. None of the 57 malpedia dumps declares an exception directory at all — _pdata_ranges is empty, so the rule never reaches its first condition. That is the corpus SMDA cares most about, and it is the honest limit of this change.

Between that and 47 false positives on one PE corpus, this is a small win, and #328 says so rather than selling it. It is here because it is free of recall risk under guards that are already measured, and because leaving one of three siblings unwidened without a reason is worse than either doing it or declining it on the record. If you would rather close #328 and keep the surface smaller, that is a reasonable call and the numbers are the argument either way.

One thing I had wrong

I filed #328 saying 14 of 1,038 on a 20-cell sample, and scaled that to roughly 80 across 120 cells. The real figure is 47. The sample over-predicted; the table above is the measurement.

I also assumed the Rust corpus was ELF, which is why the two moving cells looked like an anomaly at first. They are windows-gnu-x64 — PE x64, exactly where this applies. The corpus is mixed, and every one of its ELF and 32-bit cells is bit-identical.

Tests

Seven cases in tests/testFdeInteriorGaps.py::DeclaredInteriorOwnerPeTest, one condition each, exercising the arm through a stubbed exception directory so the ELF and PE arms are tested apart. Each was checked to fail when the guard it describes is removed: dropping the arm fails two, dropping its flag check fails one.

  • python -m pytest tests/ → 2,076 passed, 1 skipped, 2,593 subtests
  • ruff check . and ruff format --check . → clean
  • make typecheck → exit 0, 0 error-level diagnostics, no new diagnostics
  • Diff coverage against this PR's base → 100% (12 lines, 0 missing)

No bundled fixture baseline moves.

Fixed after review

The shared rule answered to the wrong architecture's flag. Both native backends fill _pdata_ranges — intel from an x64 exception directory, aarch64 from an ARM64 PE's — and each gap scan gates on its own flag, but declaredInteriorOwner named only USE_PE_X64_PDATA_INTERIOR_GAPS. On an ARM64 PE that left USE_PE_ARM64_PDATA_INTERIOR_GAPS unable to turn the ARM64 rule off, and the x64 flag able to. Default output does not move — with both flags on the gap scan reaches those addresses first — but testArm64PeInteriorGapFixture fails, five of its eight pinned addresses stopping short of off - on.

Each backend now answers for itself through a _pdataInteriorRefusalEnabled() hook, mirroring the split the two gap scans already have. The base returns the disabled answer, which is what a manager carving no exception directory should give.

The PE unit cases had to move with it. declaredOwnerOf built a bare common manager, which now inherits that answer: two of the seven cases would fail outright and the other five would pass vacuously, taking their None from the gate rather than from the condition each is about. The helper takes a manager_class and the PE class builds the backend that fills the ranges. DeclaredInteriorOwnerArm64PeTest asks all seven of the same conditions of aarch64's manager, each class asserting that the other architecture's flag does not gate it, and DeclaredInteriorOwnerUnbackedPeTest pins the inherited answer so a manager added without an override cannot start refusing on extents it never filled.

Four mutations verified, each caught by the intended case and nothing else: either backend's override answering to the other's flag, the shared gate naming a flag directly, and the base returning True.

Also documented: disjointness between the two structures is load-bearing here in a way it is not in the gap scan — a declared FDE range whose start was never recovered sets owner and then fails the recovered-owner test, so on an image carrying both the exception directory would never be asked about that address.

Full suite on the current tip: 2,136 passed, 1 skipped, 2,596 subtests, ruff clean, make typecheck exit 0.

@danielplohmann

Copy link
Copy Markdown
Owner

Blocking, and it is an interaction with #333 rather than anything wrong in the measurement. #333 merged an hour ago as 50a818a, so master now carries a real ARM64 PE fixture — and on this branch its test fails:

tests/testArm64PeInteriorGapFixture.py::testTheDeclaredExtentsRefuseTheirInteriorAddresses
AssertionError: Items in the second set but not the first:
  0x1400014E8  0x1400015F8  0x140001614  0x140001628  0x140001674

Five of the eight pinned addresses stop appearing in off - on, because with USE_PE_ARM64_PDATA_INTERIOR_GAPS off they are now refused anyway — by this PR's arm, which is gated on the x64 flag.

The cause

declaredInteriorOwner reads self._pdata_ranges and gates it on USE_PE_X64_PDATA_INTERIOR_GAPS. But that list is filled by whichever backend carved it, and each backend's gap scan gates on its own flag:

fills _pdata_ranges gap scan gates on
intel/FunctionCandidateManager.py :878, x64 exception directory :332 → USE_PE_X64_PDATA_INTERIOR_GAPS
aarch64/FunctionCandidateManager.py :287, ARM64 PE .pdata :1102 → USE_PE_ARM64_PDATA_INTERIOR_GAPS

The shared helper has one flag name for both, so on an ARM64 PE the analysis-time refusal answers to the x64 flag. Measured on arm64_pe_probe_xored, instrumenting declaredInteriorOwner:

config analysis-time refusals functions recovered
both flags on (default) 0 105
USE_PE_ARM64_PDATA_INTERIOR_GAPS off 2 (0x1400014E8, 0x140001614) 108, where master gives 113
USE_PE_X64_PDATA_INTERIOR_GAPS off 0 105

So the ARM64 flag no longer turns the ARM64 rule off, and the x64 flag would turn it off on an image it has nothing to do with.

Default output on that fixture does not move — with both flags on the gap scan reaches all eight first, which is why 0 refusals — so this is flag semantics plus a broken test, not a behaviour regression. It is also why your corpus table could not see it: there is no ARM64 PE row in it, and at defaults there is nothing to see.

The fix I would take

A backend hook rather than a flag name in shared code, mirroring the split the two gap scans already have:

# common/FunctionCandidateManager.py
def _pdataInteriorRefusalEnabled(self):
    """Whether this backend's exception-directory interior rule is on. See the two overrides."""
    return False

with intel returning USE_PE_X64_PDATA_INTERIOR_GAPS and aarch64 returning USE_PE_ARM64_PDATA_INTERIOR_GAPS; cil and dalvik inherit the False and lose nothing, since neither fills _pdata_ranges. Then declaredInteriorOwner asks the hook.

I applied that here to check it: testArm64PeInteriorGapFixture goes green and nothing else moves.

It does cost your PE unit cases, and in a way worth knowing before you write it. declaredOwnerOf builds a bare common FunctionCandidateManager, which inherits the False, so two of the seven DeclaredInteriorOwnerPeTest cases fail outright — testAnAddressInsideADeclaredExtentIsRefused and testTheElfFlagDoesNotGateTheExceptionDirectory — and the other five keep passing vacuously: they assert None and now get it from the gate rather than from the condition each one is about. So the helper has to build an intel manager for the PE arm, or the PE half of that class stops testing anything. The ELF cases are unaffected.

Worth a case pinning the architecture split as well: the ARM64 flag off, over the fixture #333 bundles, should produce #333's eight and nothing else — which is exactly the assertion that just failed.

The rest of the review, which stands

The fragment decision is right and it is the interesting difference from the gap scan: a fragment's own start is not the function covering the address, so taking the gap scan's shortcut here would be using the weaker evidence at the point the stronger one is available. Declining rather than refusing is the conservative direction.

The format-disjointness argument holds as written — but note the code makes it load-bearing in a way the gap scan does not: an .eh_frame range whose start was not recovered sets owner and then fails the recovered-owner test, so the .pdata arm is never consulted for that address. Fine while no image carries both, which is what the comment says; worth knowing it is now a structural assumption rather than an ordering one.

Rebase over a043145 and re-run — #326, #330, #331, #332 and #333 are all in master now.

@r0ny123

r0ny123 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Reproduced and fixed, and your diagnosis is exactly right — testArm64PeInteriorGapFixture fails here on this branch with the same five addresses, and the cause is the shared helper naming one architecture's flag for a list both backends fill.

Took your design. _pdataInteriorRefusalEnabled() on the common manager returning the disabled answer, intel returning USE_PE_X64_PDATA_INTERIOR_GAPS and aarch64 returning USE_PE_ARM64_PDATA_INTERIOR_GAPS, and declaredInteriorOwner asking the hook. The fixture test goes green and nothing else moves.

One correction to the note, and it was in my commit message too

cil and dalvik do not inherit the False — they never reach it. Neither implements createCandidateManager; both have standalone disassemblers that never construct a candidate manager or touch RecursiveDisassembler at all. The two native backends are the only managers this engine builds.

That does not change the design — the base still has to answer, because the bare common class is what a unit helper builds and what any manager added later inherits — but the reason is "this is the answer for a manager that carves no directory", not "these two backends rely on it". Stated that way in the docstring, the test docstring and the commit body now.

The unit cases

Exactly as you measured: left alone, testAnAddressInsideADeclaredExtentIsRefused and testTheElfFlagDoesNotGateTheExceptionDirectory fail outright and the other five pass vacuously. declaredOwnerOf takes a manager_class and the PE class builds intel's.

DeclaredInteriorOwnerArm64PeTest subclasses it with aarch64's, so all seven conditions are asked of both backends, and each class asserts that the other architecture's flag does not gate it. DeclaredInteriorOwnerUnbackedPeTest pins the inherited answer directly, so a manager added without an override cannot start refusing on extents it never filled.

Verified by mutation — each of these fails the intended case and nothing else:

mutation caught by
aarch64 override returns the x64 flag (the shipped bug) DeclaredInteriorOwnerArm64PeTest::testTheOtherArchitecturesFlagDoesNotGateThisOne, testArm64PeInteriorGapFixture
shared gate names the x64 flag directly those two plus DeclaredInteriorOwnerArm64PeTest::testTheFlagTurnsTheRefusalOff and the unbacked case
intel override returns the ARM64 flag DeclaredInteriorOwnerPeTest::testTheFlagTurnsTheRefusalOff, testTheOtherArchitecturesFlagDoesNotGateThisOne
base returns True DeclaredInteriorOwnerUnbackedPeTest

The format-disjointness note

Written into the docstring, since you are right that it is now structural rather than incidental:

Disjointness is load-bearing rather than merely tidy, unlike in the gap scan: a declared FDE range whose start was never recovered sets owner and then fails the recovered-owner test, so on an image carrying both structures the exception directory would never be asked about that address.

Left as an assumption rather than restructured — an arm that re-tries on a qualifying owner rather than any owner would be code for a case no image produces.

Sweep

For the class behind this — shared code gating on one backend's flag where a sibling reaches the same path — the whole tree has one other hit, _splitMergedFunctionsAtPdataEnds, which is guarded by backend.name != "intel" and has no ARM64 counterpart flag. Not a sibling. USE_ELF_FDE_INTERIOR_GAPS, USE_ELF_EH_FRAME_CANDIDATES and USE_MACHO_FUNCTION_STARTS are format-scoped rather than architecture-scoped and are correctly named in shared code.

Verification

Full suite on this tip: 2,136 passed, 1 skipped, 2,596 subtests. ruff clean, make typecheck exit 0 at the same diagnostic count this stack already had.

Master is merged in rather than rebased on both this and #327 — I would rather not force-push over branches you have already reviewed, so taking the rebase at merge is yours. #327 carries the docstring correction from your review there, and this still stacks on it.

@r0ny123
r0ny123 force-pushed the feat/pdata-interior-analysis-gate branch from 7b47997 to 7cad9ff Compare September 11, 2026 08:30
@r0ny123

r0ny123 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Rebased over a043145 as you asked, replacing the two merge commits — five linear commits now, the first three being #327's, and the tree is unchanged by it (git diff against the pre-rebase tip is empty), so the suite, lint and typecheck results reported above still stand as measured.

7cad9ff fix(common): let each backend say whether its own pdata rule is on
615bed6 feat(common): let the exception directory refuse an interior candidate too
a4c1513 test(tests): keep a failed claim from leaking onto the class      <- #327
12ec24a docs(common): state what the interior refusal can actually reach  <- #327
c4e3ca1 feat(common): refuse a candidate the image declares interior      <- #327

Once #327 merges this reduces to its own two commits.

@danielplohmann

Copy link
Copy Markdown
Owner

The backend hook is the fix, and 7cad9ff is the shape I checked locally before recommending it — _pdataInteriorRefusalEnabled() asked of the manager that filled _pdata_ranges, mirroring the split the two gap scans already carry, with the common class answering no because a manager that carves no directory has no extents to refuse on.

You went further than the fix in the one place it mattered. Making the PE cases build the real backend rather than a bare common manager is what stops five of the seven passing vacuously — they were getting their None from the gate rather than from the condition each case is about — and DeclaredInteriorOwnerArm64PeTest subclassing the Intel cases pins the split from both sides in the cheapest way available. DeclaredInteriorOwnerUnbackedPeTest is the one I would not have asked for and is the one that will still be right in a year: it pins what a backend added later inherits, so a new manager cannot quietly start refusing on extents it never filled.

testArm64PeInteriorGapFixture goes green, which was the blocker — five of #333's eight pinned addresses were dropping out of off - on because the ARM64 rule answered to the x64 switch.

The widened declaredInteriorOwner summary is worth having; it had still named only .eh_frame after the rule learned to read an exception directory. The added paragraph on disjointness being load-bearing here rather than merely tidy — a declared FDE range whose start was never recovered sets owner and then fails the recovered-owner test, so the directory never gets asked — is the kind of thing that is invisible until someone builds an image carrying both.

Verified stacked over a043145 with #336, #337, #327 and #335: 2,142 passed, 2 skipped, 2,596 subtests; ruff clean; ty exit 0.

No further review notes. One thing before it merges: with #336 and #337 in, this needs its own ## [Unreleased] entry — Changed or Fixed, with the -47 marked as measured on your corpora and not reproduced here, per the provenance rule from #323.

@r0ny123
r0ny123 force-pushed the feat/pdata-interior-analysis-gate branch 2 times, most recently from 959a471 to 48ae03a Compare September 11, 2026 13:05
@danielplohmann

Copy link
Copy Markdown
Owner

Two things: a merge of master is needed, and there is an entry missing.

The merge is mine, not yours

You based this on 0a0a4c6; 8a4d994 (#340) and 130fa02 (#335) have landed since, into the same Unreleased subsections. CHANGELOG.md is the only conflicting file and the resolution is purely additive — keep both sides. Nothing in src/ collides any more, since your merge of 0a0a4c6 already took #339's _failedGapResumeTarget alongside your _pdataInteriorRefusalEnabled in the intel manager.

Resolved and verified locally: 2155 passed, 2 skipped, 2596 subtests, ruff clean, ty at 274 against master's 270 — the same four None | Unknown warnings on this stack's own new lines. I would have pushed it to your branch but I will not push to a contributor's fork without asking, so it is yours to land.

The entry for this PR's actual change is missing

The Unreleased section carries exactly one (#329) entry, and it is the backend-hook fix — the flag-semantics correction from 7cad9ff, which as you correctly say does not move default output. What has no entry is 615bed6, the change this PR is named after: the exception directory refusing an interior candidate from any source. That is the one #328 measured at -47 false positives across 120 MinGW PE cells and -2 on the 2 windows-gnu-x64 Rust cells, and it does move recovery output on PE x64.

So it needs its own bullet under ### Changed, and a ### Compatibility line in the same shape as the one #327 carries — "recovery output moves on PE x64 images carrying an exception directory". Both #338 and #327 have theirs; this one is the gap.

Worth saying plainly because it is the exact failure the format was adopted to prevent, and it has appeared on the first engine PR after it landed: at release time a missing entry is indistinguishable from a change that needed none, and only you can tell the difference while the PR is open. The Changelog entry check passed here because the check is file-level — this PR does touch CHANGELOG.md — which is the acceptable false negative you and I both signed off on in #336. It is doing what it was designed to do; it just cannot catch this shape.

The figures are already written in #328, so this is transcription rather than measurement. Once it is in and master is merged, this is ready and v4.7.0 is next.

@danielplohmann

Copy link
Copy Markdown
Owner

#327 is merged as fb510c4, so this needs a rebase — and it is the squash-merge kind rather than another changelog collision.

Your branch still carries #327's three commits as their original objects (8933750, 51c92cd, b2af2df) plus its two docs commits. Master now has the same content under one squashed commit, so git sees both sides adding the same lines: src/smda/common/FunctionCandidateManager.py and tests/testFdeInteriorGaps.py both conflict. Rebasing onto fb510c4 and dropping the five commits that are now upstream leaves this PR as what it actually is — 80fc3bf (the exception-directory arm), f049a63 (the backend hook) and 48ae03a (its entry).

Two things then, and the second is the one that matters:

  1. The rebase. Mechanical, and the conflicts disappear once the duplicated commits are gone rather than needing resolution.

  2. The missing entry, still open from my note above. ## [Unreleased] carries exactly one (#329) bullet and it is 48ae03a, for the backend hook — the flag-semantics fix that, as you correctly wrote, does not move default output. 80fc3bf has nothing: no Changed bullet and no Compatibility line, and that is the change this PR is named after and the one The .pdata arm of the gap-only rules: measured at 47 false positives, and inert on dumps #328 measured at -47 false positives across 120 MinGW PE cells and -2 on the 2 windows-gnu-x64 Rust cells.

The shape to match is what #327 just landed with: a Changed bullet carrying the mechanism and the figures marked as measured on your corpora and not reproduced here, and a Compatibility line saying recovery output moves on PE x64 images carrying an exception directory. The numbers are already written in #328, so this is transcription.

Everything else is verified and stands — I ran the merged tree at 1fb296b5 before #327 landed: 2155 passed, 2 skipped, 2596 subtests, ruff clean, ty +4 on this stack's own lines. Nothing in the review is outstanding.

This is the last thing before v4.7.0. It closes #328 when it lands.

@r0ny123

r0ny123 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Both done, and the missing entry turned up something in the config comment.

The merge

Merged rather than rebased — the duplicate-add conflicts resolve without rewriting history, since my side of each block is #327's content as this PR modifies it and master's is #327 alone, so taking mine is the union rather than a choice. Verified structurally rather than by eye: the working tree against master is exactly this PR's own six files, RecursiveDisassembler correctly absent because that arm is yours now, and no method or test class appears twice.

That check earned its keep. My earlier "keep both sides" resolution had silently duplicated #327's Compatibility bullet — master's copy and mine, both kept. Removed; every bullet in ## [Unreleased] is now unique, and the section's diff against master is exactly this PR's three.

The missing entry

You are right that 80fc3bf had nothing. Written now as a Changed bullet and a Compatibility line, and re-measured on fb510c4 rather than carried forward — both #327 and #339 have landed under this since those figures were taken, and #339 moves MinGW PE recovery, so the −47 could easily have shifted.

It did not. Every figure reproduces exactly:

corpus n before → after delta TP lost
MinGW PE x64 120 93.191 → 93.235 PPV −47 FP 0
Rust windows-gnu-x64 2 — −2 FP 0
built C/C++ ELF 140 — — 140/140 bit-identical
malpedia dumps 57 — — 57/57 bit-identical

One correction, and it is to a line you quoted approvingly

The config comment said "none of the 57 declares an exception directory at all, so this evidence does not reach a mapped image." Three of them do — x64-prikorma, x64-trickbot and x64-vawtrak, carrying 282, 430 and 370 records.

The conclusion survives; the reason was wrong, and the truth is the better control. Instrumented on all three, the rule is consulted and refuses nothing — every address its guards see is declined. Reachable-and-declining says the guards hold on mapped dumps; an absence says only that the corpus never asked. The comment now says that instead.

Suite on the merged tree 2,156 passed, 1 skipped, 2,596 subtests; ruff clean; ty at 267, this box's master baseline of 263 plus the same four.

…e too

Third of the three rules that read declared evidence only at the gap
pointer, and the last one with anything left to reach. danielplohmann#327 widened the
.eh_frame arm; this does the same for the PE exception directory, under
the guards that arm established: the record's own function has to be
recovered, and its recovered extent has to surround the address.

A fragment record is not the shortcut it is in the gap scan. Its own
start is not the function covering the address, so it fails the
recovered-owner test and declines rather than refusing, which is the
conservative reading at a point where a better one is available.

It is worth much less than the .eh_frame arm, which is why it is a
change of its own rather than part of danielplohmann#327:

  120 MinGW PE x64 cells   -47 FP at identical TP and FN
  2 Rust windows-gnu-x64    -2 FP at identical TP and FN

Nothing else moves. The 140 x86-64 ELF cells, 72 AArch64 ELF cells, 23
Go cells, 11 ARM64 Mach-O cells, the 32-bit and ELF Rust cells and all
57 malpedia dumps are bit-identical, and only PE x64 cells differ in any
corpus. The dumps are the reading worth keeping: none of the 57 declares
an exception directory at all, so this evidence does not reach a mapped
image, which is the corpus this project cares most about.
Both native backends fill _pdata_ranges -- intel from an x64 exception
directory, aarch64 from an ARM64 PE's -- and each gap scan gates on its
own architecture's flag. The shared analysis-time rule named only
USE_PE_X64_PDATA_INTERIOR_GAPS, so on an ARM64 PE the refusal answered to
a switch for a format it never sees: USE_PE_ARM64_PDATA_INTERIOR_GAPS no
longer turned the ARM64 rule off, and the x64 one would.

Default output does not move -- with both flags on, the gap scan reaches
those addresses first -- but testArm64PeInteriorGapFixture fails, because
five of its eight pinned addresses stop appearing with the ARM64 flag off.

Ask a backend hook instead, mirroring the split the two gap scans already
have. Those two managers are the only ones this engine builds, so the
inherited answer is what one carving no directory should give.

The unit helper builds a bare common manager, which now inherits that no,
so the PE cases build the backend that fills the ranges; left as it was,
two of them would fail and the other five would pass vacuously, getting
their None from the gate rather than from the condition each one is about.
An ARM64 subclass of the same cases pins the split from the other side.

Also widen the summary line of declaredInteriorOwner, which still named
only .eh_frame after this rule learned to read an exception directory.
Fixed rather than Changed, and it says so: default output does not move,
because with both flags on the gap scan reaches those addresses first.
What moved is which switch controls the rule.
The Unreleased section carried only the backend-hook fix, which is the
flag-semantics correction and moves no default output. The change this PR
is named after -- a PE exception directory refusing an interior candidate
at analysis time -- had no Changed bullet and no Compatibility line, which
is the entry a consumer holding stored reports actually needs.

Re-measured on fb510c4 rather than carried forward, since both danielplohmann#327 and
danielplohmann#339 have landed under this since the figures were taken: 120 MinGW PE
x64 cells 93.191 -> 93.235 PPV at -47 false positives and 109/120
bit-identical, 2 Rust windows-gnu-x64 cells at -2, neither losing a true
positive, and the 140 C/C++ ELF cells and all 57 malpedia dumps
bit-identical. Every figure reproduces.

Also correct the malpedia reading in the config comment. It claimed none
of the 57 dumps declares an exception directory, so the evidence could not
reach a mapped image. Three of them do, carrying 282 to 430 records, and
the rule is consulted on all three and refuses nothing -- its guards
decline every address they see. The conclusion stands and the reason was
wrong, and reachable-and-declining is the stronger control of the two:
an absence shows nothing about whether the rule would have been safe.
danielplohmann added a commit that referenced this pull request Sep 11, 2026
Four times today the question 'what is left before v4.7.0' was answered by
re-reading the issue list and the PR list and working out which of them
mattered. The scope was spread across an issue acting as an ad-hoc tracker
(#322), the handover sections of out-of-repo session notes, and PR bodies.
None of those is queryable and the first two are maintained by hand.

The Unreleased section #337 introduced covers the other half well -- it is
the record of what landed and what it cost -- but it is silent on what has
not landed, which is the half the question is actually about.

A milestone per release is the thin version of the missing piece: one bucket
holding issues and PRs, a closed/total count, and a filter. It does not order
work and nothing assigns it automatically, so a PR joins when it is accepted
for the release rather than when it is opened -- otherwise it accumulates
into a wishlist and the count stops meaning anything.

v4.7.0 exists and carries #328 and #329.

No CHANGELOG entry: this changes how the repository is organised and nothing
a consumer of the package can observe. Raised in the PR, since it is the
second change today where 'every PR writes a bullet' meets a PR with nothing
user-facing to say.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@r0ny123
r0ny123 force-pushed the feat/pdata-interior-analysis-gate branch from 8245f63 to 024f4bc Compare September 11, 2026 15:39
@r0ny123

r0ny123 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto fb510c4 as you asked. The five upstream commits are gone and this PR is now the four it actually is:

024f4bc docs(docs): add the missing entry for the exception-directory arm
2dd4ab5 docs(docs): add the Unreleased entry for the backend pdata hook
9f2115c fix(common): let each backend say whether its own pdata rule is on
fa32f7f feat(common): let the exception directory refuse an interior candidate too

Zero merge commits, and the tree is unchanged by the rebase — git diff against the pre-rebase tip is empty — so the suite, lint and typecheck figures from the merged tree stand as measured rather than needing a re-run. Re-ran them anyway since the history moved: 2,156 passed, 1 skipped, 2,596 subtests, ruff clean, ty at 267 against this box's master baseline of 263.

Two conflicts came up in the replay rather than disappearing, both additive and both because the commits predate what landed under them: f049a63 puts _pdataInteriorRefusalEnabled where master now has #339's _failedGapResumeTarget, and 48ae03a adds its bullet to a ### Fixed that has since gained #338's and #335's. Both methods kept, bullet appended after the existing ones.

The diff against master is the six files this PR owns and nothing else.

@danielplohmann
danielplohmann merged commit eae880a into danielplohmann:master Sep 13, 2026
26 checks passed
@danielplohmann

Copy link
Copy Markdown
Owner

Merged as eae880a.

The backend hook is the right shape and it clears what I blocked on: testArm64PeInteriorGapFixture goes green again, and each backend now answers for its own flag rather than both answering to USE_PE_X64_PDATA_INTERIOR_GAPS. The DeclaredInteriorOwnerUnbackedPeTest case is the part I would not have asked for and is worth having - it pins the inherited answer, so a manager added later without an override cannot start refusing on extents it never filled. Same for taking the five vacuous passes seriously rather than leaving them green.

Verified on saturn against master at 2c74bd7: 2031 passed, 2 skipped, 1663 subtests on the fast tier, and the full matrix green on your side.

Two notes, neither blocking:

  • The dump result is the honest part of this PR and the reason The .pdata arm of the gap-only rules: measured at 47 false positives, and inert on dumps #328 was worth filing either way. None of the 57 malpedia dumps declares an exception directory, so on the corpus SMDA cares most about this arm is inert by construction. The changelog entry says so, which is what the provenance rule was for.
  • Declining on a fragment record rather than refusing is the right call. A fragment's start is not the covering function, and taking the weaker evidence where the stronger one is available would have been a different rule than the one measured.

This closes #328 and empties the v4.7.0 milestone. Cutting the release next, by hand - #343 is not in this milestone's scope, so it is not gating it. I will come to #343 after the tag is out.

@danielplohmann danielplohmann mentioned this pull request Sep 13, 2026
@r0ny123
r0ny123 deleted the feat/pdata-interior-analysis-gate branch September 14, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The .pdata arm of the gap-only rules: measured at 47 false positives, and inert on dumps

2 participants