From 89337502bd7e70b54d5b9ca61e99725525fcf54b Mon Sep 17 00:00:00 2001 From: r0ny123 <49360849+r0ny123@users.noreply.github.com> Date: Thu, 10 Sep 2026 21:11:09 +0530 Subject: [PATCH 1/6] feat(common): refuse a candidate the image declares interior, not only a gap one The .eh_frame rule added in #300 refuses a candidate that opens strictly inside a range the image declares, and it is reached only from the gap scan: it tests self.gap_pointer, so candidates from the prologue scan, from branch targets and from the tailcall paths never meet it. On the 72-cell AArch64 ELF corpus 2,245 of the 2,484 false positives still standing are interior to a declared range, and none of them is a gap candidate. Ask the same question where analysis is about to begin on a candidate from any source. Both of the gap rule's guards apply unchanged: a PLT is exempt because the whole table sits under one FDE, and the range's own start has to be a recovered function because an FDE can begin in the alignment padding ahead of its own. A third guard is new, and the corpus is what found it. A declared range can reach past everything its function's control flow arrives at, and refusing an address out there discards bytes nothing else claims along with any reference only those bytes carry. Without it the AArch64 corpus loses three functions, each the sole target of a call sitting in exactly that unreached tail. Requiring the owner's own recovered extent to surround the address costs 163 of the refusals and returns all three. Measured against compiler symbol tables, no corpus losing a true positive: 72 AArch64 ELF cells PPV 95.994 -> 97.063 -683 FP, TP and FN identical 140 built C/C++ ELF PPV 98.903 -> 98.969 -77 FP, TP and FN identical The 120 MinGW PE cells, 23 Go cells, 11 ARM64 Mach-O cells and all 57 malpedia dumps are bit-identical, which is the control that it reaches only images carrying an .eh_frame. Rust is bit-identical too, and not because the rule is inert there: its images decode their ranges and 25 of 26 false positives on the first cell are interior to one, but the guards decline all of them. Analysis is slightly faster, because a refused candidate is one nothing then analyses. Also moves a stray unittest.main() in the test file, which sat above a test class and so left that class undefined when the file is run directly. --- src/smda/SmdaConfig.py | 15 +++ src/smda/common/FunctionCandidateManager.py | 29 +++++ src/smda/common/RecursiveDisassembler.py | 7 ++ tests/testFdeInteriorGaps.py | 116 +++++++++++++++++++- 4 files changed, 163 insertions(+), 4 deletions(-) diff --git a/src/smda/SmdaConfig.py b/src/smda/SmdaConfig.py index 59501096..545d9f29 100644 --- a/src/smda/SmdaConfig.py +++ b/src/smda/SmdaConfig.py @@ -211,6 +211,21 @@ class SmdaConfig: # table names `dispatch`, and aarch64_static drops 0x400350 and 0x40DF30, both mid-function # instructions inside a declared FDE. None of the six carries a symbol or is a declared # start, so both baselines moved toward the truth. + # The same evidence also refuses a candidate from any other source, at the point analysis + # would begin on it, since the gap pointer reaches only what the gap scan walks to and the + # seeding scans reach the rest. That arm carries a third condition the gap scan does not + # need: the owner's own recovered extent has to surround the address. A declared range can + # reach past everything its function's control flow arrives at, and refusing an address out + # there discards bytes nothing else claims along with any reference only those bytes carry + # - without it the AArch64 corpus loses three functions reached through exactly that shape. + # Measured against compiler symbol tables, no corpus losing a true positive: + # 72 AArch64 ELF cells PPV 95.994 -> 97.063 -683 FP at identical TP and FN + # 140 built C/C++ ELF PPV 98.903 -> 98.969 -77 FP at identical TP and FN + # The 120 MinGW PE cells, 23 Go cells, 11 ARM64 Mach-O cells and all 57 malpedia dumps are + # bit-identical, the control that it reaches only images carrying an .eh_frame. Rust is too, + # and not because the rule is inert there: its images decode their ranges and 25 of 26 false + # positives on the first cell are interior to one, but the two conditions above decline all + # of them - 78 of 94 on the owner's extent, the rest on the owner not being recovered. USE_ELF_FDE_INTERIOR_GAPS = True RESOLVE_REGISTER_CALLS = True # resolve "call/jmp dword ptr [ + ]" against a runtime-built import table and diff --git a/src/smda/common/FunctionCandidateManager.py b/src/smda/common/FunctionCandidateManager.py index cd16e96d..22726b94 100644 --- a/src/smda/common/FunctionCandidateManager.py +++ b/src/smda/common/FunctionCandidateManager.py @@ -710,6 +710,35 @@ def declaredFdeRangeContaining(self, addr): index -= 1 return None + def declaredInteriorOwner(self, addr): + """The recovered function whose declared `.eh_frame` range contains `addr`, or None. + + The same evidence the gap scan already refuses on, asked at the point a candidate from + any source is about to be analysed rather than only where the gap pointer reaches. Both + of that rule's guards apply unchanged. A PLT is exempt because the whole table sits + under one FDE, so the range test reads every stub after the first as interior to the + first. And the range's own start has to be a recovered function, because an FDE can + begin in the alignment padding ahead of its function, which leaves the real entry a few + bytes in interior to nothing. + + A third guard the gap scan does not need: the owner's own recovered extent has to + surround the address. An FDE can reach past everything its function's control flow + arrives at, and refusing an address out there discards bytes nothing else claims -- + along with any reference only those bytes carry, which costs real functions elsewhere. + Inside the extent the owner already accounts for the address, so nothing is lost. + """ + if not self.config.USE_ELF_FDE_INTERIOR_GAPS: + return None + if self.isInDeclaredPltSection(addr): + return None + containing = self.declaredFdeRangeContaining(addr) + if containing is None or containing[0] not in self.disassembly.functions: + return None + borders = self.disassembly.function_borders.get(containing[0]) + if borders is None or not borders[0] <= addr < borders[1]: + return None + return containing[0] + def opensInsideDeclaredFdeRange(self, addr): """Whether `addr` falls inside a declared FDE range without being that range's start.""" return self.declaredFdeRangeContaining(addr) is not None diff --git a/src/smda/common/RecursiveDisassembler.py b/src/smda/common/RecursiveDisassembler.py index 6e2557b9..f515693a 100644 --- a/src/smda/common/RecursiveDisassembler.py +++ b/src/smda/common/RecursiveDisassembler.py @@ -261,6 +261,13 @@ def analyzeFunction(self, start_addr, as_gap=False): # state.getBlocks(); this collision path is unreachable from the gap pass today, so # the change is behavior-neutral (output stays byte-for-byte identical). return state + declared_owner = self.fc_manager.declaredInteriorOwner(start_addr) + if declared_owner is not None: + self.fc_manager.updateAnalysisAborted( + start_addr, + f"inside the declared range of function 0x{declared_owner:08x}", + ) + return state blocks_processed = 0 while state.hasUnprocessedBlocks(): blocks_processed += 1 diff --git a/tests/testFdeInteriorGaps.py b/tests/testFdeInteriorGaps.py index f0023a0d..e9926b74 100644 --- a/tests/testFdeInteriorGaps.py +++ b/tests/testFdeInteriorGaps.py @@ -15,6 +15,7 @@ import lief from smda.common.EhFrameDecoder import decodeEhFrameFdeRanges +from smda.common.FunctionCandidateManager import FunctionCandidateManager from smda.Disassembler import Disassembler from smda.SmdaConfig import SmdaConfig @@ -86,10 +87,6 @@ def testNothingTheImageNamesIsRefused(self): self.assertEqual((off & named) - on, set()) -if __name__ == "__main__": - unittest.main() - - class FdeInteriorGapRuleAArch64Test(unittest.TestCase): """The rule lives in the shared candidate manager and both gap scans consult it. @@ -138,3 +135,114 @@ def testEveryRefusedAddressSatisfiesTheRulesOwnContract(self): self.assertIsNotNone(owner, f"0x{address:x} was refused but is inside no declared range") # the condition that keeps an FDE opening in padding from refusing its own function self.assertIn(owner[0], on, f"0x{address:x} was refused by a range whose start is not a function") + + def analysedFixture(self, claim=None): + """Run the fixture, optionally having the rule claim one address, and keep the manager.""" + config = SmdaConfig() + config.CALCULATE_SCC = False + config.CALCULATE_NESTING = False + config.CALCULATE_HASHING = False + original = FunctionCandidateManager.declaredInteriorOwner + if claim is not None: + address, owner = claim + + def claiming(manager, candidate_address): + if candidate_address == address: + return owner + return original(manager, candidate_address) + + FunctionCandidateManager.declaredInteriorOwner = claiming + with tempfile.NamedTemporaryFile(suffix=".elf", delete=False) as handle: + handle.write(self.data) + temp_path = handle.name + disassembler = Disassembler(config) + try: + report = disassembler.disassembleFile(temp_path) + finally: + os.unlink(temp_path) + FunctionCandidateManager.declaredInteriorOwner = original + return disassembler.disassembler, {function.offset for function in report.getFunctions()} + + def testAnalysisItselfDeclinesWhatTheRuleClaims(self): + """The refusal is wired into analysis, not only into the gap scan. + + On this fixture every address the rule would claim is one the gap scan already + refused or the collision check already caught, so asserting over what it happens to + drop here would assert nothing. Claiming one address the fixture does recover is what + shows analysis consults the rule at all, and that it declines rather than reports. + """ + _, baseline = self.analysedFixture() + self.assertGreater(len(baseline), 200) + owner, target = sorted(baseline)[0], sorted(baseline)[1] + + backend, recovered = self.analysedFixture(claim=(target, owner)) + self.assertNotIn(target, recovered, "analysis reported an address the rule claimed") + self.assertEqual(baseline - recovered, {target}, "the claim moved more than the address it named") + # the reason is not asserted: a refused address is offered again as a gap candidate, + # and that pass records its own outcome over this one + self.assertTrue(backend.fc_manager.candidates[target].analysis_aborted) + + +class _RecoveredDisassembly: + """The two things `declaredInteriorOwner` reads about what analysis has recovered.""" + + def __init__(self, functions, borders): + self.functions = functions + self.function_borders = borders + + +def declaredOwnerOf(address, ranges, functions, borders, plt=(), enabled=True): + manager = FunctionCandidateManager(SmdaConfig()) + manager.config.USE_ELF_FDE_INTERIOR_GAPS = enabled + manager.disassembly = _RecoveredDisassembly(dict.fromkeys(functions), dict(borders)) + manager._eh_frame_fde_ranges = list(ranges) + manager._eh_frame_fde_starts = [start for start, _ in ranges] + manager._plt_ranges = list(plt) + return manager.declaredInteriorOwner(address) + + +class DeclaredInteriorOwnerTest(unittest.TestCase): + """The conditions the analysis-time refusal requires, one at a time. + + The gap scan reaches a candidate only where its pointer walks; this answers the same + question for a candidate from any source, so each condition needs a case of its own + rather than whatever a corpus happens to exercise. + """ + + RANGES = [(0x1000, 0x2000)] + FUNCTIONS = [0x1000] + BORDERS = {0x1000: (0x1000, 0x1F00)} + + def testAnAddressItsOwnerSurroundsIsRefused(self): + self.assertEqual(declaredOwnerOf(0x1500, self.RANGES, self.FUNCTIONS, self.BORDERS), 0x1000) + + def testARangeStartIsNotInteriorToItself(self): + self.assertIsNone(declaredOwnerOf(0x1000, self.RANGES, self.FUNCTIONS, self.BORDERS)) + + def testAnAddressOutsideEveryRangeIsKept(self): + self.assertIsNone(declaredOwnerOf(0x2500, self.RANGES, self.FUNCTIONS, self.BORDERS)) + + def testARangeWhoseStartWasNotRecoveredRefusesNothing(self): + # an FDE can begin in the alignment padding ahead of its function, and then the real + # entry a few bytes in is interior to nothing + self.assertIsNone(declaredOwnerOf(0x1500, self.RANGES, [], self.BORDERS)) + + def testAnAddressPastTheOwnersRecoveredExtentIsKept(self): + # the declared range reaches further than the owner's control flow arrived; refusing + # out there discards bytes nothing else claims, and any reference only they carry + short = {0x1000: (0x1000, 0x1100)} + self.assertIsNone(declaredOwnerOf(0x1500, self.RANGES, self.FUNCTIONS, short)) + + def testAnOwnerWithNoRecordedExtentRefusesNothing(self): + self.assertIsNone(declaredOwnerOf(0x1500, self.RANGES, self.FUNCTIONS, {})) + + def testAStubInADeclaredPltIsExempt(self): + # the whole table sits under one FDE, so every stub after the first reads as interior + self.assertIsNone(declaredOwnerOf(0x1500, self.RANGES, self.FUNCTIONS, self.BORDERS, plt=[(0x1400, 0x1600)])) + + def testTheFlagTurnsTheRefusalOff(self): + self.assertIsNone(declaredOwnerOf(0x1500, self.RANGES, self.FUNCTIONS, self.BORDERS, enabled=False)) + + +if __name__ == "__main__": + unittest.main() From 51c92cde93f8c92bf4468a7864d2df420809a608 Mon Sep 17 00:00:00 2001 From: r0ny123 <49360849+r0ny123@users.noreply.github.com> Date: Fri, 11 Sep 2026 12:53:10 +0530 Subject: [PATCH 2/6] docs(common): state what the interior refusal can actually reach The docstring said that inside the owner's recovered extent "the owner already accounts for the address, so nothing is lost". It does not. The rule is consulted immediately after the collision check, which tests membership in a byte-level code map, so every address that reaches it is one the owner's analysis never covered; function_borders holds only the extremes of that code, not its coverage. That leaves a residual exposure the old sentence denied: a real entry carrying no FDE of its own, sitting in a hole inside a neighbour's declared range and recovered envelope. Nothing in the format rules that shape out -- an entry with its own FDE is never interior to it, since a range's own start is excluded -- so what bounds it is the measurement, not the structure. Say that instead. --- src/smda/common/FunctionCandidateManager.py | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/smda/common/FunctionCandidateManager.py b/src/smda/common/FunctionCandidateManager.py index 22726b94..03570a46 100644 --- a/src/smda/common/FunctionCandidateManager.py +++ b/src/smda/common/FunctionCandidateManager.py @@ -725,7 +725,15 @@ def declaredInteriorOwner(self, addr): surround the address. An FDE can reach past everything its function's control flow arrives at, and refusing an address out there discards bytes nothing else claims -- along with any reference only those bytes carry, which costs real functions elsewhere. - Inside the extent the owner already accounts for the address, so nothing is lost. + Inside the extent, the address is never one the owner decoded: a candidate landing on + recovered code is refused a step earlier against a byte-level code map, and + `function_borders` records only the extremes of that code, not its coverage. What + reaches here sits in the holes -- alignment between blocks, an unreached tail, a data + island -- so the exposure is a real entry with no FDE of its own in one of them, which + loses whatever reference only its bytes carry. An entry that carries its own FDE is + never interior to it, so the shape needs a routine covered by a neighbour's range, and + nothing in the format forbids that; what bounds it is measurement rather than + structure, no true positive lost across the ELF corpora this was measured over. """ if not self.config.USE_ELF_FDE_INTERIOR_GAPS: return None From b2af2dfef247a6a235a524c0d8bcfe51c77cae2c Mon Sep 17 00:00:00 2001 From: r0ny123 <49360849+r0ny123@users.noreply.github.com> Date: Fri, 11 Sep 2026 13:14:07 +0530 Subject: [PATCH 3/6] test(tests): keep a failed claim from leaking onto the class analysedFixture() patches declaredInteriorOwner on the class and restores it in a finally, but the sample write and the disassembler construction sat outside the try. Either raising would leave the claim installed for every later test in the process, turning one failure into a cascade that does not name its cause. Put everything after the patch inside the try. --- tests/testFdeInteriorGaps.py | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/tests/testFdeInteriorGaps.py b/tests/testFdeInteriorGaps.py index e9926b74..ad660a94 100644 --- a/tests/testFdeInteriorGaps.py +++ b/tests/testFdeInteriorGaps.py @@ -152,14 +152,19 @@ def claiming(manager, candidate_address): return original(manager, candidate_address) FunctionCandidateManager.declaredInteriorOwner = claiming - with tempfile.NamedTemporaryFile(suffix=".elf", delete=False) as handle: - handle.write(self.data) - temp_path = handle.name - disassembler = Disassembler(config) + temp_path = None try: + with tempfile.NamedTemporaryFile(suffix=".elf", delete=False) as handle: + handle.write(self.data) + temp_path = handle.name + disassembler = Disassembler(config) report = disassembler.disassembleFile(temp_path) finally: - os.unlink(temp_path) + if temp_path is not None: + os.unlink(temp_path) + # restore inside the finally, and with everything after the patch inside the try: + # a failure while writing the sample or building the disassembler would otherwise + # leave the claim installed on the class for every later test in the process FunctionCandidateManager.declaredInteriorOwner = original return disassembler.disassembler, {function.offset for function in report.getFunctions()} From 9d3d924ce17460a14587c59ec29314c57e72779a Mon Sep 17 00:00:00 2001 From: r0ny123 <49360849+r0ny123@users.noreply.github.com> Date: Fri, 11 Sep 2026 17:38:56 +0530 Subject: [PATCH 4/6] docs(docs): add the Unreleased entry for the interior refusal Changed for the rule itself, Compatibility for what it means to a consumer holding stored reports, which is the half a version bump does not carry. The corpus figures are marked as measured on a043145 and as not reproducible from the bundled fixtures, which produce 0 refusals. --- CHANGELOG.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5eb9d82e..fafac735 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -74,6 +74,16 @@ past roughly six lines it belongs in the PR the entry links. ### Changed +- **(common)** Refuse a candidate from any source that the image's `.eh_frame` declares interior to a function, + not only a gap-scan candidate -- the gap pointer reaches only what the gap scan walks to, so the prologue, + reference and symbol scans seeded the rest unchecked. A procedure linkage table is exempt, the declared + range's own start has to be a recovered function, and that owner's recovered extent has to surround the + address. *Measured on `a043145` against compiler symbol tables:* 72 AArch64 ELF cells 95.994 -> 97.063 PPV + (-683 false positives) and 140 built C/C++ ELF cells 98.903 -> 98.969 (-77), both at identical true positives + and false negatives; the PE, Go, ARM64 Mach-O and 57 malpedia cells are bit-identical, which is the control + that it reaches only images carrying an `.eh_frame`. *Not reproducible from the bundled fixtures, which + produce 0 refusals.* (#327) + ### Deprecated ### Removed @@ -84,6 +94,10 @@ past roughly six lines it belongs in the PR the entry links. ### Compatibility +- Recovery output moves on ELF images carrying an `.eh_frame`: an address a declared FDE range covers is no + longer reported as a function start unless it is that range's own start. No true positive was lost on the + corpora this was measured over. (#327) + ## Older releases Entries below predate this format and are kept verbatim, in the one-line shape they were written in. They are From 60f6b70e679fc46ebe81995476483100ddb8b3a3 Mon Sep 17 00:00:00 2001 From: r0ny123 <49360849+r0ny123@users.noreply.github.com> Date: Fri, 11 Sep 2026 18:34:37 +0530 Subject: [PATCH 5/6] docs(docs): name the tree the interior-refusal figures were taken on The entry credited a043145, which is where the branch sat rather than where the numbers came from; they were carried forward from an earlier measurement. Re-measured against compiler symbol tables on 857081f, the tree this lands on, every figure reproduces exactly: 72 AArch64 ELF cells 95.994 -> 97.063 PPV at -683 false positives, 140 built C/C++ ELF cells 98.903 -> 98.969 at -77, no true positive lost on either, and the 120 MinGW PE, 23 Go, 11 ARM64 Mach-O and 57 malpedia cells all bit-identical. An absolute false-positive count is a property of every other rule in the engine on the day it was taken. These did not decay because nothing landing since moves recovery -- the releases between were tests, logging and documentation -- which is a reason to name the tree, not a reason to skip checking. --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fafac735..2ae157df 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -78,7 +78,7 @@ past roughly six lines it belongs in the PR the entry links. not only a gap-scan candidate -- the gap pointer reaches only what the gap scan walks to, so the prologue, reference and symbol scans seeded the rest unchecked. A procedure linkage table is exempt, the declared range's own start has to be a recovered function, and that owner's recovered extent has to surround the - address. *Measured on `a043145` against compiler symbol tables:* 72 AArch64 ELF cells 95.994 -> 97.063 PPV + address. *Measured on `857081f` against compiler symbol tables:* 72 AArch64 ELF cells 95.994 -> 97.063 PPV (-683 false positives) and 140 built C/C++ ELF cells 98.903 -> 98.969 (-77), both at identical true positives and false negatives; the PE, Go, ARM64 Mach-O and 57 malpedia cells are bit-identical, which is the control that it reaches only images carrying an `.eh_frame`. *Not reproducible from the bundled fixtures, which From 01885714230396570b5989802a67b6b2af07d497 Mon Sep 17 00:00:00 2001 From: r0ny123 <49360849+r0ny123@users.noreply.github.com> Date: Fri, 11 Sep 2026 18:56:12 +0530 Subject: [PATCH 6/6] docs(docs): re-verify the interior-refusal figures on the current base #339 landed under this branch and touches the same gap path, so the figures were re-run against it rather than assumed to carry. They are unchanged: 72 AArch64 ELF cells -683 false positives at 95.994 -> 97.063 PPV, 140 built C/C++ ELF cells -77 at 98.903 -> 98.969, no true positive lost on either, and 120/120 MinGW PE cells bit-identical. The PE control's own baseline moved 93.204 -> 93.191 between the two trees, which is what naming the tree is for. --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ced5e37c..64014dfe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -78,7 +78,7 @@ past roughly six lines it belongs in the PR the entry links. not only a gap-scan candidate -- the gap pointer reaches only what the gap scan walks to, so the prologue, reference and symbol scans seeded the rest unchecked. A procedure linkage table is exempt, the declared range's own start has to be a recovered function, and that owner's recovered extent has to surround the - address. *Measured on `857081f` against compiler symbol tables:* 72 AArch64 ELF cells 95.994 -> 97.063 PPV + address. *Measured on `0a0a4c6` against compiler symbol tables:* 72 AArch64 ELF cells 95.994 -> 97.063 PPV (-683 false positives) and 140 built C/C++ ELF cells 98.903 -> 98.969 (-77), both at identical true positives and false negatives; the PE, Go, ARM64 Mach-O and 57 malpedia cells are bit-identical, which is the control that it reaches only images carrying an `.eh_frame`. *Not reproducible from the bundled fixtures, which