Skip to content

fix(common): bound the LSDA reads that lead nowhere - #335

Merged
danielplohmann merged 4 commits into
danielplohmann:masterfrom
r0ny123:fix/lsda-read-budget
Sep 11, 2026
Merged

danielplohmann merged 4 commits into
danielplohmann:masterfrom
r0ny123:fix/lsda-read-budget

Conversation

@r0ny123

@r0ny123 r0ny123 commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Item 6 of #322.

The diagnosis in the issue is right

_fdeLandingPads checks the table budget before reading, and the comment there says that budget "gates the read as well as the decode". It does not. The budget is charged where a table's length field is parsed, so an LSDA failing before that point — an unsupported LPStart application mode, a short buffer, a TType offset that will not read — costs a read of up to MAX_LSDA_BYTES and charges nothing.

Measured: a section naming MAX_RECORDS distinct such pointers performs 199,999 reads handing back 12.2 GB, spending none of its budget.

But the fix as described would cost recall

"Charge the read at the point it happens" cannot mean charging it to MAX_LSDA_TABLE_BYTES. read_va is asked for MAX_LSDA_BYTES and returns whatever the section holds, so a real image asks for vastly more than it decodes:

cell reads bytes asked for tables decoded
googletest_gcc-x64_O2-static 1,473 92.1 MB ~26 KB
googletest_gcc-arm64_O2-static 1,133 70.8 MB 26,584 B
fmtheavy_linux-gnu-x64_debug 179 11.2 MB —

The 8 MB budget would be gone within about 128 of that first cell's 1,473 reads, and every LSDA after that reads as "declares nothing" — landing pads lost on an ordinary C++ binary. The existing comment's "counts what is decoded rather than what is read" is not a wording slip; it is why the current bound works.

What separates the two cases

Only the reads that lead nowhere. Those are rare on real images and universal on hostile ones:

corpus worst cell's failing reads bytes spent on them
built C/C++ ELF (140 cells) 476 29.75 MB
built AArch64 ELF (72 cells) 100 6.25 MB
Rust (24 cells) 0 0
a section built to exhaust it 199,999 12.2 GB

MAX_LSDA_FAILED_READ_BYTES = 256 MB holds the hostile shape to 4,096 reads and 256 MB — a 48× reduction — with 8.6× headroom over the heaviest real image measured.

Verification

All 447 cells across all six corpora are bit-identical, which is the assertion that this reaches only sections built to exhaust it:

corpus cells identical
built AArch64 ELF 72 72
built C/C++ (260-cell matrix) 260 260
Rust 24 24
Go 23 23
ARM64 Mach-O 11 11
malpedia dumps 57 57
  • python -m pytest tests/ → 2,063 passed, 1 skipped, 2,593 subtests
  • ruff check . and ruff format --check . → clean
  • make typecheck → exit 0, 261 diagnostics, identical to master
  • Diff coverage against e9dfe5f → 100% (9 lines, 0 missing)

Three new cases: that a decoding LSDA is not charged (the condition that keeps the bound off working images — a budget of 1 byte still reaches every table), that an exhausted bound stops calling the reader at all, and that the constant clears the heaviest real image by a stated margin. Verified to fail when the bound is removed.

One thing worth knowing about the shape

The failure has to come before the table's length field to reach the new bound at all. One that fails after it is already charged to the existing budget — which is why the same hostile section built with a too-long length field runs 512 reads rather than 200,000 and needs nothing new. My first attempt at reproducing this used exactly that shape and measured the old bound working, not the gap.

On scope

I said on the issue I would leave this until you picked the number, since sizing it is a corpus judgement. The measurements above make it one: 29.75 MB is the worst real spend, 12.2 GB the hostile one, and any cap between them works. 256 MB is my proposal rather than a finding — change the constant and nothing else moves.

Added after review

An LSDA pointer that reads back empty is now memoised by address. The empty result returns before the decode memo is written, so it is never stored — and because that memo is keyed by (table, function start), storing it there would close only the case where records share a start. Measured on a section naming one dead address from MAX_RECORDS records: 199,999 reads → 1, for both the shared-start and distinct-start shapes. The set is bounded by max_records like the decode memo is, and its worst case is the one it cannot help — a distinct dead address per record, where it holds about 10 MB (17.8 MB peak against 7.4 MB without it) and saves no read.

What the read budget charges is stated, not narrowed. The condition is that the read yielded no pads, which is broader than failing before the table length: a call-site table that parses cleanly and declares none is charged too. Across the bundled fixtures that is one LSDA of 11,656 bytes on elf_cxx_landing_pads_x64_xored and none on the other four, and the 29.75 MB worst real spend already counts it, being the same condition measured.

Three further cases, each verified to fail under a mutation of the line it covers: the dead pointer is read once however many records name it, a second dead address is still tried (a memo, not a latch), and a table that decodes cleanly with no pad is charged. Full suite 2,116 passed, 1 skipped, 2,596 subtests; ruff clean; make typecheck exit 0 at the same diagnostic count as master.

@danielplohmann

Copy link
Copy Markdown
Owner

256 MB taken as proposed. Our reading is that knowing the shape exists matters more than the exact constant, and 8.6× over the heaviest real image with a 48× cut on the hostile one is a reasonable place to stand. The number is now a decision rather than a suggestion, so it should not need revisiting unless a real image gets within an order of magnitude of it.

The path from diagnosis to fix is the part worth crediting: the issue's one-line description was wrong in a way that would have cost recall on ordinary C++ binaries, and the reason — read_va returning whatever the section holds rather than what the table needs — is now written next to the constant. The three tests are the right three, and testAnLsdaThatDecodesIsNotChargedToTheFailedReadBudget passing with max_failed_read_bytes=1 is the one that proves the bound cannot reach a working image.

Two notes, neither blocking

1. The charge condition is broader than the name. if not pads charges every LSDA that yields no pads, which includes one whose call-site table parsed cleanly and genuinely declares none — not only one that failed before its length field. Measured on the bundled fixtures, separating the two by whether the table budget was charged:

fixture decoded cleanly, no pads failed early
elf_cxx_landing_pads_x64_xored 1 (11,656 B) 0
elf_cxx_landing_pads_arm64_xored 0 0
elf_cet_landing_pads_x64_xored 0 0
aarch64_static_xored 0 0

So it is real and it is tiny — 11 KB of 256 MB on the one fixture that has any — and your 29.75 MB worst case presumably already counts these, since you measured the same condition the code implements. Two ways to settle it, and I would take the first: say so in the comment and drop "failed" from how the constant is described in the commit message (the docstring's "LSDAs that then decode to nothing" is already exactly right), or skip the charge when budget moved, which separates the two cases precisely.

2. The same shape one level down, and it predates this PR. An LSDA pointer that reads back empty returns before decoded[key] is set, so it is neither charged nor memoised: MAX_RECORDS records naming one unreadable VA still make 199,999 read_va calls. Each is cheap and none allocates 64 KB, which is why it is a nit rather than the bug you fixed — but memoising the empty result closes it in a line, and it is in the same function.

Green here on top of a043145 with #327 and #329 stacked under it, and this PR is independent of both — it can merge on its own once #329's finding is dealt with, or ahead of them if you would rather rebase once.

@r0ny123

r0ny123 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Both taken, and the second measured further than the note put it.

1. The charge condition is broader than the name — documented, not separated

Your table reproduces exactly here, instrumenting the two cases apart by whether the table budget moved:

fixture decoded cleanly, no pads failed early
elf_cxx_landing_pads_x64_xored 1 (11,656 B) 0
elf_cxx_landing_pads_arm64_xored 0 0
elf_cet_landing_pads_x64_xored 0 0
elf_ehframe_hdr_x64_xored 0 0
aarch64_static_xored 0 0

And your reading of the 29.75 MB is right: it was measured as if not pads, the condition the code implements, so it already counts these.

Taking your first option. The constant's comment now says the charge is on yielding no pads, that this also catches a table which parses cleanly and declares none, what that costs across the fixtures, and that the worst real spend already includes it. testATableThatDecodesAndDeclaresNoPadIsChargedToo pins it — a table that charges the table budget and still declares nothing exhausts the read bound — so the documented condition is the tested one, and the option-2 narrowing fails it rather than passing silently.

The commit body describes the bound as on reads that lead nowhere rather than on reads that failed; the PR title already reads that way.

2. The same shape one level down — closed, and it is not only the repeated-key case

Worth knowing before you take the one-liner: decoded[key] = set() closes half of it. The empty result returns before the memo is written, so the key is never inserted at all — which means the shape costs 199,999 reads whether or not the records share a function start. Measured on a section naming one unreadable address from MAX_RECORDS records:

shape before after
one dead address, distinct initial_location per record 199,999 reads 1
one dead address, same initial_location per record 199,999 reads 1

So the memo has to be keyed by address alone, which is also the honest key: emptiness is a property of the address, not of the function that named it.

The cost is one set bounded by max_records, like decoded is, and its worst case is the one it cannot help — a distinct dead address per record, where it holds about 10 MB and saves no read. Measured at 17.8 MB peak against 7.4 MB without it on that input; the comment says so. Two cases: the dead pointer is read once however many records name it, and a second dead address is still tried, so it is a memo and not a latch.

Verification

  • python -m pytest tests/ — full suite green, figures in the PR body
  • ruff check ., ruff format --check . clean; make typecheck exit 0 with the same diagnostic count as master
  • Each of the three new cases verified to fail under a mutation of the line it covers, and the two pre-existing ones still fail with the bound removed
  • Whole-tree sweep for the class behind note 2 — a memo whose negative result returns before the store: three other sites (DominatorTree.DFS, both ensureCandidate), none of which skips an external read, so none is a sibling

Rebasing over a043145 is your call to make at merge — I have merged master in rather than force-pushing over a branch you have already reviewed.

@r0ny123
r0ny123 force-pushed the fix/lsda-read-budget branch from 65f365f to 57f236c Compare September 11, 2026 08:48
@r0ny123

r0ny123 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Rebased over a043145 too, replacing the merge commit — two linear commits now, and the tree is unchanged by it (git diff against the pre-rebase tip is empty), so the suite, lint and typecheck results reported above still stand as measured.

57f236c fix(common): memoise an LSDA pointer that reads back empty
603ea8c fix(common): bound the LSDA reads that lead nowhere

All three are linear over a043145 now, so this one can go in whenever suits — it touches nothing #327 or #329 do.

@danielplohmann

Copy link
Copy Markdown
Owner

Both notes answered, and the memo in 57f236c is one I had not spotted.

The gap is real: an address the reader hands nothing back for returns before decoded[key] is written, so it is neither charged nor remembered, and a section naming one such address from every record calls the reader once per record. Keying the memo by address alone is right — emptiness is a property of the address, not of the function that named it, so the (lsda_va, initial_location) key could never have caught it. The placement inside if key not in decoded: and ahead of the budget gate is correct: the first read is what pays, and the repeats now cost nothing.

testEachDistinctUnreadablePointerIsStillRead is the test I would have asked for — it pins that this is a memo and not a latch, which is the way this kind of fix usually goes wrong. And the worst case being stated in the comment rather than left implicit (a distinct dead address per record, ~10 MB, saving no read) is the right way to put a bound on a set that has one.

On the charge condition: documenting it beside MAX_LSDA_FAILED_READ_BYTES and at the if not pads site is the proportionate response. The condition genuinely is broader than the constant's name — a call-site table that parses cleanly and declares no pads is charged too — and I measured the same thing on the bundled fixtures: exactly one LSDA, 11,656 bytes, on elf_cxx_landing_pads_x64_xored, none on the other four. Since the 29.75 MB worst real spend was measured under that same condition, the 256 MB bound already accounts for it, so there is nothing to re-measure. testATableThatDecodesAndDeclaresNoPadIsChargedToo pins it so the next reader does not have to rediscover that the name undersells the rule.

256 MB stands as proposed. Knowing the shape exists matters more than the constant, and 8.6x over the heaviest real image is a reasonable place to stand.

Verified stacked over a043145 with #336, #337, #327 and #329: 2,142 passed, 2 skipped, 2,596 subtests; ruff clean; ty exit 0.

No open review items — this one is done bar its ## [Unreleased] entry once #336 and #337 are in. It closes out the last item on #322.

`_fdeLandingPads` checks the call-site table budget before reading an
LSDA, and the comment says that budget "gates the read as well as the
decode". It does not. The budget is charged where a table's length field
is parsed, so an LSDA that fails before that point - an unsupported
LPStart application mode, a short buffer, a TType offset that will not
read - costs a read of up to MAX_LSDA_BYTES and charges nothing at all. A
section naming MAX_RECORDS distinct such pointers reads 12.2 GB over
199,999 reads while spending none of its budget.

Charging those reads to the table budget instead is not the fix. read_va
is asked for MAX_LSDA_BYTES and returns whatever the section holds, so a
real image asks for far more than it decodes - the heaviest cell measured
reads 92 MB to decode 26 KB of tables - and 8 MB would be gone within
about 128 of the 1,473 reads that cell makes, costing landing pads on an
ordinary C++ binary.

Only the failures need a bound, and they separate the two cases. The
heaviest real image spends 29.75 MB on reads that decode to nothing, over
476 of them; the Rust cells spend none; a hostile section spends all of
it. MAX_LSDA_FAILED_READ_BYTES holds that to 256 MB over 4,096 reads,
which is 8.6x headroom over the heaviest real image and a 48x reduction
on the hostile one.

All 447 cells across the built AArch64 ELF, C/C++, Rust, Go, ARM64 Mach-O
and malpedia corpora are bit-identical, which is the assertion that this
reaches only sections built to exhaust it.

The failure has to come before the table's length field to reach the new
bound at all; one that fails after it is already charged, which is why the
same shape built with a too-long length runs 512 reads rather than 200,000
and needs nothing new.
An address the reader hands nothing back for returns before the memo is
written, so the result is neither charged nor remembered. A section naming
one such address from every record therefore calls the reader once per
record -- 199,999 times at MAX_RECORDS, measured, and unchanged whether or
not the records share a function start, since the memo the read skips is
keyed by both. Each call is cheap and none allocates, which is why the
budgets do not see it.

Remember it by address alone: emptiness is a property of the address, not
of the function that named it. The same section now reads once. The set is
bounded by max_records like the decode memo is, and its worst case is the
one it cannot help -- a distinct dead address per record, where it holds
about 10 MB and saves no read.

Also state what the read budget actually charges. The condition is that
the read yielded no pads, which is broader than failing before the table
length: a call-site table that parses cleanly and declares none is charged
too. That case is real and negligible -- one LSDA of 11,656 bytes across
the bundled fixtures, none on four of the five -- and the 29.75 MB worst
real spend already counts it, being the same condition measured. The bound
is on reads that lead nowhere rather than on reads that failed.
Both halves in one bullet, since they are the same defect at two depths:
a read whose result the budgets never see, and a read whose result the
memo never keeps. The bit-identity across all six corpora is the cost
statement -- the bound reaches only sections built to exhaust it.
@r0ny123
r0ny123 force-pushed the fix/lsda-read-budget branch from 57f236c to 95ab01b Compare September 11, 2026 12:10
@r0ny123

r0ny123 commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Rebased? No — merged, since #339 landing under this is a base move rather than anything wrong here. master is merged in and the conflict is resolved: it was ## [Unreleased], where #339's ### Fixed bullet and this one collide by construction. Both kept.

Full suite on the merged tree 2,130 passed, 1 skipped, 2,596 subtests; ruff clean; make typecheck exit 0 at master's 263 diagnostics.

The three stacked PRs all needed the same resolution, and #329 additionally had a real source conflict — #339 and it both add a method immediately above nextGapCandidate in the Intel manager, so git could not order them. Both kept, suite green there too at 2,150.

@danielplohmann

Copy link
Copy Markdown
Owner

Rebased onto 0a0a4c6 and the Unreleased entry is in. Verified the source change is byte-identical to the tree I reviewed — only the changelog entry was added — so the review stands without re-reading it.

The entry itself is the shape #323 asked for: the mechanism, the 12.2 GB over 199,999 reads that motivates the bound, the 256 MB against the 29.75 MB heaviest real image so the headroom is legible, and all 447 cells across the six corpora are bit-identical as the cost line. Nothing to add.

All checks green, including the new Changelog entry gate on its first real engine PR. Merging — this closes out item 6 and #322 with it.

@danielplohmann
danielplohmann merged commit 130fa02 into danielplohmann:master Sep 11, 2026
26 checks passed
@danielplohmann danielplohmann mentioned this pull request Sep 13, 2026
@r0ny123
r0ny123 deleted the fix/lsda-read-budget branch September 14, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants