Repository navigation
fix(common): bound the LSDA reads that lead nowhere - #335
Conversation
|
256 MB taken as proposed. Our reading is that knowing the shape exists matters more than the exact constant, and 8.6× over the heaviest real image with a 48× cut on the hostile one is a reasonable place to stand. The number is now a decision rather than a suggestion, so it should not need revisiting unless a real image gets within an order of magnitude of it. The path from diagnosis to fix is the part worth crediting: the issue's one-line description was wrong in a way that would have cost recall on ordinary C++ binaries, and the reason — Two notes, neither blocking1. The charge condition is broader than the name.
So it is real and it is tiny — 11 KB of 256 MB on the one fixture that has any — and your 29.75 MB worst case presumably already counts these, since you measured the same condition the code implements. Two ways to settle it, and I would take the first: say so in the comment and drop "failed" from how the constant is described in the commit message (the docstring's "LSDAs that then decode to nothing" is already exactly right), or skip the charge when 2. The same shape one level down, and it predates this PR. An LSDA pointer that reads back empty returns before Green here on top of |
|
Both taken, and the second measured further than the note put it. 1. The charge condition is broader than the name — documented, not separatedYour table reproduces exactly here, instrumenting the two cases apart by whether the table budget moved:
And your reading of the 29.75 MB is right: it was measured as Taking your first option. The constant's comment now says the charge is on yielding no pads, that this also catches a table which parses cleanly and declares none, what that costs across the fixtures, and that the worst real spend already includes it. The commit body describes the bound as on reads that lead nowhere rather than on reads that failed; the PR title already reads that way. 2. The same shape one level down — closed, and it is not only the repeated-key caseWorth knowing before you take the one-liner:
So the memo has to be keyed by address alone, which is also the honest key: emptiness is a property of the address, not of the function that named it. The cost is one set bounded by Verification
Rebasing over |
65f365f to
57f236c
Compare
|
Rebased over All three are linear over |
|
Both notes answered, and the memo in The gap is real: an address the reader hands nothing back for returns before
On the charge condition: documenting it beside 256 MB stands as proposed. Knowing the shape exists matters more than the constant, and 8.6x over the heaviest real image is a reasonable place to stand. Verified stacked over No open review items — this one is done bar its |
`_fdeLandingPads` checks the call-site table budget before reading an LSDA, and the comment says that budget "gates the read as well as the decode". It does not. The budget is charged where a table's length field is parsed, so an LSDA that fails before that point - an unsupported LPStart application mode, a short buffer, a TType offset that will not read - costs a read of up to MAX_LSDA_BYTES and charges nothing at all. A section naming MAX_RECORDS distinct such pointers reads 12.2 GB over 199,999 reads while spending none of its budget. Charging those reads to the table budget instead is not the fix. read_va is asked for MAX_LSDA_BYTES and returns whatever the section holds, so a real image asks for far more than it decodes - the heaviest cell measured reads 92 MB to decode 26 KB of tables - and 8 MB would be gone within about 128 of the 1,473 reads that cell makes, costing landing pads on an ordinary C++ binary. Only the failures need a bound, and they separate the two cases. The heaviest real image spends 29.75 MB on reads that decode to nothing, over 476 of them; the Rust cells spend none; a hostile section spends all of it. MAX_LSDA_FAILED_READ_BYTES holds that to 256 MB over 4,096 reads, which is 8.6x headroom over the heaviest real image and a 48x reduction on the hostile one. All 447 cells across the built AArch64 ELF, C/C++, Rust, Go, ARM64 Mach-O and malpedia corpora are bit-identical, which is the assertion that this reaches only sections built to exhaust it. The failure has to come before the table's length field to reach the new bound at all; one that fails after it is already charged, which is why the same shape built with a too-long length runs 512 reads rather than 200,000 and needs nothing new.
An address the reader hands nothing back for returns before the memo is written, so the result is neither charged nor remembered. A section naming one such address from every record therefore calls the reader once per record -- 199,999 times at MAX_RECORDS, measured, and unchanged whether or not the records share a function start, since the memo the read skips is keyed by both. Each call is cheap and none allocates, which is why the budgets do not see it. Remember it by address alone: emptiness is a property of the address, not of the function that named it. The same section now reads once. The set is bounded by max_records like the decode memo is, and its worst case is the one it cannot help -- a distinct dead address per record, where it holds about 10 MB and saves no read. Also state what the read budget actually charges. The condition is that the read yielded no pads, which is broader than failing before the table length: a call-site table that parses cleanly and declares none is charged too. That case is real and negligible -- one LSDA of 11,656 bytes across the bundled fixtures, none on four of the five -- and the 29.75 MB worst real spend already counts it, being the same condition measured. The bound is on reads that lead nowhere rather than on reads that failed.
Both halves in one bullet, since they are the same defect at two depths: a read whose result the budgets never see, and a read whose result the memo never keeps. The bit-identity across all six corpora is the cost statement -- the bound reaches only sections built to exhaust it.
57f236c to
95ab01b
Compare
# Conflicts: # CHANGELOG.md
|
Rebased? No — merged, since #339 landing under this is a base move rather than anything wrong here. Full suite on the merged tree 2,130 passed, 1 skipped, 2,596 subtests; The three stacked PRs all needed the same resolution, and #329 additionally had a real source conflict — #339 and it both add a method immediately above |
|
Rebased onto The entry itself is the shape #323 asked for: the mechanism, the 12.2 GB over 199,999 reads that motivates the bound, the 256 MB against the 29.75 MB heaviest real image so the headroom is legible, and all 447 cells across the six corpora are bit-identical as the cost line. Nothing to add. All checks green, including the new |
Item 6 of #322.
The diagnosis in the issue is right
_fdeLandingPadschecks the table budget before reading, and the comment there says that budget "gates the read as well as the decode". It does not. The budget is charged where a table's length field is parsed, so an LSDA failing before that point — an unsupported LPStart application mode, a short buffer, a TType offset that will not read — costs a read of up toMAX_LSDA_BYTESand charges nothing.Measured: a section naming
MAX_RECORDSdistinct such pointers performs 199,999 reads handing back 12.2 GB, spending none of its budget.But the fix as described would cost recall
"Charge the read at the point it happens" cannot mean charging it to
MAX_LSDA_TABLE_BYTES.read_vais asked forMAX_LSDA_BYTESand returns whatever the section holds, so a real image asks for vastly more than it decodes:googletest_gcc-x64_O2-staticgoogletest_gcc-arm64_O2-staticfmtheavy_linux-gnu-x64_debugThe 8 MB budget would be gone within about 128 of that first cell's 1,473 reads, and every LSDA after that reads as "declares nothing" — landing pads lost on an ordinary C++ binary. The existing comment's "counts what is decoded rather than what is read" is not a wording slip; it is why the current bound works.
What separates the two cases
Only the reads that lead nowhere. Those are rare on real images and universal on hostile ones:
MAX_LSDA_FAILED_READ_BYTES = 256 MBholds the hostile shape to 4,096 reads and 256 MB — a 48× reduction — with 8.6× headroom over the heaviest real image measured.Verification
All 447 cells across all six corpora are bit-identical, which is the assertion that this reaches only sections built to exhaust it:
python -m pytest tests/→ 2,063 passed, 1 skipped, 2,593 subtestsruff check .andruff format --check .→ cleanmake typecheck→ exit 0, 261 diagnostics, identical to mastere9dfe5f→ 100% (9 lines, 0 missing)Three new cases: that a decoding LSDA is not charged (the condition that keeps the bound off working images — a budget of 1 byte still reaches every table), that an exhausted bound stops calling the reader at all, and that the constant clears the heaviest real image by a stated margin. Verified to fail when the bound is removed.
One thing worth knowing about the shape
The failure has to come before the table's length field to reach the new bound at all. One that fails after it is already charged to the existing budget — which is why the same hostile section built with a too-long length field runs 512 reads rather than 200,000 and needs nothing new. My first attempt at reproducing this used exactly that shape and measured the old bound working, not the gap.
On scope
I said on the issue I would leave this until you picked the number, since sizing it is a corpus judgement. The measurements above make it one: 29.75 MB is the worst real spend, 12.2 GB the hostile one, and any cap between them works. 256 MB is my proposal rather than a finding — change the constant and nothing else moves.
Added after review
An LSDA pointer that reads back empty is now memoised by address. The empty result returns before the decode memo is written, so it is never stored — and because that memo is keyed by
(table, function start), storing it there would close only the case where records share a start. Measured on a section naming one dead address fromMAX_RECORDSrecords: 199,999 reads → 1, for both the shared-start and distinct-start shapes. The set is bounded bymax_recordslike the decode memo is, and its worst case is the one it cannot help — a distinct dead address per record, where it holds about 10 MB (17.8 MB peak against 7.4 MB without it) and saves no read.What the read budget charges is stated, not narrowed. The condition is that the read yielded no pads, which is broader than failing before the table length: a call-site table that parses cleanly and declares none is charged too. Across the bundled fixtures that is one LSDA of 11,656 bytes on
elf_cxx_landing_pads_x64_xoredand none on the other four, and the 29.75 MB worst real spend already counts it, being the same condition measured.Three further cases, each verified to fail under a mutation of the line it covers: the dead pointer is read once however many records name it, a second dead address is still tried (a memo, not a latch), and a table that decodes cleanly with no pad is charged. Full suite 2,116 passed, 1 skipped, 2,596 subtests;
ruffclean;make typecheckexit 0 at the same diagnostic count as master.