tounicode: bound how many codes one map may name

[tannevaled]

Found by a robustness campaign against mozilla's pdf.js adversarial corpus (977 files), 2 268 public forms from 18 issuers (IRS, SSA, OPM, DOL, USCIS, US Courts, GOV.UK, CRA, CERFA, impots.gouv.fr, WIPO — plus the deliberately broken suites of veraPDF, qpdf, pdfbox, pypdf, pdfcpu and openpdf), and 33 628 arXiv figure PDFs.

The bomb

A bfrange names a run of codes in about twenty bytes:

<00000000><0000ffff><0041>

readBFRange caps the width of a single run at 65 536. Nothing capped how many runs a map could hold. So the size of the answer had nothing to do with the size of the question:

input	entries	time	memory
585 bytes	655 360	2.3 s	63 MB
2 705 bytes	3 276 800	14.6 s	264 MB
10 655 bytes	13 107 200	10.3 s	1 054 MB

A ~104 000x amplification by memory, and easy to reach: pdffont.Read then readToUnicode then ReadToUnicode runs for every font on every page, and extract reads every font on every page. A 10 KB font object costs a gigabyte; a document with twenty such fonts costs twenty.

After

input	entries	time	memory
585 bytes	262 144	47 ms	31 MB
2 705 bytes	262 144	52 ms	31 MB
10 655 bytes	262 144	52 ms	31 MB
106 055 bytes	262 144	48 ms	34 MB

The cost now stops growing: two hundred times the input buys the same answer.

The bound is measured, not guessed

Across 9 183 distinct /ToUnicode maps harvested from two independent corpora (deduplicated by content hash):

	entries
median	20
p90	76
p99	260
p99.9	65 536
max	65 536
maps above 65 536	0

The two corpora agree separately as well: 5 338 maps from pdf.js + arXiv (median 13, max 65 536) and 4 378 from the government-forms corpus (median 35, max 65 536). Neither produced a single map above 65 536, which is one whole two-byte code space — as many codes as a font of that shape can have.

The bound is 1 << 18, four times the largest map either corpus produced, so a document has to be malformed or hostile to reach it.

All 5 338 of the first corpus come back byte for byte identical. The A/B compares the sorted key/value pairs of every map before and after; the diff is empty.

The regression test

Against the parent commit, TestToUnicodeIsBoundedByItsOwnSize reports 585 bytes named 655360 codes, over the 262144 bound — taking 229 seconds to get there — and TestToUnicodeCostIsBounded then times the whole suite out at five minutes:

--- FAIL: TestToUnicodeIsBoundedByItsOwnSize (229.57s)
    585 bytes named 655360 codes, over the 262144 bound
panic: test timed out after 5m0s
	running tests:
		TestToUnicodeCostIsBounded (1m10s)

Here the whole package runs in 0.66 s.

The tests cover both shapes the attack takes — many blocks of one range each, and one block naming many — plus a block entered when the map is already full, and check that the maps anybody actually has still read: a full two-byte code space still comes back with all 65 536 codes, and a range still counts on from its first character.

Second commit: a stale dependency shipping a known denial of service

reader v0.4.1 stopped answering "which objects call themselves a catalogue?" by counting from zero to the largest object number a file names. This package was still asking for v0.4.0, so it still handed its callers the defect.

bug1980958.pdf in the pdf.js corpus is 219 bytes. No trailer, no startxref — so it can only be read by repairing it — and the last object it declares is numbered 2 147 483 647. Two thousand million map lookups for four objects:

reader.Open on 219 bytes   21.2 s, 0.0 MB allocated  ->  under 1 ms

Not one byte allocated, which is why no memory limit anywhere caught it. Measured with GOMAXPROCS(1): a goroutine waiting for a core cannot tell waiting from working.

TestATinyFileWithAHugeObjectNumber builds the file rather than committing somebody else's, and guards the dependency rather than this package's own code. It fails in 15.1 s against the parent commit.

Fuzz targets

There were none in this repository. This adds FuzzReadToUnicode and FuzzRuneOfGlyphName. Both assert a time budget as well as absence of panic, because a cost that grows without the input growing raises nothing on its own — which is exactly how this defect survived. Point CMAP_SEEDS at a directory of real CMaps to seed from a corpus; without it the built-in seeds and anything under testdata still run.

What else was looked for and not found

• All 9 183 harvested CMaps run through ReadToUnicode serially: worst case 10.6 MB from a 219 KB input, nothing over 54 ms, no panics. (Measured one at a time — an earlier concurrent run made small maps look expensive, because runtime.MemStats.TotalAlloc is process-wide.)
• 2 268 forms / 133 241 fields through pdffont.Read plus Width, Text, GlyphName and CIDToGID over the code space: no panics, no timeouts.
• pdffont.Read's lookups allocate nothing — measured at zero bytes. An earlier 823 MB figure turned out to be the harness calling 16.8 million of them without caching, not the library.

Gates

• go test ./... green, 100.0% of statements
• go vet clean, gofmt clean, CGO_ENABLED=0
• Builds for linux amd64/arm64/riscv64/loong64/ppc64le/s390x, js/wasm, darwin/arm64, windows/amd64

🤖 Generated with Claude Code