Read the ink layer of a scanned page, and stop showing its negative

[tannevaled]

Two things, because the first exposed the second and neither is right alone.

JBIG2 is decoded

A modern scanned page is two images: a low-resolution colour background stored as JPEG 2000, and a high-resolution bitonal ink layer over it stored as JBIG2. We could read the background and not the ink.

JBIG2 had been set aside on a measurement that counted filters by what they encode as page content — by that count it was in 20 documents of 3 217 and left three pages blank. It is almost never the content. It is the /Mask. Counted as what it shapes, it is in 4 089 images across the same corpus.

The mask was inside out

A stencil mask paints where its sample is 0. applyStencilMask cleared the alpha where the stencil painted rather than where it did not, so every explicit mask in the corpus showed the exact complement of itself — on a scanned page, the ink is the only part that was hidden. The test covering it asserted the same inversion, so the suite was green.

Asked which half of a two-colour page a mask of eight 0 bits and eight 1 bits paints, poppler answers the 0 half. So does this now, for a raw mask and a JBIG2 one alike.

The defect had been invisible because two halves cancelled: masks that could not be decoded took their image with them (#26), and the ones that could were drawn inside out. Decoding JBIG2 without fixing the mask made 5 of 25 medical scans worse and 2 catastrophic — 98% of pixels wrong on the worst. That is how it was found: the A/B against poppler, not the test suite.

Measured

First page of 25 medical scans, against poppler:

	before	after
improved / unchanged / worse	—	11 / 14 / 0
mean differing pixels	0.0083	0.0004
worst page	0.1087	0.0004

Which decoder

No decoder is written here. Four pure-Go candidates exist and all four advertise full segment coverage. I extracted 403 JBIG2 masks from real scans and compared every pixel against poppler's own decoder:

decoder	decoded	identical to poppler
dkrisman/gobig2	396/403	396 — 100%
jdeng/gojbig2	399/403	37 — 9%
dobbo-ca/byblos	157/403	157 — 100% (generic region only; the rest is in internal/)
xiaoqidun/jbig2	0/403	—

The one that decodes the most is wrong nearly everywhere. Counting successes would have picked it.

gobig2's seven refusals are a resource budget, not a defect: it caps a symbol at 4 megapixels and real scans carry symbols up to 8. This takes it at its default — a stream it declines is a stream not drawn, which is the rule the rest of the file follows. Its limits are process-global, so they are not raised from inside a library.

The same decoder is added to go-gfx/gfx/codec in go-gfx/gfx#23, for the standalone file form.

Gate

100% statement coverage, go vet and -race clean, nine cross-compile targets, CGO_ENABLED=0. The test fixture is synthetic, written out as bytes rather than committed, and round-trips through three independent implementations.

🤖 Generated with Claude Code