Skip to content

Draw a scanned page - #25

Merged
tannevaled merged 1 commit into
mainfrom
feat/draw-a-scanned-page
Aug 28, 2026
Merged

Draw a scanned page#25
tannevaled merged 1 commit into
mainfrom
feat/draw-a-scanned-page

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

A page whose only content is a JPEG 2000 image came out blank, and that is what
a scanned page is. Over the 250 biodiversity scans, rendering every page of
every document:

730 of 747 pages go from blank to drawn
0 pages lose anything
17 unchanged, being the pages that had something else on them

The 17 are the true control: nothing that was drawing before draws differently.

WHY THIS WAS NEARLY NOT DONE

Measured over the corpora this project had — government forms and arXiv figures
— JPEG 2000 is eleven files in sixteen hundred, and the decision taken on that
number was that it was not worth a decoder. Extending the corpus to where a
scanned page actually lives says otherwise: all 250 biodiversity scans carry
one, 248 of the 250 medical ones, and 144 of the 222 readable scanned books.
The old figure measured the corpus, not the world.

THE DECODER IS NOT WRITTEN HERE, AND NOT WRITTEN AT ALL

github.com/ajroetker/go-jpeg2000 is 20 050 lines of pure Go with no cgo, and it
decodes 1 610 of 1 610 real images taken out of eighty scanned books. It is
also registered in go-gfx/gfx/codec, which is where the fleet's codec choices
are made once.

THIS TAKES THE DECODER, NOT THE REGISTRY, AND THE REASON IS A NUMBER

go-gfx/gfx/codec brings TIFF, WEBP, BMP and ICO with it, and a PDF renderer has
no use for any of them. One of this package's consumers is a wasm binary a
browser downloads:

today 4.17 MB
the decoder on its own 5.35 MB
the whole registry 6.25 MB

Nine hundred kilobytes for four decoders nothing here calls. The codec choice
is still mutualised — the same reference library, chosen once in go-gfx — but
what is imported is the one format this package can be asked for.

WHAT IT COSTS

Decoding is real work, and the figure that looked good before was measuring a
blank page: those 747 pages took 1.1 seconds when nothing was drawn and 2
minutes 30 seconds when they were, which is about a fifth of a second for a
full-page scan.

TWO SMALLER THINGS

The size comes from the codestream rather than from the dictionary, as it does
for JPEG: a codestream carries its own, and where they disagree the one the
pixels are in is the one that can be drawn.

The "a format nothing here decodes" branch is now covered by JBIG2 rather than
by JPEG 2000, which is the honest representative of what is left: 12 of 250
court filings and 1 of 222 scanned books carry it, and it draws nothing rather
than drawing noise.

100% statement coverage, go vet and -race clean, nine cross-compile targets.

A page whose only content is a JPEG 2000 image came out blank, and that is what
a scanned page is. Over the 250 biodiversity scans, rendering every page of
every document:

  730 of 747 pages go from blank to drawn
    0 pages lose anything
   17 unchanged, being the pages that had something else on them

The 17 are the true control: nothing that was drawing before draws differently.

WHY THIS WAS NEARLY NOT DONE

Measured over the corpora this project had — government forms and arXiv figures
— JPEG 2000 is eleven files in sixteen hundred, and the decision taken on that
number was that it was not worth a decoder. Extending the corpus to where a
scanned page actually lives says otherwise: all 250 biodiversity scans carry
one, 248 of the 250 medical ones, and 144 of the 222 readable scanned books.
The old figure measured the corpus, not the world.

THE DECODER IS NOT WRITTEN HERE, AND NOT WRITTEN AT ALL

github.com/ajroetker/go-jpeg2000 is 20 050 lines of pure Go with no cgo, and it
decodes 1 610 of 1 610 real images taken out of eighty scanned books. It is
also registered in go-gfx/gfx/codec, which is where the fleet's codec choices
are made once.

THIS TAKES THE DECODER, NOT THE REGISTRY, AND THE REASON IS A NUMBER

go-gfx/gfx/codec brings TIFF, WEBP, BMP and ICO with it, and a PDF renderer has
no use for any of them. One of this package's consumers is a wasm binary a
browser downloads:

  today                     4.17 MB
  the decoder on its own    5.35 MB
  the whole registry        6.25 MB

Nine hundred kilobytes for four decoders nothing here calls. The codec choice
is still mutualised — the same reference library, chosen once in go-gfx — but
what is imported is the one format this package can be asked for.

WHAT IT COSTS

Decoding is real work, and the figure that looked good before was measuring a
blank page: those 747 pages took 1.1 seconds when nothing was drawn and 2
minutes 30 seconds when they were, which is about a fifth of a second for a
full-page scan.

TWO SMALLER THINGS

The size comes from the codestream rather than from the dictionary, as it does
for JPEG: a codestream carries its own, and where they disagree the one the
pixels are in is the one that can be drawn.

The "a format nothing here decodes" branch is now covered by JBIG2 rather than
by JPEG 2000, which is the honest representative of what is left: 12 of 250
court filings and 1 of 222 scanned books carry it, and it draws nothing rather
than drawing noise.

100% statement coverage, go vet and -race clean, nine cross-compile targets.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 1ea00aa into main Aug 28, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant