Skip to content

Hand back a fax as samples, not as the bytes it was compressed into - #4

Merged
tannevaled merged 1 commit into
mainfrom
bump-fax-and-structure
Aug 27, 2026
Merged

Hand back a fax as samples, not as the bytes it was compressed into#4
tannevaled merged 1 commit into
mainfrom
bump-fax-and-structure

Conversation

@tannevaled

@tannevaled tannevaled commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

reader v0.4.2 → v0.5.0. No code change.

reader v0.5.0 decodes /CCITTFaxDecode — Group 3 one-dimensional, Group 3
mixed, Group 4. Images gets its data from reader.Decode, so a scanned
picture used to come back with Filter set to the fax name and Data holding
bytes still compressed. Image.Filter is documented as "empty for data this has
unfiltered into plain samples"; for a fax that is now true.

Measured

Two binaries calling extract.Images on every page, each verified with
go version -m
before running:

old  github.com/go-pdfkit/reader v0.4.2   github.com/go-pdfkit/pdffont v0.3.0
new  github.com/go-pdfkit/reader v0.5.0   github.com/go-pdfkit/pdffont v0.3.0

Corpus: all 2 268 forms in /Users/Shared/pdfforms. Prevalence below is over
the 1 633 real documents; the 635 vendor test fixtures are counted separately.

files
compared 2 268
inventory differs 74
… of the 1 633 real documents 71 (4.3%)
… of the 635 vendor fixtures 3
changed which images it finds 0
open/read status changed 0

By issuer, of the 71 real: fr-cerfa 26, us-opm 23, fr-impots 19,
us-uscourts 2, uk-govuk 1.

2 861 images changed, and the filter name they arrived under says why:

images Filter before after
2 592 /CCF (empty — samples)
269 /CCITTFaxDecode (empty — samples)

Their Data goes from 527 375 bytes still compressed to 12 708 706 bytes of
samples
(24.1×). One image, verbatim from the inventory of
fr-cerfa/cerfa_10134.pdf:

before:  filter=CCITTFaxDecode  bytes=372
after :  filter=              bytes=1892

Nothing else moved. No DCTDecode or JPXDecode image changed — a JPEG is
still handed back as a JPEG, which is what the package promises.

The ImageFilter contract change

reader.ImageFilter no longer answers true for /CCITTFaxDecode or /CCF. I
checked all five consumers of this fleet: nothing calls ImageFilter. The
only two places that switch on a filter name are extract's own
noteImage, which just records whatever Decode reports, and latex's
picture(), which tests for DCTDecode and JPXDecode only and is unaffected.

Not measured

Wall clock and peak memory. Three other corpus jobs were running on this machine,
so any timing figure would be fiction. The output diff above is unaffected by
load.

Gates

go vet, gofmt, exact 100% statement coverage, and all nine cross-compile
targets pass locally with GOWORK=off CGO_ENABLED=0.

reader v0.4.2 -> v0.5.0, which decodes /CCITTFaxDecode: Group 3
one-dimensional, Group 3 mixed, and Group 4.

Images calls reader.Decode, so until now a scanned picture came back with
Filter set to the fax name and Data holding bytes nobody downstream could
read. Now Filter is empty and Data is samples, which is what the field has
always said it means.

Measured over all 2 268 forms in /Users/Shared/pdfforms, both binaries
verified with `go version -m`:

  74 of 2 268 files return a different inventory -- 71 of the 1 633 real
  documents, 3 of the 635 vendor fixtures.
  2 861 images changed: 2 592 named /CCF and 269 /CCITTFaxDecode.
  Their data goes from 527 375 bytes still compressed to 12 708 706 bytes
  of samples.
  Nothing else moved: no DCTDecode or JPXDecode image changed, and no file
  changed which images it finds or whether it opens.

Note for anyone switching on a filter name: reader.ImageFilter no longer
answers true for /CCITTFaxDecode or /CCF. Nothing in this package calls it.
@tannevaled
tannevaled merged commit cd5fadc into main Aug 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant