Hand back a fax as samples, not as the bytes it was compressed into - #4
Merged
Conversation
reader v0.4.2 -> v0.5.0, which decodes /CCITTFaxDecode: Group 3 one-dimensional, Group 3 mixed, and Group 4. Images calls reader.Decode, so until now a scanned picture came back with Filter set to the fax name and Data holding bytes nobody downstream could read. Now Filter is empty and Data is samples, which is what the field has always said it means. Measured over all 2 268 forms in /Users/Shared/pdfforms, both binaries verified with `go version -m`: 74 of 2 268 files return a different inventory -- 71 of the 1 633 real documents, 3 of the 635 vendor fixtures. 2 861 images changed: 2 592 named /CCF and 269 /CCITTFaxDecode. Their data goes from 527 375 bytes still compressed to 12 708 706 bytes of samples. Nothing else moved: no DCTDecode or JPXDecode image changed, and no file changed which images it finds or whether it opens. Note for anyone switching on a filter name: reader.ImageFilter no longer answers true for /CCITTFaxDecode or /CCF. Nothing in this package calls it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
readerv0.4.2 → v0.5.0. No code change.reader v0.5.0 decodes
/CCITTFaxDecode— Group 3 one-dimensional, Group 3mixed, Group 4.
Imagesgets its data fromreader.Decode, so a scannedpicture used to come back with
Filterset to the fax name andDataholdingbytes still compressed.
Image.Filteris documented as "empty for data this hasunfiltered into plain samples"; for a fax that is now true.
Measured
Two binaries calling
extract.Imageson every page, each verified withgo version -mbefore running:Corpus: all 2 268 forms in
/Users/Shared/pdfforms. Prevalence below is overthe 1 633 real documents; the 635 vendor test fixtures are counted separately.
By issuer, of the 71 real:
fr-cerfa26,us-opm23,fr-impots19,us-uscourts2,uk-govuk1.2 861 images changed, and the filter name they arrived under says why:
Filterbefore/CCF/CCITTFaxDecodeTheir
Datagoes from 527 375 bytes still compressed to 12 708 706 bytes ofsamples (24.1×). One image, verbatim from the inventory of
fr-cerfa/cerfa_10134.pdf:Nothing else moved. No
DCTDecodeorJPXDecodeimage changed — a JPEG isstill handed back as a JPEG, which is what the package promises.
The
ImageFiltercontract changereader.ImageFilterno longer answers true for/CCITTFaxDecodeor/CCF. Ichecked all five consumers of this fleet: nothing calls
ImageFilter. Theonly two places that switch on a filter name are
extract's ownnoteImage, which just records whateverDecodereports, andlatex'spicture(), which tests forDCTDecodeandJPXDecodeonly and is unaffected.Not measured
Wall clock and peak memory. Three other corpus jobs were running on this machine,
so any timing figure would be fiction. The output diff above is unaffected by
load.
Gates
go vet,gofmt, exact 100% statement coverage, and all nine cross-compiletargets pass locally with
GOWORK=off CGO_ENABLED=0.