Write out the scan, instead of a box saying the picture was unreadable - #3
Merged
Conversation
reader v0.4.2 -> v0.5.0, which decodes /CCITTFaxDecode.
picture() hands anything that is not a JPEG or a JPEG 2000 to samples(),
and samples() checks that there are enough bytes to fill the page before it
reads any: `len(data) < stride*h`. A fax arrived still compressed, so there
never were, so every scanned picture in a document came out as
\framebox[41.0pt]{\rule{0pt}{21.0pt}unreadable image}
Now the bytes are samples and the check passes, so the picture is written
out beside the source as a PNG and referred to properly.
Measured over all 2 268 forms in /Users/Shared/pdfforms, both binaries
verified with `go version -m`:
59 of 2 268 files produce different source -- 57 of the 1 633 real
documents. 57 longer, 0 shorter.
Across those 57: \framebox{unreadable image} 291 -> 50, and
\includegraphics 12 -> 253. Two hundred and forty-one placeholders
became figures, one for one.
before: \framebox[41.0pt]{\rule{0pt}{21.0pt}unreadable image}
after : \includegraphics[width=41.0pt,height=21.0pt]{image000.png}
The fifty that are still unreadable are not faxes; they are colour spaces
samples() does not claim to read without a renderer behind it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
readerv0.4.2 → v0.5.0. No code change.What was wrong
picture()hands anything that is not a JPEG or a JPEG 2000 tosamples(), andsamples()refuses data that cannot fill the page before it reads any of it:A
/CCITTFaxDecodeimage arrived still compressed, so there were never enoughbytes, so
picturereturned false andfigureemitted a placeholder. Everyscanned picture in a document came out as a box saying it could not be read.
That guard is the reason this was a missing figure rather than a wrong one — the
same data painted without a length check is noise, which is what
renderv0.10.0had to stop doing.
Measured
Two binaries calling
latex.Source, each verified withgo version -m:Corpus: all 2 268 forms in
/Users/Shared/pdfforms. Prevalence is over the1 633 real documents.
Across those 57 real documents:
\framebox{...unreadable image}\includegraphics241 placeholders became figures, one for one. Verbatim, from
fr-cerfa/cerfa_10983.pdf:and from
fr-cerfa/cerfa_10701.pdf, two on one page:The 50 that are still unreadable are not faxes. They are colour spaces
samples()does not claim to read without a full renderer behind it, and theywere not part of this bump.
Not measured
Wall clock and peak memory: three other corpus jobs were on the machine, so a
timing number would be fiction. The output diff does not depend on load.
Gates
go vet,gofmt, exact 100% statement coverage, nine cross-compile targets, allpass locally with
GOWORK=off CGO_ENABLED=0.