The page that came out black, and the one that came out blank - #7
Merged
Conversation
render v0.11.0 and reader v0.6.0 between them fix two things a person using
this application would have seen, and one they would not have known to look
for.
the black page a four-component JPEG written by an Adobe tool stores its
ink inverted, and Go's image/jpeg does not turn it back, so
the page came out almost solid black. 35 of 8 300 corpus
files carry one. Turning it over alone breaks eleven DVLA
forms, which carry /Decode [1 0 1 0 1 0 1 0] and were drawn
correctly only because that was ignored too; both halves
are fixed together.
the blank page reader v0.6.0 stopped a damaged stream coming back with a
nil error, which made a strict decode return nothing where
it used to return the fragment. render v0.11.0 reads every
stream through the salvaging API, so plots/P12positive01.pdf
goes 188 770 inked pixels -> 0 -> 188 770 across the three
states.
the noise 273 image masks in the real forms carry an encoded filter,
and their compressed bytes were being painted as a stencil.
236 are faxes, now decoded; the 9 that remain are JBIG2 and
are not drawn rather than drawn wrong.
Against poppler, page by page and with both given the same box, this
application's renderer now agrees on 1 633 of 1 633 real forms and 2 223 of
2 223 arXiv files, with a median of 0.19% and 0.08% of pixels differing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
render v0.11.0 and reader v0.6.0 between them fix two things a person using
this application would have seen, and one they would not have known to look
for.
the black page a four-component JPEG written by an Adobe tool stores its
ink inverted, and Go's image/jpeg does not turn it back, so
the page came out almost solid black. 35 of 8 300 corpus
files carry one. Turning it over alone breaks eleven DVLA
forms, which carry /Decode [1 0 1 0 1 0 1 0] and were drawn
correctly only because that was ignored too; both halves
are fixed together.
the blank page reader v0.6.0 stopped a damaged stream coming back with a
nil error, which made a strict decode return nothing where
it used to return the fragment. render v0.11.0 reads every
stream through the salvaging API, so plots/P12positive01.pdf
goes 188 770 inked pixels -> 0 -> 188 770 across the three
states.
the noise 273 image masks in the real forms carry an encoded filter,
and their compressed bytes were being painted as a stencil.
236 are faxes, now decoded; the 9 that remain are JBIG2 and
are not drawn rather than drawn wrong.
Against poppler, page by page and with both given the same box, this
application's renderer now agrees on 1 633 of 1 633 real forms and 2 223 of
2 223 arXiv files, with a median of 0.19% and 0.08% of pixels differing.