Turn an Adobe CMYK JPEG's ink over, and read the /Decode that turns it back - #23
Merged
Conversation
…t back
A four-component JPEG written by an Adobe tool stores its ink inverted. libjpeg
turns it back, so poppler, mupdf and pdf.js all show a figure on white paper;
Go's image/jpeg targets JPEG files rather than PDFs and does not, so the page
came out almost solid black. Not slightly wrong — black.
Found by comparing every page of the corpus against poppler at the same size
and looking at the worst disagreement, which was 99.98% of the pixels of one
arXiv figure. 35 of 8 300 files carry a CMYK JPEG, 114 images between them, 95
with the Adobe marker, and 68 of the images decode to near-black.
THE HALF THAT WOULD HAVE BROKEN ELEVEN FORMS
Turning the ink over on its own is measurably wrong. Judged against poppler,
file by file, it repairs 23 and breaks 11 — and the 11 are all DVLA forms:
fig_best_baseline.pdf 0.9990 -> 0.0000 repaired
Vas_ablation_compressed.pdf 0.8002 -> 0.0002 repaired
v890-statutory-off-road-... 0.0083 -> 0.9972 broken
application-for-vehicle-tax-v10 0.0047 -> 0.9993 broken
Their JPEGs are indistinguishable from the others — four components, Adobe
APP14, version 100, transform 2 — and the difference is not in the JPEG at all.
It is in the PDF: those eleven carry /Decode [1 0 1 0 1 0 1 0], which asks for
the samples backwards, and this package ignored /Decode on a DCT image
entirely. Two missing halves that cancelled: eleven forms were drawn correctly
only because both were absent at once, and fixing either alone turns them
black.
So both are here. The ink is turned over, and a wholly inverting /Decode turns
it back.
MEASURED AGAINST POPPLER
the 35 files before: median 0.0211, worst 0.9998, 12 under 1%
after: median 0.0017, worst 0.0218, 32 under 1%, all 35
under 5%
Nothing else moved, and every bucket of the wider corpus is the same or better:
1 633 real forms under 1%: 1 518 -> 1 523 worst 0.1775 -> 0.1775
2 223 arXiv under 1%: 2 104 -> 2 112 worst 0.9998 -> 0.6431
WHAT IS STILL NOT DONE, AND IS SAID RATHER THAN LEFT OUT
/Decode is read here only to answer one question — does it invert every
component — and only for a four-component JPEG. A grey or colour JPEG with a
/Decode array is still drawn as though it had none. No corpus file is written
that way, so there is nothing to measure and nothing has been guessed at.
100% statement coverage, go vet and -race clean, nine cross-compile targets.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A four-component JPEG written by an Adobe tool stores its ink inverted. libjpeg
turns it back, so poppler, mupdf and pdf.js all show a figure on white paper;
Go's image/jpeg targets JPEG files rather than PDFs and does not, so the page
came out almost solid black. Not slightly wrong — black.
Found by comparing every page of the corpus against poppler at the same size
and looking at the worst disagreement, which was 99.98% of the pixels of one
arXiv figure. 35 of 8 300 files carry a CMYK JPEG, 114 images between them, 95
with the Adobe marker, and 68 of the images decode to near-black.
THE HALF THAT WOULD HAVE BROKEN ELEVEN FORMS
Turning the ink over on its own is measurably wrong. Judged against poppler,
file by file, it repairs 23 and breaks 11 — and the 11 are all DVLA forms:
fig_best_baseline.pdf 0.9990 -> 0.0000 repaired
Vas_ablation_compressed.pdf 0.8002 -> 0.0002 repaired
v890-statutory-off-road-... 0.0083 -> 0.9972 broken
application-for-vehicle-tax-v10 0.0047 -> 0.9993 broken
Their JPEGs are indistinguishable from the others — four components, Adobe
APP14, version 100, transform 2 — and the difference is not in the JPEG at all.
It is in the PDF: those eleven carry /Decode [1 0 1 0 1 0 1 0], which asks for
the samples backwards, and this package ignored /Decode on a DCT image
entirely. Two missing halves that cancelled: eleven forms were drawn correctly
only because both were absent at once, and fixing either alone turns them
black.
So both are here. The ink is turned over, and a wholly inverting /Decode turns
it back.
MEASURED AGAINST POPPLER
the 35 files before: median 0.0211, worst 0.9998, 12 under 1%
after: median 0.0017, worst 0.0218, 32 under 1%, all 35
under 5%
Nothing else moved, and every bucket of the wider corpus is the same or better:
1 633 real forms under 1%: 1 518 -> 1 523 worst 0.1775 -> 0.1775
2 223 arXiv under 1%: 2 104 -> 2 112 worst 0.9998 -> 0.6431
WHAT IS STILL NOT DONE, AND IS SAID RATHER THAN LEFT OUT
/Decode is read here only to answer one question — does it invert every
component — and only for a four-component JPEG. A grey or colour JPEG with a
/Decode array is still drawn as though it had none. No corpus file is written
that way, so there is nothing to measure and nothing has been guessed at.
100% statement coverage, go vet and -race clean, nine cross-compile targets.