Draw what a damaged stream did give - #22
Merged
Merged
Conversation
reader v0.6.0 stopped flateDecode from lying: a damaged stream's inflated prefix used to come back with a nil error, so a caller could not tell a clean decode from a fragment. Fixing that makes the strict Decode this package used return nothing where it used to return the fragment — so taking the new reader without changing anything here is a REGRESSION, and the measurement says so: plots/P12positive01.pdf 188 770 inked pixels -> 0 (a blank page) uk-govuk/apply-for-help-... 48 455 -> 38 695 uk-govuk/form-n1 (x3) 42 347 -> 40 771, and two like it Every stream is now read through DecodeStreamRecovering, and every one of those pages comes back exactly as it was: 188 770 -> 0 -> 188 770. The recovery was already happening; it was happening by accident, inside a function that claimed success. Now it is asked for. WHY THIS IS THE RIGHT BEHAVIOUR AND NOT MERELY THE OLD ONE 263 streams in 212 of the 1 633 real forms — 13.0% of them — cannot be decoded cleanly. This package already draws a page as far as it got when its time runs out; a damaged stream is that same situation arriving another way, and half a figure is worth more to somebody reading a form than none of it. WHAT MAKES IT SAFE Bytes that no filter decoded are never painted. reader.Decoded keeps them in Undecoded, set INSTEAD of Data rather than beside it, so reading Data cannot reach them — a type doing the work rather than a flag anyone has to remember. That guarantee is why this change is possible at all: the same reader release found that a fax with a bad /Columns had been putting its encoded bytes where a caller would paint them, which is precisely the defect #20 had just fixed here from the other side. Two cases are still refused rather than salvaged, and the tests say so: a page whose /Contents is filtered as an image, which would run a JPEG as operators; and a stream whose filter nothing can apply, which yields no decoded bytes at all. MEASURED, THE THREE STATES SEPARATED reader v0.5.0, then v0.6.0 alone, then v0.6.0 with this change, over the first page of all 1 633 real forms and 1 334 arXiv files, each configuration run twice and identical to itself both times: real forms v0.5.0 -> v0.6.0: 6 pages change, 5 of them with less ink v0.6.0 -> +this: 4 change, all 4 back to what v0.5.0 drew v0.5.0 -> +this: 2 change, both by under thirty pixels arXiv v0.5.0 -> v0.6.0: 4 change, one of them to a blank page v0.6.0 -> +this: 3 change, the blank page back to 188 770 v0.5.0 -> +this: 2 change One arXiv figure disagrees in the other direction — 2512.03312/Fig3.pdf draws 24 913 pixels under v0.5.0, 38 525 under v0.6.0 alone, and 24 913 again with this. Which of the two is right has not been established, and it is written down here rather than left out. 100% statement coverage, go vet and -race clean, nine cross-compile targets. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
reader v0.6.0 stopped flateDecode from lying: a damaged stream's inflated
prefix used to come back with a nil error, so a caller could not tell a clean
decode from a fragment. Fixing that makes the strict Decode this package used
return nothing where it used to return the fragment — so taking the new reader
without changing anything here is a REGRESSION, and the measurement says so:
plots/P12positive01.pdf 188 770 inked pixels -> 0 (a blank page)
uk-govuk/apply-for-help-... 48 455 -> 38 695
uk-govuk/form-n1 (x3) 42 347 -> 40 771, and two like it
Every stream is now read through DecodeStreamRecovering, and every one of those
pages comes back exactly as it was: 188 770 -> 0 -> 188 770. The recovery was
already happening; it was happening by accident, inside a function that claimed
success. Now it is asked for.
WHY THIS IS THE RIGHT BEHAVIOUR AND NOT MERELY THE OLD ONE
263 streams in 212 of the 1 633 real forms — 13.0% of them — cannot be decoded
cleanly. This package already draws a page as far as it got when its time runs
out; a damaged stream is that same situation arriving another way, and half a
figure is worth more to somebody reading a form than none of it.
WHAT MAKES IT SAFE
Bytes that no filter decoded are never painted. reader.Decoded keeps them in
Undecoded, set INSTEAD of Data rather than beside it, so reading Data cannot
reach them — a type doing the work rather than a flag anyone has to remember.
That guarantee is why this change is possible at all: the same reader release
found that a fax with a bad /Columns had been putting its encoded bytes where a
caller would paint them, which is precisely the defect #20 had just fixed here
from the other side.
Two cases are still refused rather than salvaged, and the tests say so: a page
whose /Contents is filtered as an image, which would run a JPEG as operators;
and a stream whose filter nothing can apply, which yields no decoded bytes at
all.
MEASURED, THE THREE STATES SEPARATED
reader v0.5.0, then v0.6.0 alone, then v0.6.0 with this change, over the first
page of all 1 633 real forms and 1 334 arXiv files, each configuration run
twice and identical to itself both times:
real forms v0.5.0 -> v0.6.0: 6 pages change, 5 of them with less ink
v0.6.0 -> +this: 4 change, all 4 back to what v0.5.0 drew
v0.5.0 -> +this: 2 change, both by under thirty pixels
arXiv v0.5.0 -> v0.6.0: 4 change, one of them to a blank page
v0.6.0 -> +this: 3 change, the blank page back to 188 770
v0.5.0 -> +this: 2 change
One arXiv figure disagrees in the other direction — 2512.03312/Fig3.pdf draws
24 913 pixels under v0.5.0, 38 525 under v0.6.0 alone, and 24 913 again with
this. Which of the two is right has not been established, and it is written
down here rather than left out.
100% statement coverage, go vet and -race clean, nine cross-compile targets.