Read a page as far as it decoded, and still refuse one that did not - #5
Merged
Conversation
… not reader v0.6.0 keeps what a broken filter chain did produce and says so, instead of returning a fragment with a nil error. Taking it without deciding what this package wants would have changed the contract by accident in both directions, so the decision is made here and written down. A page whose content decoded part of the way is now read that far. 263 streams in 212 of the 1 633 real forms — 13.0% of them — cannot be decoded cleanly, and half a page of text is worth more to somebody searching than none of it. A page that decoded no bytes at all stays an error. That is not symmetry for its own sake: go-pdfkit/latex turns a document into LaTeX, and a conversion that silently produces an empty document from an unreadable page is worse than one that says it could not read it. Its own test caught this the moment the dependency moved, which is the test doing its job. The pair of tests states both halves. A stream of 108 bytes holding 301 pieces of text, cut to 80, gives 147 of them and no error; the same stream declared as Flate and holding no Flate at all still gives an error. Also on the fixes this brings: the /Crypt filter, which 209 files — 12.8% of the real forms — carry and which used to make 209 of them decode to nothing readable; a second map-order non-determinism, in indexObjectStreams; and files whose crypt filters say /Identity, which need no password and now open. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
reader v0.6.0 keeps what a broken filter chain did produce and says so, instead
of returning a fragment with a nil error. Taking it without deciding what this
package wants would have changed the contract by accident in both directions,
so the decision is made here and written down.
A page whose content decoded part of the way is now read that far. 263 streams
in 212 of the 1 633 real forms — 13.0% of them — cannot be decoded cleanly, and
half a page of text is worth more to somebody searching than none of it.
A page that decoded no bytes at all stays an error. That is not symmetry for
its own sake: go-pdfkit/latex turns a document into LaTeX, and a conversion
that silently produces an empty document from an unreadable page is worse than
one that says it could not read it. Its own test caught this the moment the
dependency moved, which is the test doing its job.
The pair of tests states both halves. A stream of 108 bytes holding 301 pieces
of text, cut to 80, gives 147 of them and no error; the same stream declared as
Flate and holding no Flate at all still gives an error.
Also on the fixes this brings: the /Crypt filter, which 209 files — 12.8% of
the real forms — carry and which used to make 209 of them decode to nothing
readable; a second map-order non-determinism, in indexObjectStreams; and files
whose crypt filters say /Identity, which need no password and now open.