feat(photocd): read Kodak Photo CD, the first format the bit-field decode unlocked - #659
Merged
Merged
Conversation
…code unlocked delta at exactly zero, because ~980 of them sit in tables no caller looks up: only about twenty of the 612 generated tables are reachable through `find_table` at all. That made the change capability rather than coverage, and said so. This spends some of it. `PhotoCD::Main` is the highest-value unreached table in the corpus: it declares 24 fields and ExifTool reports all 24 on `combined-samples/PhotoCD.pcd`, where OxiDex reported none. PCD was detected but unparsed -- `File:FileType: PCD` beside no PhotoCD group whatsoever, the failure mode that looks healthy in `oxidex -j` and is invisible to a bare-name comparison. Three of those tags are why this format waited for #640. `ImageWidth`, `ImageHeight` and `CompressionClass` are `1538.1`, `1538.2` and `1538.3` -- fractional entries sharing byte 1538 with `Orientation`, each masking a different slice of it. `decode_binary_table` refused every fractional entry before #640, so the table could not have been read from at all. The table alone is still not enough, and the difference matters: nine of its fields carry a `ValueConv`, three a `RawConv` and five a `Condition`, none of which the transcription runs. Each is implemented here against the Perl it reproduces, cited by line, rather than skipped: * `int8u[2]` version pairs are `tr/ /./`, a join -- `7 61` is release "7.61". Read as a number it would print 7.61 by luck and turn "1.0" into "1". A pair of 255s is `RawConv`'d to "n/a". * `ScannerPixelSize` is `join(".",unpack("H2H2",$val))`: hex digits, so `0x11 0x48` is 11.48 micrometers and not 17.72. * `CreateDate`/`ModifyDate` are `ConvertUnixTime($val,1)` -- local civil time with the offset in force at that instant, the same rendering the File group already uses -- with 0xffffffff meaning absent, not 2106. * `ImageWidth`/`ImageHeight` are `($$self{Orient} & 0x01 ? 512 : 768) * ($val * 2 || 1)`. The DataMember comes from `Orientation`'s RawConv, and Perl's `||` makes resolution code 0 a multiplier of one rather than collapsing the image to nothing. * The five scene-balance tags are `Condition => '$$self{HasSBA}'`, gated on the literal bytes "SBA" at 225; `CopyrightFileName` additionally needs `CopyrightStatus == 1`. A file without the marker must lose those tags, not render them from whatever occupies the offsets. Detection is a `PCD_IPI` signature at 2048, which is where PhotoCD.pm:454 seeks. ExifTool publishes no `%magicNumber` for PCD, but this is not an over-claim: the pinned 13.59 reports `FileType: PCD` for a copy renamed `.dat` and for one with no extension, because ProcessPCD is what confirms the type. The offset is past the 1 KiB probe, so it resolves through the same short-read fallback ISO 9660's `CD001` at 32769 already needed. The parser also contributes a bare `FileType`, which `normalize_identity_tags` uses only to fill an `Unknown` -- the one identity answer a parser is allowed to supply. Verified group-qualified and per file against the pinned 13.59 on `combined-samples/PhotoCD.pcd`: 29 keys each side, 0 missing, 0 extra, 0 value differences, including the `Composite:ImageSize`/`Megapixels` that fall out of the two masked dimensions. The unit tests build the same record synthetically rather than reading the sample corpus, which is a developer cache CI does not have. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
#640 made 993 masked fractional entries decodable and measured the corpus delta at exactly zero — because ~980 of them sit in tables no caller looks up. Only about twenty of the 612 generated tables are reachable through
find_tableat all. That PR said so rather than claiming a win. This one spends some of it.PhotoCD::Mainis the highest-value unreached table in the corpus: it declares 24 fields, ExifTool reports all 24 oncombined-samples/PhotoCD.pcd, and OxiDex reported none. PCD was detected but unparsed — a correctFile:FileType: PCDbeside no PhotoCD group at all, the failure mode that looks healthy inoxidex -j.Three of those tags are why this format waited for #640:
ImageWidth,ImageHeightandCompressionClassare1538.1/.2/.3, fractional entries sharing byte 1538 withOrientation, each masking a different slice.decode_binary_tablerefused every fractional entry before #640, so the table could not have been read from.Measured coverage: +24 tags
Full 4246-file corpus against the pinned 13.59, before and after on this exact base:
Every per-format metric outside PCD is byte-identical. PCD moves
matched_tags 0 → 24,missing_in_oxidex 24 → 0.The harness also reports
extra_in_oxidex 0 → 2for PCD. Both areComposite:ImageSize/Megapixels, which ExifTool does emit with identical values — a directexiftool -G -j -acomparison is 29 keys each side, 0 missing, 0 extra, 0 value differences. 273 composites are already classified that way corpus-wide; it is how the harness invokes ExifTool, not an over-claim.The table alone was not enough
Nine fields carry a
ValueConv, three aRawConv, five aCondition— none of which the transcription runs. Each is implemented against the Perl it reproduces and cited by line, rather than skipped:int8u[2]version pairs aretr/ /./, a join:7 61is release "7.61". Read as a number it prints 7.61 by luck and turns "1.0" into "1". A pair of 255s isRawConv'd to "n/a".ScannerPixelSizeisjoin(".",unpack("H2H2",$val))— hex digits, so0x11 0x48is 11.48 micrometers, not 17.72.CreateDate/ModifyDateareConvertUnixTime($val,1): local civil time with the offset in force at that instant, reusing the same helper the File group's timestamps use.0xffffffffmeans absent, not 2106.ImageWidth/ImageHeightare($$self{Orient} & 0x01 ? 512 : 768) * ($val * 2 || 1). The DataMember comes fromOrientation's RawConv, and Perl's||makes resolution code 0 a multiplier of one rather than collapsing the image to nothing.Condition => '$$self{HasSBA}', gated on the literal bytesSBAat 225;CopyrightFileNameadditionally needsCopyrightStatus == 1. A file without the marker must lose those tags, not render them from whatever occupies the offsets — there is a test for exactly that.Detection
A
PCD_IPIsignature at offset 2048, wherePhotoCD.pm:454seeks. ExifTool publishes no%magicNumberfor PCD, but this is not an over-claim: the pinned 13.59 reportsFileType: PCDfor a copy renamed.datand for one with no extension, becauseProcessPCDis what confirms the type. The offset is past the 1 KiB probe, so it resolves through the same short-read fallback ISO 9660'sCD001at 32769 already needed.The parser contributes a bare
FileType, whichnormalize_identity_tagsuses only to fill anUnknown— the one identity answer a parser is allowed to supply.Verification
cargo test --workspace— 47 suites, 4876 passed, 0 failed (11 new PhotoCD tests, all synthetic — they build the record in-memory rather than reading the sample corpus, which CI does not have)just verify-tables— PASS vs 13.59, 0 mismatchescargo clippy --all-features -- -D warnings,cargo fmt --all --check,tools/ci/check-corpus-guards.py,scripts/sync_tag_stats.py --check— all cleanRebased onto
303b33ad(#657, which also touched detection); the one conflict was two adjacent imports, and both formats' wiring verified present afterward.🤖 Generated with Claude Code