Skip to content

fix(detection): anchor the text-format rules, parse Radiance, read DXF's magic once - #657

Merged
swackhamer merged 2 commits into
mainfrom
claude/keen-easley-cc904b
Aug 10, 2026
Merged

fix(detection): anchor the text-format rules, parse Radiance, read DXF's magic once#657
swackhamer merged 2 commits into
mainfrom
claude/keen-easley-cc904b

Conversation

@swackhamer

@swackhamer swackhamer commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

text.contains("v ") accepted a v anywhere in the first 100 bytes of a
file. Radiance.hdr in the shared corpus opens

#?RADIANCE
oconv mat.rad sky.rad surfaces.rad

and the v that ends oconv satisfied it, so a Radiance HDR image was
dispatched to the OBJ parser, which reported HasNormals/HasTextureCoords
for an image file. Nothing surfaced it: File:FileType is decided by the magic
table and went on saying HDR, and #651 removed the ungrouped FileType: OBJ
that had been the only visible symptom.

Pulling on that turned up two more of the same shape -- a format's identity and
its parser dispatch decided by two different pieces of code, the second one
wrong.

Anchor the four text rules

ExifTool's magic patterns anchor; these did not.

  • OBJ requires ^v , ^vn or ^vt with operands, on a real line.
  • DXF requires SECTION as its own group-code record (see below).
  • GLTF requires the probe to open a JSON object and "asset" to be a real
    key. contains("{") && contains("\"asset\"") accepted any document that
    mentioned both, in either order, at any depth.
  • STL keeps its byte-0 anchor -- it never had this bug -- but now needs
    solid as a whole word (solidification matched the prefix) plus a
    corroborating facet/endsolid directive, so prose opening "solid ..." is no
    longer handed to the STL parser. That directive is looked for across the whole
    probe, since a long solid name pushes it past 100 bytes.

OBJParser::verify_signature carried a second copy of the substring rule --
which is why the parser accepted the Radiance file instead of rejecting it --
and now calls the detector's predicate.

Parse Radiance instead of detecting it

FileFormat had no HDR variant, so a file the magic table had already
identified fell through to the plain-text fallback and reported TEXT statistics
ExifTool reports for no image. src/parsers/image/radiance.rs transcribes
Radiance.pm's ProcessHDR: the KEY=value header split on its last =
as the greedy regex does, the table's eight names plus ExifTool's
AddTagToTable fallback for unlisted keys, comments, every Command line
rather than the last, and the resolution line's orientation with rows before
columns.

Read DXF's magic number once

ExifTool's pattern is ^\s*0\s+\x00?\s*SECTION\s+2\s+HEADER. filetype
compiled it for the identity tags; detect_format tested starts_with("0\n");
DXFParser::verify_signature tested 20 bytes for 0\nSECTION. Real AutoCAD
writers right-align group codes in three columns and end lines with CRLF, so
0\r\n files were called DXF by the identity tags and parsed as plain text.
Both hand-written copies now call filetype::matches_magic("DXF", ..), which is
also the only thing that keeps the two answers from drifting again.
parse_content already trimmed group codes, so nothing else had to change.

Verification

Instruments named, since the numbers are claims about them:

  • just compare-file .../Radiance.hdr (oracle: pinned ExifTool 13.59 via
    exiftool_oracle.shared()) -- MISSING 9 / WRONG 0 to MISSING 0 / WRONG 0.
    Software, View, Format, Exposure, Orientation, ImageWidth, ImageHeight, and
    the two composites that follow from the dimensions. The remaining AMBIGUOUS 1
    is Command, four values collapsing onto one bare name.
  • Corpus sweep over combined-samples via scripts/compare_file.py --
    122 WRONG before, 122 after, the same 122 lines by diff, not merely the
    same count. (122 rather than the 134 this branch started against: upstream's
    empty-UserComment fix removed 12.)
  • cargo test --workspace (full, not filtered) -- 4864 passed, 0 failed,
    47 binaries, including 25 new tests.
  • cargo clippy --workspace --all-targets -- no findings in the changed files.
  • End-to-end through the built binary: .obj, .stl, .gltf and both DXF
    spellings still reach their parsers; prose opening "solid " stops at TXT.

Known gap, not addressed here

The Radiance tags are emitted ungrouped, where ExifTool groups them under
[Radiance]. That matches what every comparable parser here already does --
DPX emits its content tags ungrouped against ExifTool's [File] -- and
compare_file.py compares by bare name, so it cannot see the difference either
way. Worth a separate pass across the parsers rather than one format's
exception.

🤖 Generated with Claude Code

swackhamer and others added 2 commits August 10, 2026 09:51
`text.contains("v ")` accepted a `v ` anywhere in the first 100 bytes, so
`Radiance.hdr` -- whose header opens `#?RADIANCE\noconv mat.rad ...` -- was
dispatched to the OBJ parser by the `v ` that ends `oconv `. It reported
HasNormals/HasTextureCoords for an image file, silently: `File:FileType`
comes from the magic table and went on saying HDR.

Anchor all four rules to the start of a line, the way ExifTool's magic
patterns do:

* OBJ requires `^v `, `^vn ` or `^vt ` with operands, not a loose substring.
* DXF requires SECTION as its own group-code record, not a keyword anywhere
  in the header. The `2`/`HEADER` records ExifTool's magic also wants are
  deliberately not required -- that would reject ENTITIES-first files this
  has always accepted.
* GLTF requires the probe to open a JSON object and `"asset"` to be a real
  key; `contains("{") && contains("\"asset\"")` matched any document that
  merely mentioned both.
* STL keeps its byte-0 anchor -- it never had this bug -- but now needs
  `solid` as a whole word (`solidification` matched the prefix) plus a
  corroborating facet/endsolid directive, so prose opening "solid ..." is
  no longer handed to the STL parser. That directive is looked for across
  the whole probe, since a long solid name pushes it past 100 bytes.

OBJParser::verify_signature carried a second copy of the same substring
rule, which is why the parser accepted the Radiance file rather than
rejecting it; it now calls the detector's predicate.

Verified: `just compare-file .../Radiance.hdr` (pinned ExifTool 13.59) is
unchanged at MISSING 9 / WRONG 0 -- the OBJ tags were EXTRA and never
scored -- while `oxidex -j` no longer emits them. Corpus sweep over
combined-samples via scripts/compare_file.py holds at 134 WRONG, the same
134 lines. `cargo test --workspace`: 4821 passed, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two defects the OBJ misroute uncovered, both of the same shape: a format's
identity and its parser dispatch were decided by two different pieces of
code, and the second one was wrong.

Radiance HDR was detected and not parsed. `FileFormat` had no HDR variant,
so a file the magic table had already identified -- `File:FileType: HDR`,
`MIMEType: image/vnd.radiance` -- fell through `detect_format` to the
plain-text fallback and reported TEXT statistics ExifTool reports for no
image. Add the variant, detect the `#?RADIANCE`/`#?RGBE` signature ahead of
the text rules, and transcribe `Radiance.pm`'s ProcessHDR: the `KEY=value`
header split on its last `=` as the greedy regex does, the table's eight
names plus ExifTool's AddTagToTable fallback for unlisted keys, comments,
every Command line rather than the last, and the resolution line's
orientation with rows before columns.

DXF read its magic number from three places. ExifTool's pattern is
`^\s*0\s+\x00?\s*SECTION\s+2\s+HEADER`; `filetype` compiled it for the
identity tags, while `detect_format` tested `starts_with("0\n")` and
DXFParser::verify_signature tested 20 bytes for `0\nSECTION`. Real AutoCAD
writers right-align group codes in three columns and end lines with CRLF,
so `  0\r\n` files were called DXF by the identity tags and parsed as plain
text. Both hand-written copies now call `filetype::matches_magic("DXF", ..)`,
which is also the only way to keep the two answers from drifting again;
`parse_content` already trimmed group codes, so nothing else had to change.

Verified: `just compare-file .../Radiance.hdr` (pinned ExifTool 13.59) goes
from MISSING 9 / WRONG 0 to MISSING 0 / WRONG 0 -- Software, View, Format,
Exposure, Orientation, ImageWidth, ImageHeight and the two composites that
follow from the dimensions. Corpus sweep over combined-samples via
scripts/compare_file.py holds at 122 WRONG, the same 122 lines (122, not
the 134 measured before this branch rebased -- upstream's empty-UserComment
fix removed 12). `cargo test --workspace`: 4864 passed, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@swackhamer
swackhamer force-pushed the claude/keen-easley-cc904b branch from 1e86409 to 67c65cd Compare August 10, 2026 14:58
@swackhamer
swackhamer merged commit 303b33a into main Aug 10, 2026
10 checks passed
@swackhamer
swackhamer deleted the claude/keen-easley-cc904b branch August 10, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant