Skip to content

refactor(parsers): stop computing the identity tags two normalizers discard - #651

Merged
swackhamer merged 1 commit into
mainfrom
claude/dazzling-dijkstra-438a4a
Aug 10, 2026
Merged

refactor(parsers): stop computing the identity tags two normalizers discard#651
swackhamer merged 1 commit into
mainfrom
claude/dazzling-dijkstra-438a4a

Conversation

@swackhamer

@swackhamer swackhamer commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Scope changed while this was open. It opened as a fix for oxidex -j printing two contradictory FileType lines. #650 then fixed the identity half at runtime and #654 the FileSize half, both by discarding what the parsers wrote. The user-visible bug is gone from main; what is left is the source-level half neither touched, plus one behavioural loose end in fits.rs.

What is left

normalize_identity_tags removes an ungrouped FileType/FileTypeExtension/MIMEType; drop_redundant_file_size removes an ungrouped FileSize. Neither changed a parser. So 45 inserts across 40 parsers still compute values that cannot reach a caller, and the parsers still read as though FileSize and MIMEType were theirs to report — the trap that produced the duplicates to begin with.

This removes them, and keeps both normalizers as the backstop for anything that reintroduces one.

FileType deliberately stays. normalize_identity_tags promotes it into an absent or Unknown File:FileType, and that fallback is the only reason these are named at all — ExifTool's tables have no row for any, and it refuses to open four outright:

file File:FileType ExifTool 13.59
.sqlite SQLite Error: Unknown file type
.evtx Windows Event Log Error: Unknown file type
.reg Registry Hive Error: Unknown file type
.hdf5 HDF5 Error: Unknown file type
EXE.elf ELF ELF executable

Deleting the parser's FileType would have dropped all five to Unknown. That is what an earlier revision of this PR did, and it is why this one does not.

fits.rs — the one behavioural loose end

FITSParser wrote File:FileType, File:FileTypeExtension and File:MIMEType directly, as hardcoded literals rather than table lookups. Being already in the File: group put it out of both normalizers' reach — they only ever removed the ungrouped spellings — and merge gives format metadata precedence over the File: group. It was the last place a parser could silently outrank ExifTool's own tables had the two ever drifted.

The generated tables carry FITS in full: the SIMPLE = {20}T magic number, the fits extension row, and ("FITS", "image/fits"). Removing the literals changes nothing on FITS.fits, which is the evidence that they were redundant.

Verification

Instruments named, per AGENTS.md.

  • Full-JSON diff of both binaries over 207 combined-samples, this branch against main@9270f94a: no key lost, none gained, no value changed (excluding File:FileAccessDate, which moves because reading updates atime). The cleanup is provably invisible, fits.rs included.
  • scripts/compare_file.py over the same corpus: WRONG 134 → 134, MISSING 1726 → 1726, graded output byte-identical.
  • cargo test --workspace: 4840 passed, 0 failed. tests/integration.rs 574 → 575 — the new guard. No test removed.
  • clippy and fmt clean; no new warning in any changed file.

Guard

parsers_do_not_answer_identity_tags_the_file_group_owns runs eight formats through the real read_metadata path and asserts a parser answers none of the three, while File:FileType still reads the expected value and File:FileSize keeps ExifTool's formatting rather than the raw byte count the parsers used to publish beside it.

Per-parser tests that asserted the removed inserts now assert their absence, in the style tests/forensic/lnk_tests.rs already established.

Not in scope

  • The 42 parsers' FileType inserts, for the reason above. Moving those six names into crate::filetype would let the fallback go, but that is a coverage change, not a cleanup.
  • Radiance.hdr reaching the OBJ parser at all: detection/text.rs matches "v " anywhere in the first 100 bytes. Harmless now that the parser's FileType is outranked by the tables, but the routing is still wrong.

🤖 Generated with Claude Code

…iscard

#650 and #654 left one answer per identity fact in the output, both by
discarding what the parsers wrote: `normalize_identity_tags` removes an
ungrouped `FileType`/`FileTypeExtension`/`MIMEType`, and
`drop_redundant_file_size` removes an ungrouped `FileSize`. Neither touched
the source. 45 inserts across 40 parsers therefore still computed values that
could not reach a caller, and the parsers still read as though `FileSize` and
`MIMEType` were theirs to report -- the trap that produced the duplicates in
the first place.

Removes them. `FileType` deliberately stays: `normalize_identity_tags`
promotes it into an absent or `Unknown` `File:FileType`, and that fallback is
the only reason SQLite, EVTX, Registry Hive, HDF5, X.509 and ELF are named at
all -- ExifTool's tables have no row for any of them, and it refuses to open
four outright. Both normalizers stay too, as the backstop for anything that
reintroduces one.

fits.rs is the one behavioural loose end. It wrote `File:FileType`,
`File:FileTypeExtension` and `File:MIMEType` directly -- hardcoded literals,
not table lookups -- which put it out of both normalizers' reach, since they
only ever removed the ungrouped spellings. `merge` gives format metadata
precedence over the `File:` group, so it was the last place a parser could
still outrank ExifTool's own tables, silently, had the two ever drifted. The
generated tables carry FITS in full (the `SIMPLE  = {20}T` magic number, the
`fits` extension row, `("FITS", "image/fits")`) and answer exactly as those
literals did.

Verified (instruments named):
- Full-JSON diff of both binaries over 207 combined-samples, this branch
  against main @9270f94a: no key lost, none gained, no value changed. The
  cleanup is provably invisible, fits.rs included -- which is the evidence
  that removing its literals cost nothing.
- `scripts/compare_file.py` over the same corpus: WRONG 134 -> 134,
  MISSING 1726 -> 1726, graded output byte-identical.
- `cargo test --workspace`: 4840 passed, 0 failed. tests/integration.rs
  574 -> 575, the new guard below; no test removed.
- clippy and fmt clean; no new warning in any changed file.

Guarded by `parsers_do_not_answer_identity_tags_the_file_group_owns`, which
runs eight formats through the real `read_metadata` path and asserts a parser
answers none of the three, while `File:FileType` still reads the expected
value and `File:FileSize` keeps ExifTool's formatting rather than the raw byte
count the parsers used to publish beside it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@swackhamer
swackhamer force-pushed the claude/dazzling-dijkstra-438a4a branch from dcf7286 to 532cb3e Compare August 10, 2026 05:03
@swackhamer swackhamer changed the title fix(parsers): report identity tags once, in the File: group refactor(parsers): stop computing the identity tags two normalizers discard Aug 10, 2026
@swackhamer
swackhamer merged commit 38b2008 into main Aug 10, 2026
10 checks passed
@swackhamer
swackhamer deleted the claude/dazzling-dijkstra-438a4a branch August 10, 2026 05:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant