feat: add timestamp and duration support in leaf data hashing (ITL-438)#8
Merged
eywalker merged 5 commits intoJun 29, 2026
Conversation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ec (ITL-438) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add is_timestamp and is_duration checks to _element_size_for_type (8 bytes each) - Add test_timestamp_types_in_schema and test_duration_types_in_schema - Add TestTimestampDurationHashing golden parity tests (15 cases) matching Rust starfix byte-for-byte Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds support for hashing Arrow timestamp and duration leaf arrays in the Python ArrowDigester, aligning behavior and golden outputs with the Rust starfix implementation (ITL-438).
Changes:
- Treat
timestampanddurationleaf buffers as fixed-width 8-byte (int64) elements in_element_size_for_type. - Add schema-serialization assertions ensuring timestamp unit/tz and duration unit are captured in serialized schema JSON.
- Add golden parity tests (with Rust-derived expected hashes) covering all timestamp units (tz-aware UTC and tz-naive) and all duration units, plus “units differ” / “tz differs” invariants.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
src/starfix/arrow_digester.py |
Adds fixed-width sizing for timestamp/duration leaf hashing (8 bytes). |
tests/test_arrow_digester.py |
Adds schema serialization tests for timestamp (unit+tz) and duration (unit). |
tests/test_golden_parity.py |
Adds Rust-aligned golden hash parity tests for timestamp/duration arrays. |
docs/metamorphic/specs/2026-06-28-timestamp-duration-hashing-design.md |
Documents design rationale and success criteria for the hashing change. |
docs/metamorphic/plans/2026-06-28-timestamp-duration-hashing.md |
Adds an implementation plan capturing steps and test strategy for the change. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
eywalker
approved these changes
Jun 29, 2026
eywalker
deleted the
eywalker/itl-438-add-support-for-timestamp-and-duration-types-in-leaf-data
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
is_timestampandis_durationchecks to_element_size_for_typeinarrow_digester.py(return 8 — int64 physical storage)test_timestamp_types_in_schemaandtest_duration_types_in_schematotest_arrow_digester.pyTestTimestampDurationHashingclass intest_golden_parity.pywith 15 tests, golden hash values taken from the Ruststarfiximplementation (byte-for-byte identical)Dependency
Golden hash values were generated from
nauticalab/starfixafter implementing the Rust fix. Python and Rust hashes are byte-for-byte identical.Test plan
pytest tests/passes (193 tests)Closes ITL-438
🤖 Generated with Claude Code