Skip to content

fix(docling)!: store binary_hash as a string to avoid 64-bit overflow - #3699

Merged
anakin87 merged 3 commits into
deepset-ai:mainfrom
SyedShahmeerAli12:fix/docling-binary-hash-stringify-clean
Jul 31, 2026
Merged

fix(docling)!: store binary_hash as a string to avoid 64-bit overflow#3699
anakin87 merged 3 commits into
deepset-ai:mainfrom
SyedShahmeerAli12:fix/docling-binary-hash-stringify-clean

Conversation

@SyedShahmeerAli12

@SyedShahmeerAli12 SyedShahmeerAli12 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes

MetaExtractor previously stored Docling's origin.binary_hash as an integer in document metadata. Since binary_hash is an unsigned 64-bit integer, values greater than 2^63 - 1 break document stores that map numeric metadata to signed 64-bit types (e.g. OpenSearch long), causing:

failed to parse field [dl_meta.origin.binary_hash] of type [long]
Numeric value (9768961288489567249) out of range of long

Following @julian-risch's guidance on #3604, binary_hash is now always stored as a string via a new _stringify_binary_hash() recursive helper applied in both extract_dl_doc_meta and extract_chunk_meta.

Reproduction

Before fix — binary_hash returned as int, overflows OpenSearch long:

# model_dump() returns: {binary_hash: 9768961288489567249}  � int, rejected by OpenSearch

After fix — stored as str, no overflow:

# _stringify_binary_hash() returns: {binary_hash: 9768961288489567249}  � safe keyword

Testing

  • Added 3 new unit tests: oversized hash in doc meta, chunk meta, and nested structures
  • All 8 TestMetaExtractor tests pass
  • ruff check clean

Notes

  • Breaking metadata-type change for callers that assumed binary_hash was int — aligns with maintainer suggestion for a major release
  • Both small and large hashes are stringified for consistency

@SyedShahmeerAli12
SyedShahmeerAli12 requested a review from a team as a code owner July 30, 2026 11:43
@SyedShahmeerAli12
SyedShahmeerAli12 requested review from anakin87 and removed request for a team July 30, 2026 11:43
@github-actions

Copy link
Copy Markdown
Contributor

Hi @SyedShahmeerAli12, thanks for your interest in contributing to Haystack! 🙏

⚠️ Issue #3604 is already being addressed by open pull request(s) #3607. Before opening a PR for an issue, please check whether a PR is already linked to it, and consider contributing to the existing PR instead. We may close duplicate PRs to keep the review queue manageable.

⚠️ You currently have 4 open pull requests in this repository (#3565, #3483, #3254 and this one). Our review capacity is limited, so please hold off opening more PRs until we've had a chance to review your first 2 open PRs. This helps us give each contribution the attention it deserves. Thank you!

This is an automated message to help us keep the review queue healthy.

@github-actions github-actions Bot added the type:documentation Improvements or additions to documentation label Jul 30, 2026
Comment thread integrations/docling/tests/test_converter.py Outdated
Comment thread integrations/docling/tests/test_converter.py Outdated
@anakin87 anakin87 changed the title fix(docling): stringify binary_hash to prevent OpenSearch long overflow fix!(docling): stringify binary_hash to prevent OpenSearch long overflow Jul 31, 2026
@anakin87 anakin87 changed the title fix!(docling): stringify binary_hash to prevent OpenSearch long overflow fix(docling)!: store binary_hash as a string to avoid 64-bit overflow Jul 31, 2026

@anakin87 anakin87 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just simplified it a bit.

Ready to go now. Thanks!

@anakin87
anakin87 merged commit 001a099 into deepset-ai:main Jul 31, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration:docling type:documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DoclingConverter: binary_hash metadata can exceed signed 64-bit integer range

2 participants