fix: store all DB hash columns as large_binary (ITL-539)

[kurodo3]

Summary

• data_function.py: variation hash fields (function_signature_hash, function_content_hash) now stored as bytes via to_prefixed_digest(), schema changed from str → bytes so they emit large_binary Arrow columns.
• result_cache.py: removed _hash_val_to_binary() tolerance shim and the _HASH_VAR_COLS conversion loop (dead code now that the source produces bytes directly). Fixed lookup() type annotation from dict[str, str] to dict[str, bytes].
• operator_node.py: NODE_CONTENT_HASH_COL written as large_binary using to_prefixed_digest(); _filter_by_content_hash() updated to compare against binary digest.
• side_effects.py: record_id_hash stored as large_binary; _write_invocation_row parameter renamed from record_id_hash_str: str to record_id_hash_bytes: bytes.
• DESIGN_ISSUES.md: documents the OperatorJobNode migration gap (no v0→v1 migration utility exists for NODE_CONTENT_HASH_COL).

Breaking change: Existing pipeline DBs written by OperatorJobNode with the old large_string schema will fail on next write due to Arrow schema mismatch. Manual table recreation is required (see DESIGN_ISSUES.md ON1). Accepted for pre-v0.1.0.

Test plan

• `uv run pytest tests/test_core/data_function/ -v` — variation hash schema tests pass
• `uv run pytest tests/test_core/test_result_cache.py -v` — binary column type tests pass, shim removal is regression-free
• `uv run pytest tests/test_core/operators/ tests/test_core/nodes/ -v` — operator node hash column tests pass
• `uv run pytest tests/test_core/side_effect_pod/ tests/test_core/side_effect_function/ -v` — side effect invocation log tests pass
• `uv run pytest tests/ -x -q` — 4602 tests pass

Fixes ITL-539

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 83.33333% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/orcapod/core/nodes/operator_node.py	75.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[Copilot]

Pull request overview

This PR completes the ITL-539 cleanup to ensure all pipeline/result DB hash-related columns are stored as Arrow large_binary by switching remaining producers to emit prefixed digests as bytes, removing now-dead string→bytes tolerance code, adding regression tests, and documenting the remaining migration gap for OperatorJobNode.

Changes:

• Emit variation/content hashes as bytes (to_prefixed_digest()) so Arrow tables store them as large_binary (not large_string).
• Remove the ResultCache variation-hash conversion shim and add tests asserting binary column types + round-tripping via ContentHash.from_prefixed_digest.
• Update OperatorJobNode and side-effect invocation logging to store/compare hash columns as large_binary, and document the lack of an operator-node migration utility.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/orcapod/core/data_function.py	Switch variation hash fields + schema to bytes so they become large_binary in Arrow.
src/orcapod/core/result_cache.py	Remove v0 tolerance shim; adjust lookup API typing (hash constraints now naturally bytes at the source).
src/orcapod/core/nodes/operator_node.py	Store NODE_CONTENT_HASH_COL as large_binary and compare using binary digests.
src/orcapod/side_effects.py	Store record_id_hash as large_binary; update _write_invocation_row signature accordingly.
tests/test_core/data_function/test_data_function.py	Update tests for bytes-based variation hash schema + round-trip decoding.
tests/test_core/test_result_cache.py	Add assertions that variation hash columns are large_binary in stored records.
tests/test_core/operators/test_operator_node.py	Add coverage for binary NODE_CONTENT_HASH_COL storage and filtering.
tests/test_core/side_effect_pod/test_side_effect_pod.py	Add coverage that invocation log stores record_id_hash as large_binary and preserves bytes.
DESIGN_ISSUES.md	Document the OperatorJobNode migration gap for NODE_CONTENT_HASH_COL.
superpowers/specs/2026-07-21-hash-columns-large-binary-design.md	Add design/spec artifact describing remaining gaps + intended fixes/tests for ITL-539.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kurodo3]

Fixed. Changed to dict[str, Any] (and the local constraints dict inside the method body). The annotation was overly restrictive — callers legitimately pass string values for non-hash columns like function_name. Also added Any to the from typing import line. Matches the underlying ArrowDatabaseProtocol.get_records_with_column_value(column_values: Mapping[str, Any]) signature.

[kurodo3]

Fixed. Added a type check for the column before calling pc.equal. If NODE_CONTENT_HASH_COL exists but is large_string (old schema), the method now raises ValueError with a message explaining the cause and pointing to DESIGN_ISSUES.md ON1 for the remediation steps (drop and recreate the affected tables), rather than letting PyArrow throw a cryptic type error.

[kurodo3]

Review round 1 — addressed in 8c69184

Two fixes in a single commit:

result_cache.py — additional_constraints type widened to Any
Changed dict[str, bytes] → dict[str, Any] for both the lookup() parameter and the local constraints dict. The previous annotation was too restrictive — callers can and do pass string values for non-hash columns (e.g. function_name). Matches the underlying ArrowDatabaseProtocol.get_records_with_column_value(column_values: Mapping[str, Any]) signature. Added Any to the from typing import line.

operator_node.py — explicit type-mismatch guard in _filter_by_content_hash()
Added a col_type check after the existing missing-column guard. If the column is present but is large_string (old DB schema), the method now raises a clear ValueError pointing to DESIGN_ISSUES.md ON1 and explaining that the affected pipeline DB tables must be dropped and recreated — rather than letting PyArrow raise a cryptic type error.