Skip to content

⚡️ Improve DD table hashing and capacity growth - #2561

Open
burgholzer wants to merge 1 commit into
mainfrom
codex/dd-table-adaptation
Open

burgholzer wants to merge 1 commit into
mainfrom
codex/dd-table-adaptation

Conversation

@burgholzer

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

Description

Aligned node addresses can leave binary compute-table buckets unused, while
fixed unique-table directories waste memory on sparse levels and build long
chains on crowded levels. Mix pointer hashes with the existing murmur64
function and add bounded, per-level unique-table growth without moving nodes.

Growth remains opt-in through DDPackageConfig::utMaxNumBucket (zero by
default). Initial capacities and other compute-table defaults are unchanged.
Capacities must be powers of two; a nonzero ceiling must cover both initial
unique-table sizes. Grown directories survive collection and reset. Bucket
views can be invalidated by insertion, but canonical nodes and owned roots
remain stable. No new dependencies or migration are required.

For example, this profile starts small and grows populated levels:

dd::DDPackageConfig config;
config.utVecNumBucket = 1024;
config.utMatNumBucket = 1024;
config.utMaxNumBucket = 1048576;
dd::Package package(4096, config);

For a 4,096-qubit zero state, this reduces unique bucket storage from 2,048 to
64 MiB. Larger multiplication caches remain an explicit choice through the
existing ctMatVecMultNumBucket field.

Exploration against main at dc78630f9d6f56a4cd1456ffa11502e12c07ec39
(unchanged DD sources at this PR's base) used Clang 23 release builds on a DGX
Spark. Three runs per setting measured direct all-pairs RZZ twisting on 64
qubits at a median 1.896 s upstream versus 0.426 s with this profile and a
262,144-entry matrix-vector cache. Compiler overlap, mixed results on other
workloads, and one dense candidate timeout limit timing conclusions; this PR
does not change the default capacity profile.

Validation: all 194 native DD tests pass, covering rehashing, canonical identity,
owned roots, collection/reset, constructor levels, invalid capacities, and fixed
hashing before level allocation. Repository lint and C++ lint pass. Existing
upstream compiler/header diagnostics are unchanged. The complexity review
removed duplicate constructor setup by reusing UniqueTable::resize.

Codex assisted with implementation, review, validation, and this description.

Checklist

  • The pull request only contains commits that are focused and relevant to this change.
  • I have added appropriate tests that cover the new/changed functionality.
  • I have updated the documentation to reflect these changes.
  • The changes follow the project's style guidelines and introduce no new warnings.
  • The changes are fully tested and pass the CI checks.
  • I have reviewed my own code changes.

If PR contains AI-assisted content:

  • Any agent that created, edited, or submitted GitHub content was explicitly authorized for that scope, as required by our AI Usage Guidelines.
  • Every agent-authored or agent-edited public text body begins with the visible disclosure 🤖 *AI text below* 🤖 (titles are exempt).
  • I have disclosed AI assistance in the PR description.
  • I confirm that I have personally reviewed and understood all AI-generated content, and accept full responsibility for it.

Mix pointer operands before masking binary compute-table hashes so
aligned node addresses do not waste cache buckets. Allow unique-table
levels to grow independently without moving canonical nodes.

Keep growth opt-in through a per-level bucket ceiling and retain grown
capacities across reset. Reuse level initialization when constructing a
unique table, and cover rehashing, roots, collection, and fixed hashing.

Assisted-by: GPT-6 via Codex
@burgholzer burgholzer added enhancement Improvement of existing feature DD Anything related to the DD package c++ Anything related to C++ code labels Sep 14, 2026
@burgholzer burgholzer self-assigned this Sep 14, 2026
@codecov

codecov Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Anything related to C++ code DD Anything related to the DD package enhancement Improvement of existing feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant