Track: Track2; Team name: topolab; Model: HMC - #416
Open
uiharu-kazari wants to merge 2 commits into
Open
Conversation
HMC (Hajij et al., arXiv:2206.00606) is a higher-order attention network on combinatorial complexes, composed of same-cell (HBS) and non-same-cell (HBNS) attention blocks over the zeroth, first and second skeletons. Attending jointly over cells of different ranks lets the model read triangle structure directly rather than inferring it from node neighbourhoods. The tests assert numerical equivalence with the TopoModelX reference implementation within a 1e-5 tolerance. Two configs cover the two attention normalizations the blocks support (softmax and plain row normalization). Also includes the TopoBench wrapper, the 72-run challenge evaluation grid, and a supplementary notebook analysing the out-of-distribution measurements that the grid produces. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
test_m_hop_matches_manual_two_hop compares the HBS block (computed via torch.sparse CSR kernels) against a dense manual reference. Their float32 rounding differs by a few ulps, and the size of the gap depends on the CPU microarchitecture of the CI runner: the test passed at atol=1e-6 on macOS arm64 and on one ubuntu runner, then failed on the upstream PR runner with identical code, seed (torch.manual_seed(0)) and torch==2.3.0. atol=1e-5 is architecture-safe while remaining orders of magnitude below any real defect: mutations of the hop matrix, weights or normalization move the output by 0.07 to 227 in the same comparison. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HMC — Combinatorial Complex Attention Neural Network (Track 2)
Checklist
Description
Paper: Hajij et al., Topological Deep Learning: Going Beyond Graph Data (arXiv:2206.00606) — Definitions 31–33 and the HMC architecture of Figure 35(b). Reference implementation: TopoModelX (
HBS/HBNS/HMC).What this PR adds
topobench/nn/backbones/combinatorial/hmc.py— higher-order attention message passing on combinatorial complexes:HBS(intra-rank attention, Definition 32, with multi-hopm_hopsupport) andHBNS(inter-rank attention with forward/reverse directions, Definition 33), composed intoHMCLayer/HMCfollowing Figure 35(b) (9 attention blocks over two message-passing levels). Attention is computed only on the nonzero entries of coalesced sparse tensors — no dense N×N materialization. Docstrings cite the paper's definitions and equations throughout.atol=1e-5. Two deliberate, documented deviations fix latent defects in the reference: (1) the HBNS reverse-attention weight split is taken at thetarget_out_channelsboundary (the reference splits atsource_out_channels, which is incorrect for unequal channels — covered by a regression test); (2)sparse_row_normguards zero rows instead of producing NaNs.topobench/nn/wrappers/combinatorial/hmc_wrapper.pyroutes the pipeline's incidence/adjacency/coadjacency matrices into the backbone.configs/model/combinatorial/hmc.yaml(softmax attention normalization, the TopoModelX network default) andhmc_row_norm_attention.yaml(plain row normalization, the TopoModelX block default).out_channels: 64states explicitly the width the challenge harness enforces.m_hop, manual unequal-channel HBNS computation pinning the reverse-split boundary, exact-value zero-row-guard tests, the TopoModelX equivalence test, gradient-flow, empty-rank-2 edge case, and invalid-argument handling.test/pipeline/test_pipeline.pyregisters both configs (end-to-end training on MUTAG).2026_tdl_challenge/run_evaluation.ipynb(only theMODEL_CONFIGline changed, protected-cell hash intact) and the auto-generatedoutputs/2026-07-25_16-09-07/results.json: the full 72-run grid (12 regimes × 2 tasks × 3 seeds), no non-finite values. Headline in-distribution means: community-detection accuracy 0.4596, triangle-counting MSE/triangles 0.4843.2026_tdl_challenge/analysis_hmc.ipynb— decomposes the full 12×12 transfer matrices: community detection is governed by the evaluation regime (difficulty, not failed transfer), triangle counting by the training regime (scale miscalibration under distribution shift, readable only on a log scale).Known upstream limitation (disclosure)
Under TopoBench's combinatorial lifting,
batch.coadjacency_2is structurally all-zero:get_combinatorial_complex_connectivityrequestscoadjacency_matrix(1, 2), which TopoNetX rejects (rank must be greater than via_rank), and the fallback substitutes a zero matrix. Consequently the 2-cell intra-rank attention block of Figure 35(b) receives no messages in any benchmark run — for this and, we believe, every combinatorial-domain submission. The root cause is in upstreamtopobench/data/utils/utils.py(outside the allowed submission file set, so this PR does not modify it); computingcoadjacency_matrix(2, 1)there would activate the block. We flag it for the organizers' attention.Measured impact (Aug 2): we implemented the reconstruction in our own wrapper on a side branch and re-ran the full official 72-run grid with the block active. Performance is statistically unchanged (community detection 0.4596 → 0.4591; triangle counting MSE/triangles 0.4843 → 0.4918), so this PR keeps the baseline wrapper and results — the inactive block is an architectural-faithfulness issue, not a performance confound, and the committed results remain exactly reproducible from the submitted code.
Issue
No linked issue — this is a TDL Challenge 2026 submission (Track 2).
Additional context
track-2-tnn? (No triage rights.)