Track: Track2; Team name: howyadoin; Model: CTNN

[dario-loi]

Track

Track 2 — Topological Neural Networks (TNNs)

Team Name

howyadoin

• Dario Loi — Northeastern University
• Gabriele Onorato — Northeastern University

Model

Copresheaf Topological Neural Network (CTNN)

Status

Ready for review

Summary

This PR contributes a TopoBench-native implementation of Copresheaf
Topological Neural Networks (CTNN) by Hajij et al. (NeurIPS 2025). A CTNN
gives every cell of a combinatorial complex its own feature space (a stalk)
and every directed neighbour relation y -> x a learnable transport map
rho_{y->x}: F(y) -> F(x). Messages are carried into the receiver's frame
before aggregation, which makes propagation direction-aware; a cellular
sheaf glues data through a shared edge stalk instead and is therefore
symmetric.

Implementation

• Backbone with the transport-map catalogue of Table 18, copresheaf
  attention over one neighborhood, the Definition 10 layer, and the
  stacked model (topobench/nn/backbones/simplicial/ctnn.py).
• Per-head linear map (HeadwiseLinear) so transport is evaluated
  independently per attention head, as Table 18 specifies, rather than
  sharing weights the way torch.nn.Linear would.
• Hydra config configs/model/simplicial/ctnn.yaml (SheafFC, the map of
  Appendix H.5) plus ctnn_spd.yaml and ctnn_diag.yaml for the two
  other entries of the catalogue the paper applies to topological
  domains.
• Unit tests asserting the paper's equations rather than tensor shapes:
  attention is row-stochastic per receiver and head, the aggregated
  message matches a dense per-pair reference for Eq. (7) and (12),
  otimes is summation, SheafFC starts at the identity, SheafSPD has
  eigenvalues at least one, the diagonal map gates into (0, 1),
  transport is directional (rho_{y->x} != rho_{x->y}^T), and every
  registered parameter receives gradient. 44 tests in
  test/nn/backbones/simplicial/test_ctnn.py, including an overfit check
  and degenerate complexes with no 2-cells and no 1-cells.
• All three configs added to test/pipeline/test_pipeline.py so CI
  exercises each transport map end to end.
• Sphinx page for the new module (docs/api/).
• Challenge notebook run end to end over the full GraphUniverse grid
  (12 settings × 3 seeds × 2 experiments = 72 runs); the auto-generated
  results.json is included. The grid was run with simplicial/ctnn_diag,
  with diagonal transport maps. More expressive variants are available, but led to
  severe overfitting on the small GraphUniverse datasets.

Notes on faithfulness

Three conventions the paper leaves open, we collect our assumptions here and
reiterate them in the docstrings:

• The query/key width p equals the stalk width, the setting Table 18 assumes
  when it sizes transport as W in R^{2d x d^2}.
• beta runs once per layer over the combined message, following
  Definition 10 and the layer displayed in Appendix H.5. Algorithm 1 instead
  writes two sequential updates, one after self-attention and one after
  cross-attention, which makes the result depend on the order the
  neighborhoods happen to be listed in.
• Per-head messages are concatenated with no output projection. Algorithm 1
  writes none; the cost accounting under Table 19 charges an O(n d^2) step
  to "combine head outputs", and the full-width beta that immediately
  follows mixes heads instead.

Two deliberate deviations from Appendix H.5, both flagged inline in the
configs:

• Appendix H.5 defines its 0 <-> 0 adjacency through 2-cells
  (2-up_adjacency-0 here). GraphUniverse graphs have average degree 1-5, so
  triangle-adjacency is nearly empty and would silence the 0 <- 0 path on
  most nodes; the configs use up_adjacency-0 instead.
• Stalk width is d = 4 (channels=64, heads=16), which is what the paper
  uses for its classification tasks (Section 6.3, Appendix H.3.1, H.4). The
  d = 16 of Section 6.1 and Appendix H.5 is for physics regression, and
  because SheafFC transport carries W in R^{2d x d^2} per head it puts 59%
  of the backbone's weights in the transport maps at that width, which
  overfits GraphUniverse.

One TopoBench integration detail: Definition 7 builds the
copresheaf neighborhood matrix by replacing each nonzero of the binary
matrix with a map, so the stored scalar carries no information beyond being
nonzero. The zeros have to be filtered by value rather than read off the
sparsity pattern, because TopoBench materialises up_adjacency-r straight
from toponetx, which stores an explicit zero on the diagonal; reading
indices() alone would treat those as a self-loop y = x on every cell.
test_stored_zeros_are_not_neighbours pins this down.

We have turned dropout to 0.5 to combat overfitting,achieving results that outmatch those of the average submission to the challenge. We speculate that with further regularization, the model could achieve even better results.

Reference

Hajij, Bastian, Osentoski, Kabaria, Davenport, Dawood, Cherukuri,
Kocheemoolayil, Shahmansouri, Lew, Papamarkou, Birdal, "Copresheaf Topological
Neural Networks: A Generalized Deep Learning Framework," NeurIPS 2025.

• ArXiv: https://arxiv.org/abs/2505.21251

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[dario-loi]

@levtelyatnikov

To faithfully reproduce the paper, I'd need to set trainer.gradient_clip_val=0.8 (Appendix H.5). I suspect that it would be of great help to the results since the model seems under-regularized (just raising dropout to 0.5 gave us ~+10% accuracy).

Unfortunately I cannot tune gradient clipping just from the model's YAML, so I won't do that for fear of invalidating the submission. If you tell me that introducing an override for trainer is ok, I can re-run the experiments.