Skip to content

[Network Gap] Implement posterior log-ratio topic networks and stable consensus clustering #172

Description

@seonghobae

Buyer problem

TEPP promises topic/event relationship networks and stable clusters, but protected main does not yet compute posterior-aware associations, uncertainty, edge stability, or consensus communities from fitted topic measurements. #71 correctly rejects raw-simplex Euclidean geometry and provides pair metrics; it is not a production network estimator.

Product outcome

Implement the network_analysis vertical after the CPU topic estimator exists:

posterior topic coordinates
→ valid log-ratio / logistic-normal association estimates
→ uncertainty and resampling stability
→ edge admission and multiplicity policy
→ repeated Leiden community detection
→ co-assignment consensus clusters
→ evidence-bound network and cluster artifacts

Acceptance criteria

  • Raw topic proportions cannot enter ordinary Pearson correlation, Euclidean clustering, graphical lasso, or ESEM as if unconstrained.
  • The default network uses posterior logistic-normal or orthonormal log-ratio coordinates and propagates posterior uncertainty.
  • Each edge records effect size, interval, posterior/selection probability, bootstrap/seed stability, sample basis, threshold, correction policy, and provenance.
  • Optional conditional-dependence estimation operates only in a valid transformed space and is explicitly labeled experimental until separately validated.
  • Positive stable associations feed repeated Leiden community detection; negative associations remain tension/opposition edges rather than membership glue.
  • Consensus clustering is derived from a co-assignment matrix across posterior draws/seeds/bootstraps; unstable topics may remain unclustered.
  • Cluster labels and explanations are evidence-bound LLM hypotheses only and cite exact topic/evidence identifiers; they cannot redefine numerical membership.
  • Known-truth simulations recover edge sign, precision/recall, interval coverage, selection stability, community ARI/NMI or pair metrics, and unclustered cases under varying sparsity, overlap, prevalence, and sample size.
  • Relabeled cluster identities remain equivalent; label switching is handled without hard-coded labels.
  • Results export versioned JSON, JSON-LD, GraphML, CSV, and exact-value tables with source/model/config/cutoff hashes.
  • Large-K performance uses bounded Rust parallelism and measured memory; no dense quadratic artifact is silently materialized beyond configured limits.
  • Production statement/branch coverage and public docs remain 100%; property/fuzz tests cover singular covariance, zero/near-zero topics, nonfinite values, disconnected graphs, all-negative graphs, and unstable communities.

Product proof

Provide a realistic anonymized corpus fixture and a synthetic known-truth graph fixture. Demonstrate that the production network recovers the declared structure and that naïve raw-proportion Pearson/Euclidean alternatives create materially worse or compositionally spurious results.

Dependency and scope

This issue depends on a real fitted topic posterior, not solely #48 coordinate transforms. It does not include the final visual workspace, which is tracked separately, and it does not permit network association to be called causation.

Authority

Use PRD §§9 and 14, Architecture, ADR 0005/0012, and the APA 7 research register.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions