diff --git a/CHANGELOG.md b/CHANGELOG.md
index 691b1c7ef..f61eef918 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,18 @@ Full release notes with details on each version: [GitHub Releases](https://githu
## Unreleased
+- Feat: Sigma.js + graphology fallback for large `graph.html` views, plus batched/reconciled community labeling at scale. vis-network's live client-side forceAtlas2 physics simulation is genuinely slow/laggy past a few hundred nodes regardless of hardware — verified in production on a 1083-community monorepo graph (apa, 2026-07-03). Step 6 of the skill now generates a separate WebGL Sigma.js render (`graph_sigma.html`) with the layout precomputed offline in Python (`nx.forceatlas2_layout`, no client-side physics) once the view exceeds ~300 nodes; the full recipe lives in the new `references/sigma-viz.md`, shipped to every split-platform skill variant. Step 5's community labeling also gains a batched-subagent flow for 100+ communities (sample each community's top-degree nodes, dispatch one `general-purpose` subagent per ~15-20-community batch in parallel) plus a required cross-batch label-reconciliation pass — a naive single-pass label run on the same graph left 338 of 1083 communities (31%) sharing a label with another before reconciliation.
+ The Sigma view also fixes a node-sizing bug (`autoRescale` must be disabled alongside `itemSizesReference: "positions"`, per sigma's own docs — otherwise sizes stay anchored to the pre-rescale coordinate frame and don't track zoom) and recalibrates node size against the layout's own median nearest-neighbor distance instead of a fixed constant, so communities stay legible instead of overlapping into a blob as node count grows. It now also surfaces each community's dominant content kind as an icon (code/document/paper/image/rationale/concept, via `@sigma/node-image`'s `NodePictogramProgram`) and dominant top-level module as color, plus a left-side panel to filter by entity kind and by relation type (grouped into calls/structure/imports/references/documentation-and-concepts/groups, covering both code and non-code relations) and a clickable module legend to isolate one part of the codebase at a time.
+ Two more critical rendering bugs were caught and fixed only by actually opening a generated file in a real browser (Chrome via Playwright), not by reading the code: (1) disabling `autoRescale` also disables sigma's automatic "fit the camera to the graph" on load, so without an explicit `fitViewportToNodes` call (from `@sigma/utils`) the camera pointed at empty space and the page rendered as a solid black screen with nothing visible; (2) the `@sigma/node-image` and `@sigma/utils` ESM imports failed at module-load time because esm.sh resolved their `sigma` peer dependency to a broken literal path instead of the pinned version — that import throws before any of the script runs, which is the more fundamental cause of the black screen (the camera fix alone wasn't sufficient). Both `@sigma/*` imports now pin the dependency explicitly (`?deps=sigma@3.0.3`). A third bug (icon SVGs missing explicit `width`/`height`, which `@sigma/node-image`'s raster path needs to size a data-URI icon) was caught by an independent second-opinion review before it shipped. Verified end-to-end against the real 20,097-node/1083-community production graph that motivated this feature: 342 aggregated communities render correctly with icons, module colors, and a populated filter panel, zero console errors.
+ A further usability pass, driven by feedback after actually using the rendered graph: (1) the default forceAtlas2 parameters packed 87% of nodes within 15% of the bounding-box center on the same real production graph — genuinely unreadable, not just suboptimal — fixed with `linlog=True`, a `node_size` repulsion halo proportional to member count, and a much higher `scaling_ratio`; (2) edges were a flat gray regardless of relation — they're now colored AND labeled by their dominant relation bucket (`renderEdgeLabels: true`; sigma has no independent edge-label zoom threshold, but ties edge-label visibility to node-label visibility, which manages clutter automatically); (3) nodes are now draggable for manual layout tidying (sigma's own `downNode`/`moveBody`/`upNode`/`upStage` pattern, with the `setCustomBBox` freeze required to stop one dragged node from panning every other node), with a reset button that restores original positions; (4) the click panel now lists each community's actual representative source files, linked via `file://` when `.graphify_root` confirms the graph is being viewed on the machine that generated it; (5) `doc_ref` (a newer AST-derived file_type not in the original six-value schema) now gets an icon instead of silently miscategorizing as code; (6) the filter panel's entity-kind icons and relation-type rows now render visibly (white icons, not black-on-black; a color swatch per relation type matching its edge color).
+ The layout spread was tuned twice: the first pass (`gravity=0.5, scaling_ratio=20`) measured better on a numeric spread proxy but was still reported as too bunched to read labels without manually dragging nodes apart — a second pass (`gravity=0.15, scaling_ratio=80`) actually fixes it, verified by reading real labels at the default fit-all zoom on the 342-community production graph, not by trusting the proxy metric a second time. Node labels also gained their own small background box (sigma's default label renderer is bare `fillText` with no settings-level background option; `defaultDrawNodeLabel` is the documented override point for a fully custom renderer) since plain white text was illegible over a dense, colorful tangle of edges.
+ The click panel was reworked again based on direct feedback: the file list is no longer capped with a "+N more" — Step 1 now sends every distinct file, rendered as a scrollable, filterable list (a filter textbox above a scrollable div, not a flat dump) since a 300-member community can have dozens of files. Clicking a file now opens a movable dialog (dragged by its header, plain screen-pixel CSS `left`/`top`, unrelated to sigma's graph/camera coordinate spaces) showing an embedded content preview — Step 1 reads a bounded preview (3000 characters) from disk for the first 8 files of each community's list, since embedding full content for every file in every community would bloat the self-contained HTML far more than paths alone do. Embedding real file content surfaced a new instance of the script-tag-collision risk: a previewed HTML/JS file (e.g. a Vite `index.html` with `` (#1685, thanks @Kyzcreig). `_TOKENIZER.encode(content)` raises `ValueError` by default when the text contains a special token, which aborted packing on docs/corpora that merely mention these strings. Both `encode` sites now pass `disallowed_special=()` so such text is tokenized as ordinary bytes.
- Fix: the Ollama backend no longer multiplies a hang by the retry count (#1686, thanks @Kyzcreig). A stalled local model would wedge for `timeout * (max_retries + 1)`, which with the default 6 retries turned one long stall into a very long one. Ollama now defaults to zero client-side retries (a local model that stalls will not un-stall on retry); set `GRAPHIFY_MAX_RETRIES` to opt back in. Other backends are unchanged. Note: the underlying stall is non-deterministic and driven by the model server, so this bounds the wait rather than eliminating the hang.
diff --git a/graphify/skill-agents.md b/graphify/skill-agents.md
index 2e19931fa..0d02e4a31 100644
--- a/graphify/skill-agents.md
+++ b/graphify/skill-agents.md
@@ -474,8 +474,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -531,6 +547,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill-amp.md b/graphify/skill-amp.md
index 2e19931fa..0d02e4a31 100644
--- a/graphify/skill-amp.md
+++ b/graphify/skill-amp.md
@@ -474,8 +474,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -531,6 +547,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill-claw.md b/graphify/skill-claw.md
index b3542438d..5c337677e 100644
--- a/graphify/skill-claw.md
+++ b/graphify/skill-claw.md
@@ -477,8 +477,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -534,6 +550,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill-codex.md b/graphify/skill-codex.md
index e6e411ea0..f28ef4cd3 100644
--- a/graphify/skill-codex.md
+++ b/graphify/skill-codex.md
@@ -474,8 +474,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -531,6 +547,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill-copilot.md b/graphify/skill-copilot.md
index b3542438d..5c337677e 100644
--- a/graphify/skill-copilot.md
+++ b/graphify/skill-copilot.md
@@ -477,8 +477,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -534,6 +550,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill-droid.md b/graphify/skill-droid.md
index d1147b903..afa602ccc 100644
--- a/graphify/skill-droid.md
+++ b/graphify/skill-droid.md
@@ -474,8 +474,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -531,6 +547,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill-kilo.md b/graphify/skill-kilo.md
index c45578165..b4892b39c 100644
--- a/graphify/skill-kilo.md
+++ b/graphify/skill-kilo.md
@@ -477,8 +477,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -534,6 +550,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill-kiro.md b/graphify/skill-kiro.md
index b3542438d..5c337677e 100644
--- a/graphify/skill-kiro.md
+++ b/graphify/skill-kiro.md
@@ -477,8 +477,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -534,6 +550,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill-opencode.md b/graphify/skill-opencode.md
index 684e0e223..e923319b3 100644
--- a/graphify/skill-opencode.md
+++ b/graphify/skill-opencode.md
@@ -469,8 +469,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -526,6 +542,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill-pi.md b/graphify/skill-pi.md
index b3542438d..5c337677e 100644
--- a/graphify/skill-pi.md
+++ b/graphify/skill-pi.md
@@ -477,8 +477,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -534,6 +550,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill-trae.md b/graphify/skill-trae.md
index ac16b925d..72fbd9937 100644
--- a/graphify/skill-trae.md
+++ b/graphify/skill-trae.md
@@ -475,8 +475,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -532,6 +548,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill-vscode.md b/graphify/skill-vscode.md
index 31a3352f7..c9592c649 100644
--- a/graphify/skill-vscode.md
+++ b/graphify/skill-vscode.md
@@ -473,8 +473,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -530,6 +546,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill-windows.md b/graphify/skill-windows.md
index 5a55c577b..cd9091aef 100644
--- a/graphify/skill-windows.md
+++ b/graphify/skill-windows.md
@@ -499,8 +499,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -556,6 +572,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skill.md b/graphify/skill.md
index b3542438d..5c337677e 100644
--- a/graphify/skill.md
+++ b/graphify/skill.md
@@ -477,8 +477,24 @@ Substitute `IS_DIRECTED` and `INPUT_PATH` as in Step 4. If a `GRAPH HEALTH WARNI
Read `graphify-out/.graphify_analysis.json`. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
+**At scale (100+ communities), do NOT hand-label one at a time and do NOT fall back to a naive path-prefix heuristic (e.g. title-casing the common directory prefix).** Both break down past a handful of communities — the heuristic in particular produces duplicated-segment junk ("Backend Backend", "Frontend Frontend") and word-soup names that look plausible until you actually read them. Verified in production on a 1000+ community monorepo graph (apa, 2026-07-03): a heuristic pass needed a full redo.
+
+What works instead, for communities with >=20 nodes (the ones that actually surface in God Nodes / navigation):
+1. For each such community, sample its ~15 highest-degree (most connected) member nodes — not the full member list, communities can have 300+ nodes.
+2. Batch communities into groups of ~15-20 per subagent (write each batch's samples to a small JSON file under `graphify-out/`).
+3. Dispatch one `general-purpose` subagent per batch (all in the same message, in parallel — same pattern as Step 3 Part B), each told to read its batch file and write a JSON `{community_id: name}` map based on genuine reasoning about the sampled node labels/paths, not just repeating directory names.
+4. Merge all batch results.
+
+For the long tail of small communities (<20 nodes) where subagent cost isn't justified, a path-prefix heuristic is an acceptable lighter-weight fallback — but dedupe repeated words/segments, strip trailing file extensions, and spot-check a random sample of ~15 outputs across different sizes before trusting it. If duplicated-word or generic ("Community N") names appear in the sample, fix the heuristic before applying it to the rest.
+
Then regenerate the report and save the labels for the visualizer:
+**Reconcile the FULL label set for cross-batch duplicates before regenerating the report — this is a required final pass, not optional polish.** Subagent batches only see their own slice and the small-community heuristic naturally collapses siblings from the same directory (e.g. many different test-file clusters under the same folder), so duplicates are expected even after the steps above — verified in production: 338 of 1083 communities (31%) shared a label with at least one other community before reconciliation (apa monorepo, 2026-07-03). After all labels are assembled:
+1. Group community IDs by their current label.
+2. For every group with >1 member, pick a distinguishing keyword from each member's OWN node labels (word-frequency count, excluding stopwords and words already in the shared base label) and append it, e.g. `"Agent Gateway MCP Client" -> "Agent Gateway MCP Client — Server"` / `"— Additional"`.
+3. If a collision still remains after that (rare — e.g. multiple communities extracted from the very same source image), append a numeric suffix `" (1)"`, `" (2)"` as a last resort.
+4. Re-check for zero duplicates before moving on.
+
```bash
$(cat graphify-out/.graphify_python) -c "
import sys, json
@@ -534,6 +550,8 @@ graphify export html # auto-aggregates to community view if graph > 5000 nodes
# or: graphify export html --no-viz
```
+**If the resulting view (aggregated community graph, or the raw graph for small corpora) has more than ~300 nodes, ALSO generate a Sigma.js + graphology version and present that as the primary interactive file instead of vis-network's `graph.html`.** vis-network runs a live single-threaded JS forceAtlas2 physics simulation on load (200 iterations before it disables itself) — verified in production this is genuinely slow/laggy past a few hundred nodes (apa monorepo, 1083-community aggregated view, 2026-07-03), independent of hardware. Sigma.js (WebGL rendering) with the layout **precomputed offline in Python** (`nx.forceatlas2_layout`, no client-side physics at all) fixes this at the root rather than swapping one JS physics engine for another. See `references/sigma-viz.md` for the full recipe (meta-graph construction, offline layout, self-contained HTML template loading sigma/graphology from esm.sh). Below the 300-node threshold, vis-network's `graph.html` is fine as-is — the extra build step isn't worth it.
+
### Steps 6b-8 - Wiki, Neo4j, FalkorDB, SVG, GraphML, MCP, benchmark (only on their flags)
These run only when their flag is present (`--wiki`, `--neo4j`/`--neo4j-push`, `--falkordb`/`--falkordb-push`, `--svg`, `--graphml`, `--mcp`) or, for the token-reduction benchmark, when `total_words` exceeds 5,000. A default run with no export flags skips all of them. See `references/exports.md` for each one. Run any `--wiki` export before Step 9 cleanup so `.graphify_labels.json` is still available.
diff --git a/graphify/skills/agents/references/sigma-viz.md b/graphify/skills/agents/references/sigma-viz.md
new file mode 100644
index 000000000..62931a554
--- /dev/null
+++ b/graphify/skills/agents/references/sigma-viz.md
@@ -0,0 +1,1011 @@
+# graphify reference: Sigma.js + graphology visualization for large graphs
+
+Load this when Step 6's `graph.html` (vis-network) would render more than ~300 nodes — either the aggregated community view on a large corpus, or the raw graph on a smaller one that still clusters into 300+ communities. vis-network runs a live, single-threaded JS forceAtlas2 physics simulation on load (canvas 2D rendering); past a few hundred nodes this stabilization pass is genuinely slow regardless of hardware. The fix is not swapping to a different JS physics engine — it's removing client-side physics entirely: precompute the layout once in Python (fast, uses networkx's optimized implementation) and render only, with sigma.js's WebGL renderer instead of vis-network's canvas 2D renderer.
+
+Output file: `graphify-out/graph_sigma.html` (self-contained, opens directly like `graph.html`).
+
+Beyond the raw performance fix, this view also encodes things vis-network's `graph.html` doesn't surface at a glance: each community's **dominant content kind** (code/document/paper/image/rationale/concept, drawn as a small icon), its **dominant module** (the top-level directory most of its members live under, drawn as color), a **left-side filter panel** for both entity kind and relation type, **edges colored and labeled by relation type**, **draggable nodes** for manual layout tidying, node labels rendered on their own small background box for readability over a dense tangle of edges, and a **click panel listing every source file a community represents**, filterable and scrollable, where clicking a file opens a **movable dialog with an embedded content preview** (falling back to a working `file://` link when opened on the machine that generated the graph) — so a code-heavy corpus with a handful of docs sprinkled in doesn't read as one undifferentiated blob, and clicking a community gets you to real file content, not just a label.
+
+Good spread alone doesn't read as *structure* — a well-separated layout with every edge drawn at full opacity still looks like a hairball, because nothing visually ties related things together beyond a small color dot. Three more things address that directly: the offline layout **clusters communities by module** (a stronger pull between same-module communities than raw topology alone would produce, so color-coding and position now agree), a **soft region outline is drawn behind each module's cluster** (a translucent hull, not just individual node colors), and **edges below a weight percentile are hidden by default** (with a checkbox to show everything, and any edge touching the currently-highlighted node always shows regardless of weight) so the default view reads as "these are the load-bearing connections" instead of every single one at once.
+
+## Step 1 — build the meta-graph and precompute layout in Python
+
+Adjust `MIN_COMMUNITY_SIZE` (20 is a reasonable default — communities below this rarely appear in God Nodes / navigation and just add render cost) and `INPUT_PATH`/label source to match the current run.
+
+```python
+import json
+import math
+import subprocess
+import networkx as nx
+from pathlib import Path
+from collections import Counter, defaultdict
+
+g_data = json.loads(Path('graphify-out/graph.json').read_text(encoding='utf-8'))
+labels = json.loads(Path('graphify-out/.graphify_labels.json').read_text(encoding='utf-8'))
+# .graphify_root records the absolute scan root on THIS machine, letting the
+# click panel link straight to a source file. file:// links only resolve on
+# the machine that generated the graph - degrade to a plain (still useful,
+# copyable) relative path when the marker is missing.
+_root_marker = Path('graphify-out/.graphify_root')
+SCAN_ROOT = _root_marker.read_text(encoding='utf-8').strip() if _root_marker.exists() else None
+REPO_NAME = Path(SCAN_ROOT).name if SCAN_ROOT else None
+# Best-effort - a scan root that isn't a git repo (or has no git binary
+# available) is common enough (a docs corpus, a partial checkout) that this
+# must degrade to None rather than raise; the header line below just omits
+# the branch when it's unknown instead of showing a misleading placeholder.
+try:
+ GIT_BRANCH = subprocess.run(
+ ['git', '-C', SCAN_ROOT, 'rev-parse', '--abbrev-ref', 'HEAD'],
+ capture_output=True, text=True, timeout=5,
+ ).stdout.strip() or None if SCAN_ROOT else None
+except (OSError, subprocess.SubprocessError):
+ GIT_BRANCH = None
+
+# Maps every relation graphify emits (AST-structural and LLM-semantic alike) to
+# one of six buckets the filter panel toggles as a group. Keep this in sync
+# with the relation vocabulary in graphify/extract.py if new relations are
+# added there — an unmapped relation falls into 'other' rather than crashing.
+RELATION_BUCKETS = {
+ 'calls': 'calls', 'indirect_call': 'calls', 'instantiates': 'calls',
+ 'contains': 'structure', 'defines': 'structure', 'method': 'structure',
+ 'implements': 'structure', 'inherits': 'structure',
+ 'imports': 'imports', 'imports_from': 'imports', 'dynamic_import': 'imports',
+ 're_exports': 'imports', 'depends_on': 'imports', 'crate_depends_on': 'imports',
+ 'requires_env': 'imports',
+ 'references': 'references', 'references_constant': 'references', 'uses': 'references',
+ 'uses_static_prop': 'references', 'bound_to': 'references', 'listened_by': 'references',
+ 'cites': 'docs', 'conceptually_related_to': 'docs', 'shares_data_with': 'docs',
+ 'semantically_similar_to': 'docs', 'rationale_for': 'docs',
+ 'participate_in': 'groups', 'implement': 'groups', 'form': 'groups',
+}
+
+def top_level_dir(source_file: str) -> str:
+ if not source_file or '/' not in source_file:
+ return '(root)'
+ return source_file.split('/')[0]
+
+PREVIEW_CAP = 8 # only the first N files (of the complete list) get embedded content
+PREVIEW_CHARS = 3000 # per-file cap - this is a preview for a dialog, not a full viewer
+
+def read_preview(source_file: str) -> "str | None":
+ """Best-effort file preview for the click-panel dialog. None means
+ "not attempted" (beyond PREVIEW_CAP) or "unreadable" (missing, binary,
+ permission error, no SCAN_ROOT to resolve against) - the dialog falls
+ back to just the file:// link in either case, never raises."""
+ if not SCAN_ROOT:
+ return None
+ try:
+ text = (Path(SCAN_ROOT) / source_file).read_text(encoding='utf-8', errors='replace')
+ except (OSError, UnicodeDecodeError):
+ return None
+ truncated = len(text) > PREVIEW_CHARS
+ return text[:PREVIEW_CHARS] + ('\n… (truncated)' if truncated else '')
+
+MIN_COMMUNITY_SIZE = 20
+node_attrs = {n['id']: n for n in g_data['nodes']}
+comms = defaultdict(list)
+for n in g_data['nodes']:
+ comms[n['community']].append(n['id'])
+
+significant = {cid: members for cid, members in comms.items() if len(members) >= MIN_COMMUNITY_SIZE}
+dropped = len(comms) - len(significant)
+if not significant:
+ # Every community is below the threshold (plausible on a corpus that
+ # clusters into many small communities) - fall back to all of them
+ # rather than crashing on an empty meta-graph downstream.
+ largest = max((len(m) for m in comms.values()), default=0)
+ print(f'No community reaches MIN_COMMUNITY_SIZE={MIN_COMMUNITY_SIZE} (largest: {largest}) - showing all {len(comms)} communities instead.')
+ significant = comms
+ dropped = 0
+node_to_community = {m: cid for cid, members in significant.items() for m in members}
+
+meta = nx.Graph()
+for cid, members in significant.items():
+ # Dominant file_type (majority vote) drives the icon; dominant top-level
+ # directory drives the color. Both are approximations at the community
+ # level — a mixed community shows its majority kind/module, not a blend.
+ type_counts = Counter(node_attrs[m].get('file_type', 'code') for m in members)
+ dir_counts = Counter(top_level_dir(node_attrs[m].get('source_file', '')) for m in members)
+ # Full distinct file list for the click panel - the UI renders it as a
+ # scrollable, filterable list rather than truncating, so send everything
+ # rather than a capped sample. Only the first PREVIEW_CAP get embedded
+ # content (see read_preview) - a 300-member community's worth of full
+ # file contents would bloat the HTML far more than its path list does.
+ distinct_files = sorted({node_attrs[m]['source_file'] for m in members if node_attrs[m].get('source_file')})
+ files_out = [
+ {
+ 'path': f, 'url': f'file://{SCAN_ROOT}/{f}' if SCAN_ROOT else None,
+ 'preview': read_preview(f) if i < PREVIEW_CAP else None,
+ }
+ for i, f in enumerate(distinct_files)
+ ]
+ meta.add_node(
+ cid, member_count=len(members), label=labels.get(str(cid), f'Community {cid}'),
+ file_type=type_counts.most_common(1)[0][0], module=dir_counts.most_common(1)[0][0],
+ files=files_out,
+ )
+
+edge_counts = Counter()
+edge_buckets = defaultdict(Counter)
+# NetworkX <= 3.1 serializes edges as 'links'; some older graph.json files use
+# 'edges' instead (same compatibility hazard graphify/build.py and
+# graphify/affected.py already guard against) - accept either.
+links_key = 'links' if 'links' in g_data else 'edges'
+for link in g_data[links_key]:
+ cu, cv = node_to_community.get(link['source']), node_to_community.get(link['target'])
+ if cu is not None and cv is not None and cu != cv:
+ key = (min(cu, cv), max(cu, cv))
+ edge_counts[key] += 1
+ edge_buckets[key][RELATION_BUCKETS.get(link.get('relation', ''), 'other')] += 1
+
+# Hyperedges (3+ node group relations - participate_in/implement/form) carry
+# no source/target, so they never appear in the loop above. Remap each to the
+# distinct communities its members span and count it into every pairwise
+# combination, mirroring how graphify/export.py's vis-network aggregated view
+# remaps hyperedges to community IDs - otherwise 'groups' in RELATION_BUCKETS
+# is unreachable and the filter panel's "Groups" checkbox always no-ops.
+for he in g_data.get('hyperedges', []):
+ he_communities = sorted({node_to_community[m] for m in he.get('nodes', []) if m in node_to_community})
+ for i in range(len(he_communities)):
+ for j in range(i + 1, len(he_communities)):
+ key = (he_communities[i], he_communities[j])
+ edge_counts[key] += 1
+ edge_buckets[key][RELATION_BUCKETS.get(he.get('relation', ''), 'groups')] += 1
+
+for (cu, cv), w in edge_counts.items():
+ meta.add_edge(cu, cv, weight=w, buckets=dict(edge_buckets[(cu, cv)]))
+
+# Module clustering: boost the LAYOUT-ONLY weight (not the real edge weight,
+# which still drives rendered thickness/labels) for same-module edges, so
+# forceAtlas2's attraction pulls same-module communities toward each other
+# more than the raw topology alone would. Without this, module color-coding
+# (and the hull below) is the only thing tying a module together visually -
+# nodes can be color-matched but scattered anywhere, which reads as chaotic
+# even once the general bunching problem is fixed. Tuned jointly with
+# scaling_ratio below, not independently - re-verify both together on a
+# real multi-hundred-node corpus rather than nudging just one.
+#
+# A too-high boost actively backfires: it doesn't just plateau, it makes
+# bunching WORSE for the largest modules specifically, because a big module
+# has many more same-module edges, so the boost compounds - the two largest
+# modules on the real production graph visibly collapsed into their own
+# dense sub-hairball at boost=15 even though the *average* within/between
+# distance ratio looked fine, and was reported as still-too-clumped against
+# real output. boost=6 (down from boost=15) prioritizes breaking up that
+# hairball over maximizing module separation: within/between distance ratio
+# goes 0.550 -> 0.727 (modules less tightly separated, an accepted
+# trade-off since the hull overlay still visually groups a module even when
+# its nodes aren't packed tightly) while the fraction of nodes within 15%
+# of the bounding-box center drops 0.28 -> 0.12 and the median
+# nearest-neighbor spacing roughly triples (3.9 -> 10.5 units) - measured
+# directly against the current corpus snapshot, not assumed from an older
+# pass, since corpus growth alone can shift these numbers even with
+# unchanged parameters. If module separation ever matters more than spread
+# for a given corpus, raise this back toward 12-15 and re-verify the
+# hairball doesn't return.
+MODULE_CLUSTER_BOOST = 6
+for u, v, d in meta.edges(data=True):
+ same_module = meta.nodes[u]['module'] == meta.nodes[v]['module']
+ d['layout_weight'] = d['weight'] * MODULE_CLUSTER_BOOST if same_module else d['weight']
+
+# offline layout — no client-side physics needed at all.
+#
+# linlog=True (logarithmic attraction) plus a node_size repulsion halo
+# proportional to member count spreads communities out by actual graph
+# structure instead of collapsing almost everything into one dense blob
+# near the centroid. The naive defaults (linear attraction, scaling_ratio=4,
+# no node_size) packed the vast majority of nodes within 15% of the
+# bounding-box center on a real production graph - unreadable, and exactly
+# the "nodes are bunched, labels aren't readable" failure mode reported
+# multiple times against real output, not a synthetic worst case.
+# gravity=0.15 + scaling_ratio=100 (raised again from an earlier 80, paired
+# with the MODULE_CLUSTER_BOOST=6 change above) is the current best
+# verified balance - always verify spread AND module separation together,
+# visually in a real browser, before tuning either further; a purely
+# numeric spread proxy has under-predicted "good enough" before, and
+# pushing scaling_ratio up alone (without lowering the module boost) trades
+# away module separation for less bunching rather than improving both.
+# scaling_ratio is raised so far past its default (2.0) specifically to
+# compensate for linlog's gentler pull once nodes spread out - lowering it
+# re-collapses the layout fast at intermediate values.
+max_members = max((meta.nodes[n]['member_count'] for n in meta.nodes), default=1)
+node_size = {n: 3 + 12 * (meta.nodes[n]['member_count'] / max_members) ** 0.5 for n in meta.nodes}
+pos = nx.forceatlas2_layout(
+ meta, max_iter=1000, gravity=0.15, scaling_ratio=100.0, linlog=True,
+ node_size=node_size, seed=42, weight='layout_weight',
+)
+# A "RuntimeWarning: invalid value encountered in divide" from networkx's
+# linlog attraction calculation is expected and benign (transient zero-
+# distance case during iteration) - verified it does not produce NaN/Inf in
+# the final positions on a real 342-node production graph. Don't treat it as
+# a failure signal.
+
+degrees = dict(meta.degree())
+xs = [float(p[0]) for p in pos.values()]
+ys = [float(p[1]) for p in pos.values()]
+xr, yr = (max(xs) - min(xs)) or 1, (max(ys) - min(ys)) or 1
+scaled = {n: ((float(pos[n][0]) - min(xs)) / xr * 1000, (float(pos[n][1]) - min(ys)) / yr * 1000) for n in meta.nodes()}
+
+# Density-aware sizing: a fixed absolute size range (e.g. "3 to 15") looks
+# fine at a few dozen communities but overlaps into an unreadable blob once
+# forceAtlas2 packs 300+ communities into the same normalized space — the
+# same size value covers a much larger *fraction* of the available room as
+# node count grows. Calibrate against the layout's own median nearest-
+# neighbor distance instead, so sizes stay legible regardless of density.
+def _nearest_neighbor_dists(points):
+ dists = []
+ for i, (x1, y1) in enumerate(points):
+ best = min((math.hypot(x1 - x2, y1 - y2) for j, (x2, y2) in enumerate(points) if j != i), default=None)
+ if best is not None:
+ dists.append(best)
+ return dists
+
+nn = _nearest_neighbor_dists(list(scaled.values()))
+median_nn = sorted(nn)[len(nn) // 2] if nn else 30.0
+SIZE_MIN = max(2.0, median_nn * 0.12)
+SIZE_MAX = max(SIZE_MIN * 2, median_nn * 0.45)
+
+nodes_out = []
+for n in meta.nodes():
+ x, y = scaled[n]
+ deg, mc = int(degrees.get(n, 0)), int(meta.nodes[n]['member_count'])
+ nodes_out.append({
+ 'key': str(n), 'label': meta.nodes[n]['label'],
+ 'x': round(x, 2), 'y': round(y, 2),
+ 'size': round(SIZE_MIN + (SIZE_MAX - SIZE_MIN) * (mc / max_members) ** 0.5, 2),
+ 'degree': deg, 'members': mc,
+ 'fileType': meta.nodes[n]['file_type'], 'module': meta.nodes[n]['module'],
+ 'files': meta.nodes[n]['files'],
+ })
+edges_out = [
+ {'source': str(u), 'target': str(v), 'weight': int(d.get('weight', 1)), 'buckets': d.get('buckets', {})}
+ for u, v, d in meta.edges(data=True)
+]
+
+def _convex_hull(points):
+ """Andrew's monotone chain, pure stdlib - avoids adding scipy as a
+ dependency of this doc just for one shape per module."""
+ pts = sorted(set(points))
+ if len(pts) < 3:
+ return pts
+ def cross(o, a, b):
+ return (a[0] - o[0]) * (b[1] - o[1]) - (a[1] - o[1]) * (b[0] - o[0])
+ lower = []
+ for p in pts:
+ while len(lower) >= 2 and cross(lower[-2], lower[-1], p) <= 0:
+ lower.pop()
+ lower.append(p)
+ upper = []
+ for p in reversed(pts):
+ while len(upper) >= 2 and cross(upper[-2], upper[-1], p) <= 0:
+ upper.pop()
+ upper.append(p)
+ return lower[:-1] + upper[:-1]
+
+# Module hull shading: a soft region outline behind each module's cluster,
+# reinforcing the module-clustering boost above with an explicit visual
+# boundary rather than relying on the viewer to infer "these dots are all
+# the same color, therefore a group" in a dense view. Expand outward from
+# the hull's own centroid so the shape has breathing room around the actual
+# node positions instead of hugging them exactly.
+HULL_MARGIN = 30
+by_module_points = defaultdict(list)
+for n in meta.nodes():
+ by_module_points[meta.nodes[n]['module']].append(scaled[n])
+hulls_out = {}
+for module, points in by_module_points.items():
+ hull = _convex_hull(points)
+ if len(hull) < 3:
+ continue # 1-2 communities in this module - no polygon to draw
+ cx = sum(p[0] for p in hull) / len(hull)
+ cy = sum(p[1] for p in hull) / len(hull)
+ expanded = []
+ for x, y in hull:
+ dx, dy = x - cx, y - cy
+ dist = math.hypot(dx, dy) or 1
+ expanded.append([round(x + dx / dist * HULL_MARGIN, 2), round(y + dy / dist * HULL_MARGIN, 2)])
+ hulls_out[module] = expanded
+
+Path('graphify-out/.graphify_sigma_data.json').write_text(
+ json.dumps({
+ 'nodes': nodes_out, 'edges': edges_out, 'hulls': hulls_out,
+ 'repoName': REPO_NAME, 'scanRoot': SCAN_ROOT, 'gitBranch': GIT_BRANCH,
+ }, ensure_ascii=False), encoding='utf-8')
+print(f'meta graph: {len(nodes_out)} nodes, {len(edges_out)} edges — layout precomputed, size range {SIZE_MIN:.1f}-{SIZE_MAX:.1f}, {dropped} communities below MIN_COMMUNITY_SIZE={MIN_COMMUNITY_SIZE} dropped')
+```
+
+**Important**: cast every numpy value (`forceatlas2_layout` returns numpy floats) to plain Python `float`/`int` before `json.dumps` — numpy scalars aren't JSON-serializable and will raise `TypeError: Object of type float32 is not JSON serializable`.
+
+## Step 2 — write the HTML template
+
+Write this template to `graphify-out/graph_sigma.html`, with a literal `__GRAPH_DATA__` placeholder where the data goes (substituted in Step 3 — do NOT try to embed the JSON directly while authoring the template, string-templating that much escaping inline is error-prone).
+
+Key implementation notes:
+- **Library loading**: sigma@3 ships CJS/ESM only, no browser UMD global. Load libraries as ES modules from `esm.sh` (`https://esm.sh/sigma@3.0.3`, `https://esm.sh/graphology@0.25.4`, `https://esm.sh/@sigma/node-image@3.0.0?deps=sigma@3.0.3`, `https://esm.sh/@sigma/utils@3.0.0?deps=sigma@3.0.3`) via `
+