diff --git a/CHANGELOG.md b/CHANGELOG.md index 691b1c7ef..f61eef918 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,18 @@ Full release notes with details on each version: [GitHub Releases](https://githu ## Unreleased +- Feat: Sigma.js + graphology fallback for large `graph.html` views, plus batched/reconciled community labeling at scale. vis-network's live client-side forceAtlas2 physics simulation is genuinely slow/laggy past a few hundred nodes regardless of hardware — verified in production on a 1083-community monorepo graph (apa, 2026-07-03). Step 6 of the skill now generates a separate WebGL Sigma.js render (`graph_sigma.html`) with the layout precomputed offline in Python (`nx.forceatlas2_layout`, no client-side physics) once the view exceeds ~300 nodes; the full recipe lives in the new `references/sigma-viz.md`, shipped to every split-platform skill variant. Step 5's community labeling also gains a batched-subagent flow for 100+ communities (sample each community's top-degree nodes, dispatch one `general-purpose` subagent per ~15-20-community batch in parallel) plus a required cross-batch label-reconciliation pass — a naive single-pass label run on the same graph left 338 of 1083 communities (31%) sharing a label with another before reconciliation. + The Sigma view also fixes a node-sizing bug (`autoRescale` must be disabled alongside `itemSizesReference: "positions"`, per sigma's own docs — otherwise sizes stay anchored to the pre-rescale coordinate frame and don't track zoom) and recalibrates node size against the layout's own median nearest-neighbor distance instead of a fixed constant, so communities stay legible instead of overlapping into a blob as node count grows. It now also surfaces each community's dominant content kind as an icon (code/document/paper/image/rationale/concept, via `@sigma/node-image`'s `NodePictogramProgram`) and dominant top-level module as color, plus a left-side panel to filter by entity kind and by relation type (grouped into calls/structure/imports/references/documentation-and-concepts/groups, covering both code and non-code relations) and a clickable module legend to isolate one part of the codebase at a time. + Two more critical rendering bugs were caught and fixed only by actually opening a generated file in a real browser (Chrome via Playwright), not by reading the code: (1) disabling `autoRescale` also disables sigma's automatic "fit the camera to the graph" on load, so without an explicit `fitViewportToNodes` call (from `@sigma/utils`) the camera pointed at empty space and the page rendered as a solid black screen with nothing visible; (2) the `@sigma/node-image` and `@sigma/utils` ESM imports failed at module-load time because esm.sh resolved their `sigma` peer dependency to a broken literal path instead of the pinned version — that import throws before any of the script runs, which is the more fundamental cause of the black screen (the camera fix alone wasn't sufficient). Both `@sigma/*` imports now pin the dependency explicitly (`?deps=sigma@3.0.3`). A third bug (icon SVGs missing explicit `width`/`height`, which `@sigma/node-image`'s raster path needs to size a data-URI icon) was caught by an independent second-opinion review before it shipped. Verified end-to-end against the real 20,097-node/1083-community production graph that motivated this feature: 342 aggregated communities render correctly with icons, module colors, and a populated filter panel, zero console errors. + A further usability pass, driven by feedback after actually using the rendered graph: (1) the default forceAtlas2 parameters packed 87% of nodes within 15% of the bounding-box center on the same real production graph — genuinely unreadable, not just suboptimal — fixed with `linlog=True`, a `node_size` repulsion halo proportional to member count, and a much higher `scaling_ratio`; (2) edges were a flat gray regardless of relation — they're now colored AND labeled by their dominant relation bucket (`renderEdgeLabels: true`; sigma has no independent edge-label zoom threshold, but ties edge-label visibility to node-label visibility, which manages clutter automatically); (3) nodes are now draggable for manual layout tidying (sigma's own `downNode`/`moveBody`/`upNode`/`upStage` pattern, with the `setCustomBBox` freeze required to stop one dragged node from panning every other node), with a reset button that restores original positions; (4) the click panel now lists each community's actual representative source files, linked via `file://` when `.graphify_root` confirms the graph is being viewed on the machine that generated it; (5) `doc_ref` (a newer AST-derived file_type not in the original six-value schema) now gets an icon instead of silently miscategorizing as code; (6) the filter panel's entity-kind icons and relation-type rows now render visibly (white icons, not black-on-black; a color swatch per relation type matching its edge color). + The layout spread was tuned twice: the first pass (`gravity=0.5, scaling_ratio=20`) measured better on a numeric spread proxy but was still reported as too bunched to read labels without manually dragging nodes apart — a second pass (`gravity=0.15, scaling_ratio=80`) actually fixes it, verified by reading real labels at the default fit-all zoom on the 342-community production graph, not by trusting the proxy metric a second time. Node labels also gained their own small background box (sigma's default label renderer is bare `fillText` with no settings-level background option; `defaultDrawNodeLabel` is the documented override point for a fully custom renderer) since plain white text was illegible over a dense, colorful tangle of edges. + The click panel was reworked again based on direct feedback: the file list is no longer capped with a "+N more" — Step 1 now sends every distinct file, rendered as a scrollable, filterable list (a filter textbox above a scrollable div, not a flat dump) since a 300-member community can have dozens of files. Clicking a file now opens a movable dialog (dragged by its header, plain screen-pixel CSS `left`/`top`, unrelated to sigma's graph/camera coordinate spaces) showing an embedded content preview — Step 1 reads a bounded preview (3000 characters) from disk for the first 8 files of each community's list, since embedding full content for every file in every community would bloat the self-contained HTML far more than paths alone do. Embedding real file content surfaced a new instance of the script-tag-collision risk: a previewed HTML/JS file (e.g. a Vite `index.html` with ` + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a + + +``` + +## Step 3 — inject the data and clean up + +```python +import json +import re +from pathlib import Path + +data = json.loads(Path('graphify-out/.graphify_sigma_data.json').read_text(encoding='utf-8')) +html = Path('graphify-out/graph_sigma.html').read_text(encoding='utf-8') +payload = json.dumps(data, ensure_ascii=False) +# A community label OR an embedded file preview containing a literal +# block and corrupt it. File previews make +# this a routine occurrence, not a rare edge case: any previewed HTML/JS/XML +# file (a Vite index.html, a