Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ There is **no test suite**. Verify by hand:

- Frontend: `docker compose exec frontend npm run build` and `docker compose exec frontend npm run lint` (or run them in `frontend/` without Docker).
- Backend: with `docker compose up` running, `GET localhost:8000/api/health`.
- Analyzer or language-plugin changes: `docker compose exec backend python -m backend.analyzer <path>` on `sample_project`, `sample_rust_project`, `sample_typescript_project` and `sample_c_project` before and after, and compare function and edge counts. An unexplained drop in edges is a regression.
- Analyzer or language-plugin changes: `docker compose exec backend python -m backend.analyzer <path>` on `sample_project`, `sample_rust_project`, `sample_typescript_project`, `sample_c_project` and `sample_cpp_project` before and after, and compare function and edge counts. An unexplained drop in edges is a regression.

CI (`.github/workflows/pr-checks.yml`) only runs the frontend build and the backend health check. It does not lint and does not exercise the analyzer, so green CI says little about analysis correctness.

Expand All @@ -54,6 +54,7 @@ CI (`.github/workflows/pr-checks.yml`) only runs the frontend build and the back
- **The theme is applied in two places.** The inline script in `frontend/index.html` sets `data-theme` before first paint, and `frontend/src/theme.ts` owns it afterwards. They share the storage key and the resolution rules; keep them in sync. Colors belong in the tokens at the top of `index.css`, never as literals (the function card header and the language badge text are the deliberately theme-independent surfaces).
- **Paths must be relative to the picked folder's contents**, with the folder's own name stripped (`localFiles.ts`). Module-name derivation and import resolution in the plugins depend on it.
- **Vocabulary lives in `CONTEXT.md`; use its terms.** The code hasn't caught up on two of them: **File mode** is still `"folder"` in the code (`OrgMode`, `FolderFrame`, `layoutFolderGraph`), and **Trace up / Trace down** is still `showFlow`. Don't rename these as a drive-by; use the glossary term in prose and the code name in code.
- **`.h` is C or C++ per load, not per file.** The C++ plugin's `claims()` takes every `.h` when the picked root holds any C++-only extension (`.cc`, `.cpp`, `.cxx`, `.hpp`, `.hh`, `.hxx`), so the same header is C in one project and C++ in another, and one load's headers never split between the two.
- Function and file ids are namespaced by file path so languages can't collide when merged.
- Only calls between functions that were actually loaded are resolved. Calls inside Rust macro invocations are invisible.
- CORS is `*` on purpose: the GitHub Pages frontend calls the Render API cross-origin.
Expand Down
2 changes: 1 addition & 1 deletion CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ _Avoid_: Start function, root function
What a source file is written in, as shown by its badge, its Explorer icon and its syntax highlighting. A file has exactly one Language.

**Language plugin**:
The support for one programming language, or for a family of closely related ones whose files import each other (TypeScript and JavaScript share one): the piece that knows how to read those languages' functions and their calls. One plugin may read several Languages; each file keeps its own.
The support for one programming language, or for a family of closely related ones whose files import each other (TypeScript and JavaScript share one): the piece that knows how to read those languages' functions and their calls. One plugin may read several Languages; each file keeps its own. C and C++ are separate plugins, so no Call edge joins a C file to a C++ one (a `.h` file goes to whichever of the two reads its Picked root).
_Avoid_: Parser

### Roots
Expand Down
11 changes: 7 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ The goal is to make the flow of a codebase visible instead of jumping between fi
| Rust | `.rs` |
| TypeScript | `.ts`, `.tsx`, `.mts`, `.cts` |
| JavaScript | `.js`, `.jsx`, `.mjs`, `.cjs` |
| C | `.c`, `.h` |
| C | `.c`, `.h` (unless the folder also holds C++ files) |
| C++ | `.cc`, `.cpp`, `.cxx`, `.hpp`, `.hh`, `.hxx`, plus `.h` when the folder also holds C++ files |

More languages can be added without touching the frontend - see [Backend](#backend) below.

Expand All @@ -46,7 +47,7 @@ npm install
npm run dev
```

Open http://localhost:5173 and load a folder or file from the sidebar. Try `sample_project` (Python), `sample_rust_project` (Rust), `sample_typescript_project` (TypeScript and JavaScript) or `sample_c_project` (C) for a quick demo.
Open http://localhost:5173 and load a folder or file from the sidebar. Try `sample_project` (Python), `sample_rust_project` (Rust), `sample_typescript_project` (TypeScript and JavaScript), `sample_c_project` (C) or `sample_cpp_project` (C++) for a quick demo.

## Codebase

Expand All @@ -57,16 +58,18 @@ sample_project/ Python demo fixture used for manual testing
sample_rust_project/ Rust demo fixture used for manual testing
sample_typescript_project/ TypeScript + JavaScript demo fixture used for manual testing
sample_c_project/ C demo fixture used for manual testing
sample_cpp_project/ C++ demo fixture used for manual testing
```

### Backend

- `analyzer.py`: the dispatcher. Groups the files it's given by extension, hands each slice to its language plugin, and merges the results into one graph. Function and file ids are namespaced by path, so merging across languages can't collide.
- `analyzer.py`: the dispatcher. Groups the files it's given by extension, hands each slice to its language plugin, and merges the results into one graph. An extension normally belongs to one plugin, but a plugin may claim one conditionally per load (`conditional_extensions` and `claims()` in `languages/base.py`); that is how `.h` goes to C or C++. Function and file ids are namespaced by path, so merging across languages can't collide.
- `languages/`: one module per language, each implementing the same `LanguagePlugin` contract (`languages/base.py`) - parse every file, extract each function's signature/docstring/source lines, and resolve which loaded functions it calls (direct calls, `module.func()`, `self.method()`, constructors/associated functions). Only calls between functions that were actually loaded get resolved. Adding a language means adding one new module here and registering it in `languages/__init__.py`; nothing else changes.
- `python.py`: parses with the stdlib `ast`.
- `rust.py`: parses with `tree-sitter` + `tree-sitter-rust` (functions, `impl` methods, `use`-based and bare-path module resolution). Calls made inside a macro invocation (`println!`, `format!`, ...) aren't visible to it - macro arguments are an opaque token stream, not parsed expressions.
- `typescript.py`: one plugin for TypeScript and JavaScript, parsed with `tree-sitter` + `tree-sitter-typescript` (the `typescript` grammar for `.ts`, `.tsx` for everything else). It stamps each file with its own language, so a `.ts` file and a `.js` file get their own badge but can call each other. Reads ESM and basic CommonJS imports, resolves relative specifiers and (by unique path suffix) aliases, and ignores `.d.ts` and minified files. There's no separate "JS mode": a missing annotation is just a missing annotation. Its docstring lists exactly what it resolves and what it can't (JSX elements, inheritance, dynamic calls, ...).
- `c.py` (with `cfamily.py`, tree-sitter helpers meant to be shared with a C++ plugin): parses with `tree-sitter` + `tree-sitter-c` and owns `.c` and `.h`. Only definitions are Functions; a prototype in a header never is, but its comment documents a definition that has none. Calls are resolved by name in C's own visibility order: the same file, then files reached through quoted `#include "..."` (and the `.c` file implementing an included header), then the one non-static definition anywhere in the picked folder. A `static` function is visible only in its own file, and two or more candidates mean no wire rather than a guess. Nothing is preprocessed or compiled, so functions defined by macros aren't Functions, calls hidden in macro bodies are invisible, every `#ifdef` branch is parsed, and function-pointer calls (`cb(x)`, `s->handler(x)`) aren't linked, including a call through a parameter or local variable named like a loaded function. A file tree-sitter can't fully parse is flagged but keeps the functions it recovered.
- `c.py` (with `cfamily.py`, tree-sitter helpers shared by the C and C++ plugins): parses with `tree-sitter` + `tree-sitter-c` and owns `.c`, and `.h` unless the C++ plugin claims it. Only definitions are Functions; a prototype in a header never is, but its comment documents a definition that has none. Calls are resolved by name in C's own visibility order: the same file, then files reached through quoted `#include "..."` (and the `.c` file implementing an included header), then the one non-static definition anywhere in the picked folder. A `static` function is visible only in its own file, and two or more candidates mean no wire rather than a guess. Nothing is preprocessed or compiled, so functions defined by macros aren't Functions, calls hidden in macro bodies are invisible, every `#ifdef` branch is parsed, and function-pointer calls (`cb(x)`, `s->handler(x)`) aren't linked, including a call through a parameter or local variable named like a loaded function. A file tree-sitter can't fully parse is flagged but keeps the functions it recovered.
- `cpp.py`: parses with `tree-sitter` + `tree-sitter-cpp`, owns `.cc`, `.cpp`, `.cxx`, `.hpp`, `.hh` and `.hxx`, and claims `.h` when the picked folder holds any of those (a C-only folder's headers stay C). C and C++ are separate plugins, so no wire joins a C file to a C++ one. Functions are definitions with a body: free functions, methods defined in or out of their class, constructors, destructors and operators; lambdas aren't Functions, and a template is one Function. `qualname` is the scoped name (`ns::Class::method`); overloads defined in one file get their parameter types appended (`add(int)`, `at(int) const`), a name defined once keeps its plain name. Calls resolve by name through C++'s scopes (the enclosing class and its loaded bases, enclosing namespaces, `using` declarations, then C's include-based visibility), and when a step finds several candidates the call links only if the argument count leaves exactly one. A member call resolves by the receiver's declared type (a parameter, local variable or data member; smart pointers and aliases unwrapped), falling back to the one loaded method of that name when the type isn't visible (`auto`, a template parameter). Nothing is preprocessed or compiled: no template instantiation, no argument-dependent lookup, no overload resolution by argument types; operator expressions and calls through function pointers or `std::function` aren't linked, and virtual dispatch isn't followed, so a call through a `Base *` links to `Base::run`, never to every override. Macros hide functions and calls as in C.
- `main.py`: the FastAPI app. `GET /api/browse` powers the server-side folder picker, `POST /api/analyze` runs the analyzer and returns the graph as JSON. Also serves the built frontend in production.

### Frontend
Expand Down
20 changes: 19 additions & 1 deletion backend/analyzer.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
"""Multi-language dispatcher: groups sources by extension, hands each slice
to its language plugin (see backend/languages/), and merges the results.

An extension goes to the first plugin that claims it for this load through
`conditional_extensions` / `claims()` (C++ taking `.h` when the root holds a
C++-only file), and otherwise to its unconditional owner.

Every plugin returns the same {"files", "functions", "edges"} shape (see
backend/languages/base.py), so merging across languages is just
concatenation - function/file ids are already namespaced by file path, so
Expand All @@ -12,8 +16,10 @@
import json
import os
import sys
from collections.abc import Collection

from .languages import EXTENSION_TO_LANGUAGE, LANGUAGES, SKIP_DIRS, SUPPORTED_EXTENSIONS
from .languages.base import LanguagePlugin


def _collect_source_files(root: str) -> list[str]:
Expand All @@ -29,15 +35,27 @@ def _collect_source_files(root: str) -> list[str]:
return found


def _owner(ext: str, paths: Collection[str]) -> LanguagePlugin | None:
"""The plugin that takes `ext` in this load: the first (registry order) to
claim it conditionally, otherwise its unconditional owner."""
for lang in LANGUAGES:
if ext in lang.conditional_extensions and lang.claims(ext, paths):
return lang
return EXTENSION_TO_LANGUAGE.get(ext)


def analyze_sources(sources: dict[str, str], root: str = "") -> dict:
"""Analyze in-memory sources, e.g. files picked and read in the browser.

sources: rel_path (forward slashes) -> file content, already read.
"""
by_language: dict[str, dict[str, str]] = {}
owners: dict[str, LanguagePlugin | None] = {}
for rel_path, content in sources.items():
ext = os.path.splitext(rel_path)[1]
lang = EXTENSION_TO_LANGUAGE.get(ext)
if ext not in owners:
owners[ext] = _owner(ext, sources.keys())
lang = owners[ext]
if lang is None:
continue
by_language.setdefault(lang.id, {})[rel_path] = content
Expand Down
13 changes: 9 additions & 4 deletions backend/languages/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,27 @@

To add a language: implement a LanguagePlugin in a new module (see
python.py / rust.py) and add an instance to LANGUAGES below. Nothing
outside this package needs to change - analyzer.py dispatches purely off
file extension.
outside this package needs to change - analyzer.py dispatches off file
extension (and, for an extension a plugin only claims conditionally, such as
C++ taking `.h`, off that plugin's `claims()`; see base.py).
"""

from __future__ import annotations

from .base import SKIP_DIRS, LanguagePlugin
from .c import CLanguage
from .cpp import CppLanguage
from .python import PythonLanguage
from .rust import RustLanguage
from .typescript import TypeScriptLanguage

LANGUAGES: list[LanguagePlugin] = [PythonLanguage(), RustLanguage(), TypeScriptLanguage(), CLanguage()]
LANGUAGES: list[LanguagePlugin] = [PythonLanguage(), RustLanguage(), TypeScriptLanguage(), CLanguage(), CppLanguage()]

# Each extension's unconditional owner; a plugin's `conditional_extensions` are not in here (see analyzer.py).
EXTENSION_TO_LANGUAGE: dict[str, LanguagePlugin] = {ext: lang for lang in LANGUAGES for ext in lang.extensions}

SUPPORTED_EXTENSIONS: frozenset[str] = frozenset(EXTENSION_TO_LANGUAGE)
SUPPORTED_EXTENSIONS: frozenset[str] = frozenset(
ext for lang in LANGUAGES for ext in lang.extensions | lang.conditional_extensions
)

__all__ = ["SKIP_DIRS", "LanguagePlugin", "LANGUAGES", "EXTENSION_TO_LANGUAGE", "SUPPORTED_EXTENSIONS"]
29 changes: 24 additions & 5 deletions backend/languages/base.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
"""Shared types and the plugin contract every backend/languages/*.py module implements.

Each language plugin gets a slice of sources (already filtered to its own
extensions) and returns the same {"files", "functions", "edges"} shape,
built from the same FunctionInfo schema, so the frontend contract stays
identical no matter which languages produced it.
Each language plugin gets a slice of sources (already filtered to the
extensions it owns or claimed for this load) and returns the same
{"files", "functions", "edges"} shape, built from the same FunctionInfo
schema, so the frontend contract stays identical no matter which languages
produced it.
"""

from __future__ import annotations

from abc import ABC, abstractmethod
from collections.abc import Collection
from dataclasses import dataclass, field

# Directories skipped while walking a disk tree, shared across all languages
Expand Down Expand Up @@ -86,10 +88,27 @@ class LanguagePlugin(ABC):
id: str # "python", "rust" - stamped onto each file/function unless the plugin sets its own
label: str # "Python", "Rust"
extensions: frozenset[str] # {".py"}, {".rs"} - dot-prefixed, lowercase
# Extensions this plugin MAY claim, in addition to `extensions` - dot-prefixed,
# lowercase. `extensions` are owned unconditionally (C keeps `.h` as the
# fallback); a conditional extension goes to this plugin only for a load where
# `claims()` says so (C++ takes `.h` when the root holds a C++-only file).
conditional_extensions: frozenset[str] = frozenset()

def claims(self, ext: str, paths: Collection[str]) -> bool:
"""Whether this plugin takes files with the extension `ext` (one of its
`conditional_extensions`) in this load.

`paths` is every relative path (forward slashes) handed to the analyzer,
of any language. The dispatcher asks once per distinct extension per
load, in registry order; the first plugin to claim it wins, and when none
does the unconditional owner (`extensions`) gets the files. Default: False.
"""
return False

@abstractmethod
def analyze(self, sources: dict[str, str]) -> dict:
"""sources: rel_path (forward slashes) -> content, already filtered to `extensions`.
"""sources: rel_path (forward slashes) -> content, already filtered to
`extensions` plus any `conditional_extensions` this plugin claimed.

Returns {"files": [...], "functions": {...}, "edges": [...]} - the
same shape the top-level analyzer.py dispatcher returns, scoped to
Expand Down
3 changes: 2 additions & 1 deletion backend/languages/c.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""C language plugin: tree-sitter-based analysis of C source trees.

Owns `.c` and `.h` (until C++ lands, a header is always C). Nothing is
Owns `.c` and `.h`, except that the C++ plugin claims `.h` when the picked
root contains a C++-only extension; a C-only root is unaffected. Nothing is
preprocessed or compiled: the plugin reads the text of each file as written,
and resolves calls by name, following the way a C program's names are
actually visible.
Expand Down
Loading
Loading