Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ plus the jobs the view enqueues (`send_invoice_email.delay`, `rebuild_ledger.sen

## App

`loadpath serve --port 7345` opens a local desktop-style UI. Tokens stay on the machine in `~/.loadpath/settings.json`. AI is used **only** for residual uncertainty the graph cannot close.
`loadpath serve --port 7345` opens a local desktop-style UI. Tokens stay on the machine in `~/.loadpath/settings.json`. AI is used **only** for residual uncertainty the graph cannot close. The rail and Settings page ship a dozen themes (Obsidian, Nord, Solarized, Paper, high-contrast, …); the choice stays in `localStorage`.

### Review

Expand Down Expand Up @@ -143,11 +143,16 @@ AST is enough for review. If you need live `_meta` (db_table, resolved relations
| `serializers_are_the_only_published_contract` | Zod/form fields must not drift from the serializer |
| `no_queryset_in_serializer` | Serializers must not run querysets |
| `celery_tasks_must_be_idempotent_on_model_pk` | Celery and Dramatiq tasks take a model pk |
| `queryset_nplusone` | Loops over querysets that touch related objects need `select_related` / `prefetch_related` |
| `cascade_crosses_context` | `on_delete=CASCADE` must not blast into another bounded context |
| `migration_blast_radius` | `RemoveField` / `DeleteModel` still referenced by the typed graph |

Waivers live under `waivers:` in the same file. Reviewers are the `owners` of the bounded contexts in the impact subgraph.

Impact walk skips permission/app/context hubs, does not climb `renders` into the App shell, and does not follow cross-context `relates_to` (an Invoice FK to UserProfile does not pull identity into a billing review).

Review also folds in a **churn & coupling** slice of git history (CodeScene-style hotspots, bus factor, temporal coupling, cyclomatic complexity on the changed functions) scoped to the same impact files — not a whole-repo hotspot map.

## How confidence is scored

Line coverage on changed files is the wrong metric. Loadpath scores the **impact subgraph**:
Expand Down Expand Up @@ -186,4 +191,4 @@ CI installs Chromium and runs the full suite.

## What this is not

Not CodeRabbit (comments without a closed impact set). Not CodeScene (historical coupling). Not django-orm-lens (models only). Not a generic SCIP call graph. The product is review as load-path inspection.
Not CodeRabbit (comments without a closed impact set). Not a CodeScene clone (we do not replace its hotspot maps; we only score churn/coupling on the load path). Not django-orm-lens (we do not boot an ER explorer; we reuse its N+1 / cascade / blast-radius heuristics inside the typed graph). Not a generic SCIP call graph. The product is review as load-path inspection.
8 changes: 8 additions & 0 deletions fixtures/demo_monorepo/backend/billing/services.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,11 @@ def recalculate_total(invoice: Invoice) -> Invoice:
invoice.total = invoice.total
invoice.save(update_fields=["total"])
return invoice


def overdue_account_emails():
"""Classic N+1: related FK accessed per row with no select_related."""
names = []
for invoice in Invoice.objects.filter(status="open"):
names.append(invoice.account.email)
return names
3 changes: 3 additions & 0 deletions fixtures/demo_monorepo/loadpath.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ rules:
- serializers_are_the_only_published_contract
- no_queryset_in_serializer
- celery_tasks_must_be_idempotent_on_model_pk
- queryset_nplusone
- cascade_crosses_context
- migration_blast_radius
django_root: backend
react_root: frontend/src
openapi_paths:
Expand Down
3 changes: 3 additions & 0 deletions loadpath.yml.example
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ rules:
- serializers_are_the_only_published_contract
- no_queryset_in_serializer
- celery_tasks_must_be_idempotent_on_model_pk
- queryset_nplusone
- cascade_crosses_context
- migration_blast_radius
django_root: backend
react_root: frontend/src
openapi_paths: []
Expand Down
161 changes: 161 additions & 0 deletions src/loadpath/architecture/rules.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
"no_queryset_in_serializer": "Serializers must not run querysets.",
"celery_tasks_must_be_idempotent_on_model_pk": "Celery and Dramatiq tasks must take a model pk/id, not a full object payload.",
"async_tasks_must_be_idempotent_on_model_pk": "Celery and Dramatiq tasks must take a model pk/id, not a full object payload.",
"queryset_nplusone": "Querysets iterated in a loop must select_related/prefetch_related related objects they touch.",
"cascade_crosses_context": "on_delete=CASCADE must not blast into another bounded context.",
"migration_blast_radius": "Destructive migrations must not drop fields/models still referenced on the load path.",
}


Expand Down Expand Up @@ -63,6 +66,12 @@ def evaluate(store: GraphStore, config: LoadpathConfig, changed_ids: set[str] |
findings.extend(_queryset_in_serializer(store))
if "celery_tasks_must_be_idempotent_on_model_pk" in enabled or "async_tasks_must_be_idempotent_on_model_pk" in enabled:
findings.extend(_task_idempotency(store, changed_ids))
if "queryset_nplusone" in enabled:
findings.extend(_nplusone(store))
if "cascade_crosses_context" in enabled:
findings.extend(_cascade_crosses_context(store, config))
if "migration_blast_radius" in enabled:
findings.extend(_migration_blast_radius(store))

for f in findings:
f.waived = _waived(config, f.rule, f.node_id)
Expand Down Expand Up @@ -282,3 +291,155 @@ def _task_idempotency(store: GraphStore, changed_ids: set[str] | None) -> list[F
)
)
return out


def _nplusone(store: GraphStore) -> list[Finding]:
out: list[Finding] = []
for node in store.nodes():
hits = (node.get("extra") or {}).get("nplusone") or []
for hit in hits:
accessed = ", ".join(hit.get("accessed") or []) or "related fields"
fix = hit.get("suggested_fix") or ".select_related()"
out.append(
Finding(
rule="queryset_nplusone",
severity=RuleSeverity.WARNING,
message=(
f"{node['name']} loops `{hit.get('loop_var')}` over a queryset and touches {accessed} "
f"without {fix} ({node.get('file_path')}:{hit.get('line')})"
),
node_id=node["id"],
file_path=node.get("file_path"),
extra=hit,
)
)
return out


def _cascade_crosses_context(store: GraphStore, config: LoadpathConfig) -> list[Finding]:
out: list[Finding] = []
fields = {n["id"]: n for n in store.nodes([NodeType.FIELD])}
models = {n["id"]: n for n in store.nodes([NodeType.MODEL])}
for edge in store.edges():
if edge["type"] != EdgeType.RELATES_TO.value:
continue
if (edge.get("extra") or {}).get("on_delete") != "CASCADE":
continue
field = fields.get(edge["src"])
target = models.get(edge["dst"]) or store.get_node(edge["dst"])
if not field or not target:
continue
src_model = field["qualified_name"].rsplit(".", 1)[0]
src_app = (field.get("extra") or {}).get("app") or src_model.split(".")[0]
dst_app = (target.get("extra") or {}).get("app") or (target.get("qualified_name") or "").split(".")[0]
sctx = field.get("context") or config.context_for_django_app(src_app)
dctx = target.get("context") or config.context_for_django_app(dst_app)
if sctx and dctx and sctx != dctx:
out.append(
Finding(
rule="cascade_crosses_context",
severity=RuleSeverity.WARNING,
message=(
f"Deleting {target.get('qualified_name')} ({dctx}) CASCADE-deletes "
f"{src_model} via {field['name']} ({sctx})"
),
node_id=field["id"],
file_path=field.get("file_path"),
extra={"target_context": dctx, "on_delete": "CASCADE"},
)
)
return out


def _migration_blast_radius(store: GraphStore) -> list[Finding]:
out: list[Finding] = []
for op in store.nodes([NodeType.MIGRATION_OP]):
extra = op.get("extra") or {}
kind = extra.get("op")
if kind not in {"RemoveField", "DeleteModel"}:
continue
args = extra.get("args") or []
app = extra.get("app") or ""
still: list[dict] = []
if kind == "RemoveField":
model = extra.get("model_name") or (args[0] if args else None)
field = extra.get("field_name") or (args[1] if len(args) > 1 else None)
if not model or not field:
continue
still.extend(_remaining_field_refs(store, app, model, field))
else:
model = extra.get("model_name") or extra.get("name") or (args[0] if args else None)
if not model:
continue
still.extend(_remaining_model_refs(store, app, model))
seen: set[str] = set()
uniq: list[dict] = []
for node in still:
if node["id"] in seen:
continue
seen.add(node["id"])
uniq.append(node)
if not uniq:
continue
names = ", ".join(sorted({n["qualified_name"] for n in uniq})[:6])
out.append(
Finding(
rule="migration_blast_radius",
severity=RuleSeverity.WARNING,
message=f"{op['name']} still referenced by {names}",
node_id=op["id"],
file_path=op.get("file_path"),
extra={"op": kind, "still": [n["id"] for n in uniq[:8]]},
)
)
return out


def _qnames(store: GraphStore, ntype: NodeType, qname: str) -> list[dict]:
want = qname.lower()
return [n for n in store.nodes([ntype]) if (n.get("qualified_name") or "").lower() == want]


def _ids_for(store: GraphStore, ntype: NodeType, qname: str) -> set[str]:
ids = {node_id(ntype, qname)}
ids.update(n["id"] for n in _qnames(store, ntype, qname))
return ids


def _remaining_field_refs(store: GraphStore, app: str, model: str, field: str) -> list[dict]:
still = list(_qnames(store, NodeType.FIELD, f"{app}.{model}.{field}"))
model_ids = _ids_for(store, NodeType.MODEL, f"{app}.{model}")
serializer_ids: set[str] = set()
want_model = f"{app}.{model}".lower()
for edge in store.edges():
if edge["type"] != EdgeType.SERIALIZES.value:
continue
dst = (edge.get("dst") or "").lower()
if edge["dst"] in model_ids or dst.endswith(":" + want_model):
serializer_ids.add(edge["src"])
for edge in store.edges():
if edge["type"] != EdgeType.HAS_FIELD.value or edge["src"] not in serializer_ids:
continue
child = store.get_node(edge["dst"])
if child and child.get("name") == field:
still.append(child)
return still


def _remaining_model_refs(store: GraphStore, app: str, model: str) -> list[dict]:
still = list(_qnames(store, NodeType.MODEL, f"{app}.{model}"))
model_ids = _ids_for(store, NodeType.MODEL, f"{app}.{model}")
want = f"{app}.{model}".lower()
for edge in store.edges():
dst = (edge.get("dst") or "").lower()
if edge["dst"] not in model_ids and not dst.endswith(":" + want):
continue
if edge["type"] in {
EdgeType.SERIALIZES.value,
EdgeType.QUERIES_MODEL.value,
EdgeType.RELATES_TO.value,
}:
src = store.get_node(edge["src"])
if src:
still.append(src)
return still
3 changes: 3 additions & 0 deletions src/loadpath/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@
"serializers_are_the_only_published_contract",
"no_queryset_in_serializer",
"celery_tasks_must_be_idempotent_on_model_pk",
"queryset_nplusone",
"cascade_crosses_context",
"migration_blast_radius",
]


Expand Down
51 changes: 35 additions & 16 deletions src/loadpath/extractors/django.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,10 @@ def _kw(call: ast.Call, key: str) -> ast.AST | None:
return None


def _truthy(node: ast.AST | None) -> bool:
return isinstance(node, ast.Constant) and node.value is True


def _app_from_path(rel: str) -> str | None:
parts = Path(rel).parts
# backend/billing/models.py → billing
Expand Down Expand Up @@ -323,6 +327,8 @@ def _model(self, node: ast.ClassDef) -> None:
on_delete = _name(od)
extra["on_delete"] = on_delete.split(".")[-1] if on_delete else None
extra["related_name"] = _const_str(_kw(stmt.value, "related_name"))
extra["db_index"] = _truthy(_kw(stmt.value, "db_index"))
extra["unique"] = _truthy(_kw(stmt.value, "unique"))
field_node = self.add_node(NodeType.FIELD, fname, field_q, stmt.lineno, extra)
self.add_edge(model.id, field_node.id, EdgeType.HAS_FIELD)
if rel_to:
Expand Down Expand Up @@ -921,14 +927,23 @@ def extract_migrations(rel_path: str, source: str, config: LoadpathConfig) -> Ex
s = _const_str(a) or _name(a)
if s:
args_repr.append(s)
for kw in node.keywords:
if kw.arg in {"name", "model_name"}:
s = _const_str(kw.value) or _name(kw.value)
if s:
args_repr.append(s)
label = f"{short}({', '.join(args_repr[:3])})"
kw: dict[str, str] = {}
for keyword in node.keywords:
if not keyword.arg:
continue
s = _const_str(keyword.value) or _name(keyword.value)
if s:
kw[keyword.arg] = s
extra = {"op": short, "app": app, "args": args_repr, **kw}
if short == "RemoveField":
extra["model_name"] = kw.get("model_name") or (args_repr[0] if args_repr else None)
extra["field_name"] = kw.get("name") or (args_repr[1] if len(args_repr) > 1 else None)
elif short == "DeleteModel":
extra["model_name"] = kw.get("name") or (args_repr[0] if args_repr else None)
label_bits = [extra.get("model_name") or "", extra.get("field_name") or ""]
label_bits = [b for b in label_bits if b] or args_repr[:3]
label = f"{short}({', '.join(label_bits)})"
qname = f"{app}.{Path(rel_path).stem}.{label}"
extra = {"op": short, "app": app, "args": args_repr}
n = Node(
id=node_id(NodeType.MIGRATION_OP, qname),
type=NodeType.MIGRATION_OP,
Expand All @@ -949,16 +964,17 @@ def extract_migrations(rel_path: str, source: str, config: LoadpathConfig) -> Ex
extra={"op": short},
)
)
if short == "RemoveField" and args_repr:
model = args_repr[0]
field = args_repr[1] if len(args_repr) > 1 else "?"
graph.edges.append(
Edge(
src=n.id,
dst=node_id(NodeType.FIELD, f"{app}.{model}.{field}"),
type=EdgeType.DESTRUCTIVE_MIGRATION,
if short == "RemoveField":
model = extra.get("model_name")
field = extra.get("field_name")
if model and field:
graph.edges.append(
Edge(
src=n.id,
dst=node_id(NodeType.FIELD, f"{app}.{model}.{field}"),
type=EdgeType.DESTRUCTIVE_MIGRATION,
)
)
)
return graph


Expand All @@ -974,6 +990,9 @@ def extract_django_file(rel_path: str, source: str, config: LoadpathConfig) -> E
return g
extractor = DjangoExtractor(rel, source, config)
extractor.visit(tree)
from loadpath.orm.nplusone import apply_nplusone

apply_nplusone(extractor.graph, tree)
return extractor.graph


Expand Down
1 change: 1 addition & 0 deletions src/loadpath/orm/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""ORM-lens style static checks that feed the load-path graph."""
Loading
Loading