Skip to content

Commit 45feb0e

Browse files
tzhouamclaude
andcommitted
kb init: deepen by owner within each module and always say why it stopped
The afd-plugin pilot re-run on #258 wrote 36 rules on 3 owner pages, reached 74% rule-bearing coverage of an 85% target, and stopped with no note and nothing unfinished. The hottest files still without rules (v1/worker/attention_model_runner.py, ffn_model_runner.py, ...) sit in modules already visited, but their most specific owners (attention-runtime, ffn-runtime) differ from the module's majority owner, and the loop went module by module, so it never reached them. - Deepen now works on groups: the files of one module that share one most specific owner. Groups are ordered by churn; one bounded generator call per group sees only that group's files, and evidence must lie in them. A group whose owner already bears rules is skipped (counted). Caps, budget reservation, D5, pinned claims and the advisory judge are unchanged. - The stop reason is always recorded in the notes (and so the PR body): target reached, budget exhausted (the remaining groups stay in unfinished, labelled with their owner), or "every hot group visited; target not reached (X% < Y%)" with the five hottest files still without rules. Hot files no route reaches are listed too. The pass never ends below target silently. Replayed offline on the pilot (no model calls): from the pre-deepen state the loop makes 5 calls (execution-platforms, model-integration, connectors, attention-runtime, ffn-runtime) and reaches 81.8%; after the module-loop re-run, exactly the attention-runtime and ffn-runtime groups remain to visit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent aa5523a commit 45feb0e

4 files changed

Lines changed: 205 additions & 59 deletions

File tree

‎doc/architecture/SPEC/kb_service.md‎

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -429,16 +429,21 @@ x 必须为正。因阈值停下的调用(`stop_reason="max_budget"`)→ `Mo
429429
与时钟无关(沿 first-parent 的提交时间不一定递减:过期的提交跳过而不是就此停止;路径用 `-z` 读取,非 ASCII 文件名不被
430430
引号转义);合并/squash 式默认分支上每个 first-parent 提交就是一个合入的 PR。这就是设计 §7.3 的 PR 窗口,从镜像离线
431431
读取(不走 GitHub API,没有速率限制)。
432-
- `init_deepen`(阶段 3):按窗口内源文件改动数(`make_include`:源根、`exclude`、语言后缀)给模块排序,从高到低:没有路由
433-
的模块跳过(记入 notes);模块的 owner 是以**最具体**方式(`most_specific`)拥有其文件最多的 owner——同时也覆盖它的更宽
434-
owner 从不拿走组件的规则;该 owner 已承载规则的模块跳过(对指标无增益)——owner **承载规则**指 owner 页本身有生效规则,
435-
或(仅当 owner 是入口页/prose 页时)**同一目录**下有含生效规则的页(不看子目录,否则任何组件有了规则仓库入口就算承载;
436-
owner 本身是规则页却没有生效规则时不承载,不看兄弟页)。其余模块每个
437-
一次生成调用:模块代码带行号、字节上限(这是深度阶段,读代码本身),产出带行范围证据的规则 → 证据必须在本模块文件内 →
438-
共用筛查(D5、钉点证据与声明、落位、咨询性判定)→ 经 `ops.apply_operations` 追加到 owner 的规则页(owner 页本身是规则页
439-
时用它,否则用旁边的 `rules.md`;已有规则只追加、不修改,满页转兄弟页)。每写完一个模块重算承载规则的 PR 加权覆盖率,
440-
达到 `coverage_target`、预算用完或热模块用尽即停。报告:路由/承载规则两种覆盖率的 before/after、窗口 PR 数、仍无规则的热路径(`uncovered_hot`:没有承载规则的 owner 到达的
441-
源文件(按最具体 owner 判断),包括已路由但 owner 无规则的;`unrouted_hot` 另列完全无路由的)。
432+
- `init_deepen`(阶段 3):工作单位是**组**——一个模块里最具体 owner(`most_specific`,并列时取路由顺序第一个)相同的
433+
文件(`_groups`)。所以一个模块的文件分属多个 owner 时,每个 owner 各深化一次,热文件不会因为它所在模块的多数 owner
434+
已有规则而被漏掉(afd-plugin 试点:`attention_model_runner.py` 等热文件的最具体 owner 与模块多数 owner 不同,按模块
435+
循环永远到不了它们)。组按窗口内这些文件的改动数(`make_include`:源根、`exclude`、语言后缀)降序(同数按模块、owner):
436+
没有路由的热文件记入 notes(前 5 个);owner 已承载规则的组跳过(对指标无增益,计数记入 notes)——owner **承载规则**指
437+
owner 页本身有生效规则,或(仅当 owner 是入口页/prose 页时)**同一目录**下有含生效规则的页(不看子目录,否则任何组件
438+
有了规则仓库入口就算承载;owner 本身是规则页却没有生效规则时不承载,不看兄弟页)。其余组每个一次生成调用:只给**该组**
439+
的文件(带行号、字节上限;这是深度阶段,读代码本身),产出带行范围证据的规则 → 证据必须在本组文件内 → 共用筛查(D5、
440+
钉点证据与声明、落位、咨询性判定)→ 经 `ops.apply_operations` 追加到 owner 的规则页(owner 页本身是规则页时用它,否则用
441+
旁边的 `rules.md`;已有规则只追加、不修改,满页转兄弟页)。每写完一组重算承载规则的 PR 加权覆盖率,达到
442+
`coverage_target`、预算用完或热组用尽即停,并**总是**在 notes(因而在 PR 正文)里写一条 `deepen stopped: …`:达到目标;
443+
预算用完(剩余组进 `unfinished`,标 `module <m> (owner <o>)`);或“每个热组都已访问;未达目标(X% < Y%)”——低于目标时
444+
附上仍无规则的最热文件(前 5 个)。低于目标从不静默结束。报告:路由/承载规则两种覆盖率的 before/after、窗口 PR 数、仍无
445+
规则的热路径(`uncovered_hot`:最具体 owner 不承载规则的源文件,包括已路由但 owner 无规则的;`unrouted_hot` 另列完全
446+
无路由的)。
442447
- 同一 PR 打开 shadow:`flip_to_shadow` 对 adapter manifest 做**文本**编辑,只改 `knowledge_lifecycle` 块里同缩进的
443448
`enabled`(→ `true`,缺失则插在块头下)与 `mode`(→ `shadow`)两行,注释、顺序与其他行逐字保留;没有该块 → 在任何模型调用
444449
之前 `blocked`。`_check_flip` 校验:只能改这一个 manifest,`check_flip_to_shadow`(除这两个键外解析值不变、head 开启且为

‎src/infermatrix_copilot/kb_service/init_deepen.py‎

Lines changed: 75 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -3,18 +3,23 @@
33
44
Depth pass. The PR window is the pin's own first-parent history (bounded by
55
``init.pr_window_count`` / ``pr_window_max_age_days``, measured from the
6-
pin's commit time). Modules are taken by churn in that window, highest first;
7-
a module's owner is the one that owns most of its files MOST SPECIFICALLY
8-
(``init_coverage.most_specific``: a catch-all prefix never wins over a
9-
component's), and a module whose owner already bears rules is skipped (it adds
10-
nothing to the metric). For each other module one generator call reads its code at the
11-
pin (byte-bounded, with line numbers) and proposes rules with line-ranged
12-
evidence; the rules go through the shared screening — the docs redundancy
13-
filter (D5), evidence inside the module, pinned claims, placement, the
14-
advisory judge — and are appended to the owner's rule page (never editing an
15-
existing rule; a full page spills to a sibling). The pass stops when the
16-
rule-bearing PR-weighted coverage reaches ``init.coverage_target``, the budget
17-
runs out, or no hot module is left.
6+
pin's commit time). The unit of work is a GROUP: the files of one module that
7+
share one most specific owner (``init_coverage.most_specific``: a catch-all
8+
prefix never wins over a component's), so a module whose files belong to
9+
several owners is deepened once per owner — a hot file is never left behind
10+
because its module's majority owner already has rules. Groups are taken by
11+
churn in the window, highest first; a group whose owner already bears rules is
12+
skipped (it adds nothing to the metric). For each other group one generator
13+
call reads the group's code at the pin (byte-bounded, with line numbers) and
14+
proposes rules with line-ranged evidence; the rules go through the shared
15+
screening — the docs redundancy filter (D5), evidence inside the group's
16+
files, pinned claims, placement, the advisory judge — and are appended to the
17+
owner's rule page (never editing an existing rule; a full page spills to a
18+
sibling). The pass stops when the rule-bearing PR-weighted coverage reaches
19+
``init.coverage_target``, the budget runs out, or no hot group is left, and it
20+
always says which in the record's notes (and so the PR body): a pass never
21+
ends below the target without the reason and the hottest files still
22+
without rules.
1823
1924
The same PR turns the repository's knowledge service on in shadow mode: a
2025
textual edit of exactly the ``enabled`` / ``mode`` lines of the adapter's
@@ -42,6 +47,7 @@
4247
MAX_CODE_BYTES = 100_000
4348
MAX_CODE_FILES = 60
4449
MAX_HOT_LIST = 30
50+
MAX_STOP_FILES = 5
4551
_HEADER = re.compile(r"^knowledge_lifecycle:\s*(?:#.*)?$")
4652
_KEY_LINE = re.compile(r"^(?P<indent>[ \t]+)(?P<key>enabled|mode):(?P<space>[ \t]*)(?P<value>[^#\n]*?)"
4753
r"(?P<comment>[ \t]+#[^\n]*)?(?P<eol>\r?\n?)$")
@@ -127,31 +133,46 @@ def _build(self, tree: Path) -> InitRecord:
127133
max_age_days=init.pr_window_max_age_days)
128134
include = make_include(init.source_roots, init.exclude, suffixes(language))
129135
before = pr_weighted_coverage(prs, owners, include=include, rule_pages=self._rule_pages(owners))
130-
churn = self._churn(prs, modules, include)
136+
groups, unrouted = self._groups(prs, modules, owners, include)
137+
if unrouted:
138+
self.record.notes.append(f"{len(unrouted)} hot source file(s) no route reaches (modules stage), "
139+
f"no rules: " + ", ".join(f"{p} ({n})" for p, n in unrouted[:MAX_STOP_FILES]))
131140
self._ids = self._id_source()
132141
written: dict[str, str] = {}
133142
evidence = []
134143
current = before
135-
for index, (key, count) in enumerate(churn):
136-
if current.rule_bearing_ratio >= init.coverage_target:
137-
self.record.notes.append(f"coverage target {init.coverage_target:.0%} reached; "
138-
f"{len(churn) - index} hot module(s) left as they are")
144+
target = init.coverage_target
145+
skipped = 0
146+
stop = ""
147+
for index, (key, owner, files, count) in enumerate(groups):
148+
if current.rule_bearing_ratio >= target:
149+
stop = (f"coverage target {target:.0%} reached; "
150+
f"{len(groups) - index} hot group(s) left as they are")
139151
break
140-
files = modules[key]["files"]
141-
owner = self._owner(files, owners)
142-
if owner is None:
143-
self.record.notes.append(f"module {key}: no route reaches it (modules stage), no rules")
144-
continue
145152
if owner.path in self._rule_pages(owners):
146-
continue # its owner already bears rules: nothing to add to the metric
153+
skipped += 1 # its owner already bears rules: nothing to add to the metric
154+
continue
147155
try:
148156
kept = self._deepen(tree, key, files, owner)
149157
except BudgetExhausted as exc:
150-
self.record.unfinished += [f"module {k}: {exc}" for k, _ in churn[index:]]
158+
self.record.unfinished += [f"module {k} (owner {o.owner}): {exc}" for k, o, _, _ in groups[index:]]
159+
stop = (f"budget exhausted with {len(groups) - index} hot group(s) left; "
160+
f"{self._short_of(current.rule_bearing_ratio, target)}")
151161
break
152162
written.update({c.rule_id: c.section for c in kept})
153163
evidence += [e for c in kept for e in c.evidence]
154164
current = pr_weighted_coverage(prs, owners, include=include, rule_pages=self._rule_pages(owners))
165+
if not stop:
166+
stop = (f"coverage target {target:.0%} reached; no hot group left"
167+
if current.rule_bearing_ratio >= target
168+
else f"every hot group visited; {self._short_of(current.rule_bearing_ratio, target)}")
169+
if current.rule_bearing_ratio < target:
170+
hottest = self._without_rules(prs, owners, include)[:MAX_STOP_FILES]
171+
if hottest:
172+
stop += "; hottest files without rules: " + ", ".join(f"{p} ({n})" for p, n in hottest)
173+
self.record.notes.append(f"deepen stopped: {stop}")
174+
if skipped:
175+
self.record.notes.append(f"{skipped} hot group(s) skipped: their owner already bears rules")
155176
self.record.coverage = {
156177
"pr_routed": {"before": round(before.routed_ratio, 4), "after": round(current.routed_ratio, 4)},
157178
"pr_rule_bearing": {"before": round(before.rule_bearing_ratio, 4),
@@ -168,12 +189,36 @@ def _build(self, tree: Path) -> InitRecord:
168189

169190
# -- the window ----------------------------------------------------------------
170191
@staticmethod
171-
def _churn(prs, modules, include) -> list[tuple[str, int]]:
172-
"""Modules by source-file changes in the window, highest first."""
173-
from .init_coverage import churn_by_module
192+
def _short_of(ratio: float, target: float) -> str:
193+
return f"target not reached ({ratio:.0%} < {target:.0%})"
174194

175-
filtered = [[f for f in files if include(f)] for files in prs]
176-
return [(k, n) for k, n in churn_by_module(filtered, modules) if n > 0]
195+
@staticmethod
196+
def _groups(prs, modules, owners: list[Owner], include
197+
) -> tuple[list[tuple[str, Owner, list[str], int]], list[tuple[str, int]]]:
198+
"""The hot groups — ``(module, owner, files, churn)`` for every module
199+
and most specific owner of some of its files, churn = changes of those
200+
files in the window — highest churn first (ties by module, then owner),
201+
and the changed source files of modules no route reaches, by churn. A
202+
file several owners own equally goes to the first in routing order."""
203+
changes: Counter[str] = Counter()
204+
for files in prs:
205+
for path in {PurePosixPath(f.strip().strip("/")).as_posix() for f in files}:
206+
if include(path):
207+
changes[path] += 1
208+
grouped: dict[tuple[str, str], tuple[Owner, list[str]]] = {}
209+
unrouted: Counter[str] = Counter()
210+
for key in sorted(modules):
211+
for path in modules[key]["files"]:
212+
owner = next(iter(most_specific(path, owners)), None)
213+
if owner is None:
214+
if changes[path]:
215+
unrouted[path] = changes[path]
216+
continue
217+
grouped.setdefault((key, owner.owner), (owner, []))[1].append(path)
218+
groups = [(key, owner, files, sum(changes[f] for f in files))
219+
for (key, _), (owner, files) in grouped.items()]
220+
groups = sorted((g for g in groups if g[3] > 0), key=lambda g: (-g[3], g[0], g[1].owner))
221+
return groups, sorted(unrouted.items(), key=lambda item: (-item[1], item[0]))
177222

178223
def _without_rules(self, prs, owners: list[Owner], include) -> list[tuple[str, int]]:
179224
"""Changed source files of the window whose most specific owners bear
@@ -186,17 +231,6 @@ def _without_rules(self, prs, owners: list[Owner], include) -> list[tuple[str, i
186231
missed[path] += 1
187232
return sorted(missed.items(), key=lambda item: (-item[1], item[0]))
188233

189-
@staticmethod
190-
def _owner(files: list[str], owners: list[Owner]) -> Owner | None:
191-
"""The owner that owns most of a module's files most specifically
192-
(routing order breaks ties): a broader owner that merely also reaches
193-
them never takes a component's rules."""
194-
counts = {o.owner: 0 for o in owners}
195-
for path in files:
196-
for owner in most_specific(path, owners):
197-
counts[owner.owner] += 1
198-
return max((o for o in owners if counts[o.owner]), key=lambda o: counts[o.owner], default=None)
199-
200234
# -- rule pages ------------------------------------------------------------------
201235
def _active_rules(self, path: str) -> int:
202236
text = self.head.get(path)
@@ -285,7 +319,7 @@ def validate(data: dict) -> None:
285319
for candidate in self._to_candidates(data, page, f"code:{key}"):
286320
outside = [e.get("path") for e in candidate.evidence if str(e.get("path") or "") not in inside]
287321
if outside:
288-
self._drop(candidate, f"evidence outside module {key}: {outside}")
322+
self._drop(candidate, f"evidence outside module {key}'s files owned by {owner.owner}: {outside}")
289323
continue
290324
candidates.append(candidate)
291325
kept = self._screen(candidates)

0 commit comments

Comments
 (0)