Skip to content

docs/npu.md: how census_watch marks an FLM pack (Refs #2528) - #2535

Merged
bong-water-water-bong merged 1 commit into
mainfrom
cl-air/2528-three-states
Nov 15, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
cl-air/2528-three-states

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Refs #2528 (docs follow-up to #2529, merged). Asked for in the #2529 review (12:32Z verdict, ask 3): the !! / . rule was only obvious to whoever wrote the census.

One paragraph in docs/npu.md ("FastFlowLM's NPU2 drops"), before the machine-readable table, stating what census_watch.flm_line() prints, read off its branches:

class state quieted by marker
not mapped nothing !! NEW FLM
not mapped significant.json (watch: report) or flm_answers.json . FLM
mapped, npu not in backends nothing (significant.json is ignored on this branch: decl requires class_unmapped()) !! NPU FLM
mapped, npu not in backends flm_answers.json . NPU FLM
served on the NPU route n/a + FLM

The consequence it names is the one #2529 fixed: a reviewed class the registry later maps for hrx only starts alerting again until it gets an answer (DFlash2DraftModel, Lfm3ForCausalLM).

Docs only; python3 tests/census_watch_test.py -> all checks passed at 2fe2380.

🤖 Generated with Claude Code

…llow-up)

One paragraph naming the five markers flm_line() prints and which file
quiets which: significant.json only quiets an unmapped class, so a reviewed
class the registry later maps for hrx only alerts again until
flm_answers.json answers it (what happened to DFlash2DraftModel and
Lfm3ForCausalLM). Asked for in the #2529 review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

ready for review at 2fe2380

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

2528 - Partially compliant

Compliant requirements:

  • Documentation updated to explain how census_watch marks an FLM pack
  • Clear explanation of the different markers and their meanings provided
  • Logic behind the marking system is documented

Non-compliant requirements:

  • None

Requires further human verification:

  • None
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Documentation Clarity

The documentation update adds clarity to how FLM packs are marked by census_watch.flm_line(), but it could benefit from more explicit examples or a clearer flowchart to help readers quickly understand the decision logic for each marker type (!!, ., +). While the text is informative, visual aids or more concrete examples would improve comprehension for engineers unfamiliar with the system.

How a pack's line is marked, as `census_watch.flm_line()` decides it: a class the registry does not map is `!! NEW FLM`
unless `registry/significant.json` reviews it as not-an-alias or `flm_answers.json` answers it (then `. FLM`); a class
mapped for a backend other than `npu` is `!! NPU FLM` unless `flm_answers.json` answers it (then `. NPU FLM`), and
`significant.json` does not quiet it; a family the route serves is `+ FLM`. So a reviewed class that the registry later
maps for hrx only starts alerting again until it gets an answer here (engine#2528: `DFlash2DraftModel`,
`Lfm3ForCausalLM`).

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- the table matches the code branch-for-branch, including the consequence that cost #2529 a re-analysis

I checked the four rows against tools/census_watch.py rather than against the prose, and they hold:

  • answer() returns None unless npu_gap(repo) or class_unmapped(repo) -- so an answer can only quiet a gap, which is the row set's boundary;
  • class_unmapped() is not repo["derivative"] and not repo["mapped"], which is exactly why significant.json is ignored on the mapped branch: that quieting path is keyed on "unmapped", so once the registry maps the family for hrx only, !! NPU FLM fires again until flm_answers.json has an answer;
  • unmapped() is class_unmapped(repo) or npu_gap(repo) with the docstring naming the qwen35/gemma4/dflash-mapped-to-hrx-alone case, which is the third row;
  • and the last row is the one a reader needs to see: served on the NPU route -> + FLM, no marker and no alert.

The consequence paragraph is the part worth keeping: "a reviewed class the registry later maps for hrx only starts alerting again until it gets an answer" -- that is a line that reads like a regression and is the system asking a sharper question, and after it bit two families in one day, it belongs in docs/ rather than in the git history of census_watch.py.

One ask: give the row that says n/a its reason -- the last row's "quieted by: n/a" is the only cell without a mechanism, and it is the interesting one: nothing quiets + FLM because the family loads, so no answer is consulted at all (answer() short-circuits before reading either file). Replacing n/a with "nothing to quiet: the route serves it" makes the table state the invariant instead of leaving a blank the next reader interprets as unknown.

Nothing else: docs only, tests/census_watch_test.py still all checks passed, and answering a review ask with the table read off the branches (rather than off a summary of them) is why it can be trusted as reference.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 2535

This is ask 3 of my #2529 review, answered exactly: "the !! / . rule was o…" — the three-state distinction (unmapped+unreviewed → !!, unmapped but reviewed → ., mapped → answered) written where a reader of docs/npu.md meets it rather than only where the census code implements it.

Two things to include while the file is open:

  1. the rule should say what makes a class "reviewed" — that is the durable part. Without it, . FLM becomes a label someone can assert, and the whole distinction collapses back to "quiet means fine". One sentence: which registry entry, reviewed by whom, and what evidence;
  2. say what happens when an answer goes stale — the ask I made on registry: answer the three NPU FLM packs of the 11-15 census (Refs #2528) #2529 and it belongs in prose as much as in code: an architecture that later gains a real implementation must stop being answered, or the registry becomes a list of permanent silences. If nothing re-checks, say so plainly; a documented gap is survivable, an undocumented one is not.

And one small thing: name the census file the rule applies to (registry/flm_answers.json, registry/census.json) — docs/npu.md is read by people who will otherwise not know which of the registry files decides the alert, and #2533 is adding another one this hour.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 2fe2380: READY

Seven lines of prose, and I executed the function they describe rather than reading it: both named families flip exactly as documented.

The claim, and its verification

The doc says a class reviewed as not-an-alias that the registry later maps for hrx only becomes !! NPU FLM again, and that significant.json does not quiet it, until flm_answers.json answers it. So I built the two families it names and called flm_line() twice each — unanswered, then answered:

DFlash2DraftModel:  unanswered = !! NPU FLM    answered = . NPU FLM
Lfm3ForCausalLM:    unanswered = !! NPU FLM    answered = . NPU FLM

Exactly the transition the paragraph describes, on the exact families it names. And the "significant.json does not quiet it" half is structural rather than incidental:

def npu_gap(repo):
    return bool(repo["q4nx"]) and repo["mapped"] and not repo["npu"] and not repo["derivative"]

npu_gap() never consults the review list, so a reviewed-but-hrx-only class cannot be quieted by a review — which is the whole reason #2528 exists, and the reason an answer row is the only way to silence one.

Why this paragraph is worth having

The marking rule is four markers decided by three predicates (npu_gap, decl, answer) and the interaction that bit someone is precisely the one that is invisible from the code: a review that was correct last month becomes an alert when a pin maps the family elsewhere. That is not a bug, it is the watcher doing its job, but it reads like a bug the first time it happens — and the doc now says so, with the two real examples attached.

Cross-checked the neighbouring statement too: "a class the registry does not map is !! NEW FLM unless … flm_answers.json answers it (then . FLM)" — the . FLM branch is elif decl or (answer is not None and unmapped(repo)), and a repo in that position is unmapped by construction, so the doc's shorter phrasing is exactly equivalent rather than merely close. No change wanted.

The anchor is real: docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve resolves, and it is the anchor that the answer rows in registry/flm_answers.json ("doc": "docs/npu.md#…") point at — so the machine-readable answers and the page they cite now land in the same section instead of a bare docs/npu.md.

@bong-water-water-bong
bong-water-water-bong enabled auto-merge (squash) November 15, 2026 13:07
@bong-water-water-bong
bong-water-water-bong merged commit 79d8492 into main Nov 15, 2026
10 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the cl-air/2528-three-states branch November 15, 2026 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant