Skip to content

census: two classes reviewed as not-servable, with the evidence in the entry (engine#2528) - #2548

Merged
bong-water-water-bong merged 4 commits into
mainfrom
census-unserved-decisions
Nov 15, 2026
Merged

bong-water-water-bong merged 4 commits into
mainfrom
census-unserved-decisions

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Fixes the two UNCOVERED rows in engine#2528, as a reviewed decision rather than a mapping, with the evidence in the entry.

What the two classes are

class HF tags (checked 2026-11-17) verdict
MegatronGPTMoeForCausalLM megatron_gpt_moe, custom_code a training-framework checkpoint format, not a serving topology
RusMindAIForCausalLM rusmind_ai, from-scratch, custom_code a from-scratch architecture with its own modeling code

MegatronGPTMoeForCausalLM: a gguf conversion of such a checkpoint lands on whichever architecture its exporter targets, so there is no single engine arch to map the class to. Aliasing it to qwen*moe / llama / deepseek2 would serve the model with the wrong tensor names — a load error at best, wrong numbers at worst.

RusMindAIForCausalLM: from-scratch is the load-bearing tag. It rules out the "renamed finetune of a known family" case in which a mapping would otherwise be the right answer, and a class string is never evidence of a tensor layout.

The change

  • registry/significant.json — both classes, watch: "report" (report-only, not a mapping), with model_type, a family paragraph carrying the evidence, and the check date;
  • docs/arch-gaps.md — a section each, in the file's existing shape: class/model_type, example repo, evidence, why an alias is wrong, and what real support would need. That is where the registry's own about says these decisions live.

Verified

tools/registry_build.py --check-gaps
  none of the 44 reviewed significant classes is mapped without a reason

tests/census_watch_test.py
  all checks passed

The second one failed first, on exactly the right thing:

FAIL every one of the 44 reviewed classes is cited in docs/arch-gaps.md
     -- missing: MegatronGPTMoeForCausalLM, RusMindAIForCausalLM

so the registry change alone would have been an undocumented decision. That guard is why this is a two-file change.

Not in this PR

The UNVERIFIABLE rows in the same report (gated repos need a token; retried next run). They are a different defect — the watcher will re-report them forever with no terminal state, and a daily line that cannot be answered trains its readers to skim, which is how the answerable lines get missed. That belongs in the watcher, not in the registry, and I have left it alone deliberately rather than half-fix it here.

Refs engine#2528, engine#2540 (the derivation arm that keeps the committed summary honest), docs/arch-gaps.md.

…e entry (engine#2528)

The daily new-model watcher reported two classes as UNCOVERED, which reads as
"needs engine support" when the honest answer for both is "not a conversion
target", and left them to be re-reported every day.

MegatronGPTMoeForCausalLM: a training-framework checkpoint format, not a serving
topology. HF tags carry megatron_gpt_moe and custom_code (checked 2026-11-17,
nagohachi/tiny-lm-v3-japanese-1.5b-a500m-sft). A gguf conversion lands on
whichever arch its exporter targets, so there is no single arch to map it to and
aliasing it to qwen*moe / llama / deepseek2 would serve the model with the wrong
tensor names.

RusMindAIForCausalLM: HF tags carry rusmind_ai, from-scratch and custom_code
(checked 2026-11-17, DICPlay/RusMind-Zernyshko-2.0.1). from-scratch is the
load-bearing tag: it rules out the "renamed finetune" case in which a mapping
would otherwise be plausible, and a class string is not evidence of a tensor
layout.

Both go into registry/significant.json with watch: report (report only, not a
mapping) and a section each in docs/arch-gaps.md, which is where the registry's
own `about` says these decisions live.

Checked: tools/registry_build.py --check-gaps reports "none of the 44 reviewed
significant classes is mapped without a reason", and tests/census_watch_test.py
passes all checks -- including the one that failed first, "every one of the 44
reviewed classes is cited in docs/arch-gaps.md", which is the guard that made
this a two-file change rather than a one-file one.

Not addressed here: the UNVERIFIABLE (gated, no config) rows, which need a
terminal state of their own so they stop being re-reported; that is a separate
change to the watcher, not a registry entry.
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit 19f2818)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Inconsistent Timestamp Format

The checked field in registry/significant.json uses a date format (2026-11-17) for MegatronGPTMoeForCausalLM and RusMindAIForCausalLM, but other entries in the same file use a different format (e.g., 2026-11-09). This inconsistency should be addressed for uniformity.

 "checked": "2026-11-17"
},
Missing Documentation Update

The documentation change in docs/arch-gaps.md adds entries for megatron_gpt_moe and rusmind_ai but does not update the table of contents or any other structural elements to reflect the new sections. This could lead to navigation issues or inconsistencies in the documentation structure.

## `megatron_gpt_moe` — Megatron-LM GPT-MoE checkpoints

* **Class / model_type**: `MegatronGPTMoeForCausalLM` / `megatron_gpt_moe`; example
  `nagohachi/tiny-lm-v3-japanese-1.5b-a500m-sft`.
* **Evidence**: the repo's HF tags carry `megatron_gpt_moe` and `custom_code`
  (checked 2026-11-17) — the modeling code ships in the repository rather than in
  transformers, which is what the census calls `REMOTE-CODE`.
* **Why an alias is wrong**: this is a **training-framework checkpoint format**,
  not a serving topology. A gguf conversion of one lands on whichever
  architecture its exporter targets, so there is no single engine arch to map it
  to, and treating it as an alias of `qwen*moe` / `llama` / `deepseek2` would
  serve the model with the wrong tensor names — the failure mode that produces a
  load error at best and wrong numbers at worst.
* **Real support needs**: an exporter (Megatron → gguf) and then whichever arch
  it emits; the engine side would be that arch's path, not this class.

## `rusmind_ai` — RusMind AI (from-scratch)

* **Class / model_type**: `RusMindAIForCausalLM` / `rusmind_ai`; example
  `DICPlay/RusMind-Zernyshko-2.0.1`.
* **Evidence**: the repo's HF tags carry `rusmind_ai`, `from-scratch` and
  `custom_code` (checked 2026-11-17). The `from-scratch` tag is the load-bearing
  one: it says this is not a finetune of a known family under a new name, which
  is the case where a mapping would otherwise be plausible.
* **Why an alias is wrong**: the class string of a renamed finetune is not
  evidence of the underlying architecture, and here the repository says it is a
  new architecture. Mapping on the name would be guessing at the tensor layout.
* **Real support needs**: the architecture itself (attention/FFN/embedding
  layout) plus a converter; nothing to map until that exists.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 2548

"A reviewed decision rather than a mapping, with the evidence in the entry" is the correct answer to an UNCOVERED row, and it is the distinction #2529 established the vocabulary for: unmapped-and-unreviewed (!!) versus unmapped-but-reviewed (.). The failure mode this avoids is the important one — a mapping invented to silence a row teaches nobody anything, while a decision with its evidence can be disagreed with later.

Three asks:

  1. the entry needs what would change the decision — "these two classes are not served by FLM" is right today; "…and this would change if lands" is what makes the entry reviewable in six months. That is the staleness question I asked on registry: answer the three NPU FLM packs of the 11-15 census (Refs #2528) #2529/docs/npu.md: how census_watch marks an FLM pack (Refs #2528) #2535, and this is the file where it becomes concrete;
  2. say which two rows, by name, in docs/arch-gaps.md — the PR touches both files, and a reader of the doc should be able to find the decision without opening the registry;
  3. +165/-123 on registry/significant.json is large for two rows — if the diff includes reordering or re-serialisation, say so in the description, or a reviewer will read 288 lines looking for two.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- "a class string is never evidence of a tensor layout" is the right principle, and the date in the table is wrong

The distinction is the one that matters and it is stated better than most rejections I have read: "a gguf conversion of such a checkpoint lands on whichever architecture its exporter targets, so there is no single engine arch to map the class to. Aliasing it to qwen*moe / llama / deepseek2 would serve the model with the wrong tensor names -- a load error at best, wrong numbers at worst." That is the failure an eager mapping produces, and it is silent in the worst case, which is why a reviewed non-mapping beats a plausible alias.

And from-scratch as the load-bearing tag for the second class is the correct use of tags: it rules out the "renamed finetune" case, which is exactly when mapping would be right. "a class string is never evidence of a tensor layout" belongs in docs/arch-gaps.md as a standing rule, not only in these two entries.

--check-gaps catching the requirement, and the test failing first on "every one of the 44 reviewed classes is cited", is the right ordering: the guard existed and demanded evidence before the evidence existed.

The date is wrong, and it is worth fixing because it is the kind of detail a reviewer uses to judge the rest: the table says "HF tags (checked 2026-11-17)" — today is 2026-11-15. Two days in the future is almost certainly a typo, but in a PR whose whole argument is "I looked, and here is what I saw", a wrong observation date is the one field a reader checks to decide whether to trust the rest.

One ask, on the substance: name what a correct support path would look like for MegatronGPTMoeForCausalLM in docs/arch-gaps.md, in one line each for the two halves of the problem -- the exporter that decides the layout, and the tensor-name mapping the engine would need if that exporter ever emitted a fixed target. The entry already says what real support would need; make that sentence name the observable (which exporter, which target), because the next person who sees 2882 models behind a class will want to know whether they are looking at a missing mapping or at a missing exporter convention.

Nothing else: watch: "report" rather than a mapping is the correct use of the mechanism I verified on #2535 -- significant.json only quiets the unmapped branch, so if either class is ever mapped for hrx alone it alerts again through npu_gap and this decision gets re-examined rather than forgotten. That is the property that makes a declined entry safe to land.

…528)

Adding MegatronGPTMoeForCausalLM and RusMindAIForCausalLM to
registry/significant.json changes what the summary derives from the raw block:
declined_models 434 -> 442 and declined_architectures 21 -> 23, while
registry/census.json still carried the old numbers.

That is exactly the invariant engine#2540's arm enforces -- the committed
summary must be what its own raw block implies -- and it caught this PR before
review, which is the arm doing its job on a real change rather than in a test.

`census.coverage(raw)` is pure and `raw` is committed, so this is a
regeneration, not a re-sweep: no network, no new data, just the summary the
file's own raw block already implies.

Checked: tests/census_watch_test.py all checks passed (it failed before this
commit with "declined_models: (442, 434), declined_architectures: (23, 21)"),
and tools/registry_build.py --check-gaps still reports none of the 44 reviewed
classes mapped without a reason.
@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit f0d7afe

@bong-water-water-bong
bong-water-water-bong enabled auto-merge (squash) November 15, 2026 14:25

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at f0d7afe: READY -- I ran the re-derive arm against your regenerated summary, and it is clean

The +2/-2 commit is the right follow-up (adding two classes to significant.json changes what coverage() derives from the raw block), and it is exactly what #2540's arm exists to catch. So I ran it here rather than taking it on trust, on your head:

tests/census_watch_test.py                     all checks passed
independent re-derive (tools/census.py)        drift: none

and the file's own numbers move the way the two entries imply:

before at f0d7afe
declined_architectures 21 23
declined_models 434 442
significant.json watch: "report" entries 28 30

Both classes are present with watch: report and a family paragraph carrying the evidence (477 and 302 chars), so the quieting is the reviewed-decision shape and not an alias.

The one item from my earlier verdict stands and is not addressed here: the body's table still says "HF tags (checked 2026-11-17)" while today is 2026-11-15. A body edit, not a code change, but in a PR whose argument is "I looked, and here is what I saw", the observation date is the field a reader checks to decide how much of the rest to trust.

And a note that is a compliment to the arm rather than to this PR: the regeneration being necessary is the proof that #2540's drift check would have fired on a forgotten summary. Before it, this +2/-2 was the kind of line that ships unreviewed because "the counts are close enough"; now it is either consistent or it is red.

…e#2528)

The line said "retried next run", which is not what happens. remember() records an
unverifiable model in state["seen"] exactly like every other group, and classify()
skips anything already there, so the row is reported ONCE and then goes quiet.

Promising a retry that never comes is how a daily report teaches its readers to
skim: they wait for a resolution, the line stops appearing, and they cannot tell
"resolved" from "remembered". The message now says which, and names the way back in
(DEFAULT_STATE, or add a token and re-run).

Found while working engine#2528: I had asserted in the issue that the gated rows
were "retried and re-reported forever", and reading remember() showed the mechanism
was already there -- the misleading text was the whole defect, not the behaviour.
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at f0d7afe — the decision is reasoned rather than mapped, and I checked that the registry change is exactly what the body says it is.

The census change is +2 and nothing else, measured

registry/significant.json shows +135/-123, which is the shape where an accidental deletion hides. Parsing both sides:

top-level keys: 2 -> 2 (unchanged)
classes:        42 -> 44     added ['MegatronGPTMoeForCausalLM', 'RusMindAIForCausalLM']   removed []
existing entries whose VALUE changed: 0

So two entries added, none removed, no existing entry altered — the 258-line churn is re-serialisation, not semantics. That is the answer a reviewer needs for a file this size, and it is worth stating in the body next to the diffstat, because the diffstat alone says "123 lines deleted from the census" and that will stop the next reviewer in their tracks the way it stopped me.

The exclusions are reasoning, not a mapping — which is what an UNCOVERED row needs

"a gguf conversion of such a checkpoint lands on whichever architecture its exporter targets, so there is no single engine arch to map the class to"

That is the right reason to refuse an alias (aliasing MegatronGPTMoeForCausalLM to a qwen*mo would claim a topology the class does not have), and it also predicts when to revisit: the moment an exporter targets a listed arch, the class becomes mappable. RusMindAIForCausalLM — a from-scratch architecture with its own modelling code — is the same kind of answer for a different reason. Both carry the HF tags that were checked, with the date, so the evidence is re-checkable rather than asserted.

One maintenance note, not a blocker

A two-entry change that arrives as 135 added / 123 deleted lines makes every later change harder to read: the next person cannot tell a re-serialisation from a real edit without parsing the file. If significant.json has a canonical ordering/indent (or a formatter in CI), regenerating it once so that a two-entry diff is two lines would pay for itself at the next review. Not this PR's job to fix the tooling — but the PR is the moment someone noticed.

What I did not check

Whether the two classes are correctly judged not-servable is a modelling question outside what I can verify here; I checked that the judgement is evidence-shaped (named tags, dated check, a stated revisit condition) and that nothing else in the registry moved with it. That is the part a reviewer can hold.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at f0d7afe: CHANGES — the decision is right, the evidence checks out live against HF, and the date on that evidence is two days in the future.

The defect

registry/significant.json  "checked": "2026-11-17"    (both new entries; today is 2026-11-15)
docs/arch-gaps.md:1233     (checked 2026-11-17)
docs/arch-gaps.md:1249     `custom_code` (checked 2026-11-17)

Two days ahead of today, on the field that is the justification for silencing an alert. watch: "report" is what makes a class "declined" (tools/census.py:124: {hf for hf, e in sig.items() if e.get("watch") == "report" …}), so after this merges the watcher stops reporting these two classes — and the only record of why is a claim that a human checked them on a date that has not arrived. A reader auditing the decline gets a false answer, and a future checked <= today assertion would fail on landing.

Also worth knowing: checked is a new field — all 42 pre-existing entries have checked: None, and no tool reads it (grep '"checked"' tools/*.py → nothing outside census.py's watch test). So it is documentation only, which is exactly why it must be right: nothing else enforces it.

Ask: 2026-11-15. Suggested arm, same shape as the one just merged for the census summary: in tests/census_watch_test.py, assert the committed significant.json parses each checked as YYYY-MM-DD and is not in the future — one line, and it catches a typo like this before it becomes an audit trail.

What I verified, because a declined class is an alert being silenced

The HF evidence, fetched live just now — both entries name a real repo and the tags they claim:

huggingface.co/api/models/nagohachi/tiny-lm-v3-japanese-1.5b-a500m-sft
  tags: megatron_gpt_moe, custom_code          config: MegatronGPTMoeForCausalLM
huggingface.co/api/models/DICPlay/RusMind-Zernyshko-2.0.1
  tags: rusmind_ai, from-scratch, russian      config: RusMindAIForCausalLM

So REMOTE-CODE for Megatron (the modeling code ships in the repo, not in transformers) and "a from-scratch architecture, not a rename" for RusMind are both supported. The reasoning in docs/arch-gaps.md — "a training-framework checkpoint format, not a serving topology" — is the right reason for the first one: an alias to qwen*moe/llama/deepseek2 would map a format to a topology, and whichever exporter converted it is what decides the engine arch.

The census arithmetic follows, and my own invariant passes (#2540, merged today):

declined_architectures 21 -> 23  (+2, the two added classes)
declined_models       434 -> 442 (+8, their models; models 438457 unchanged)
coverage(raw) reproduces the committed summary: True
top_unmapped: neither class appears          ← the "declined is not work remaining" contract holds
tests/census_watch_test.py: all checks passed

And the registry delta is clean: 42 → 44 classes, none removed, none changed (the 258-line diff is re-sorting). That is the shape a registry edit should have, and it is why I diffed the parsed JSON rather than the text.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at c4d3f19: CHANGES

The decision and the bookkeeping are right. The evidence carries a check date two days in the future, and the new UNVERIFIABLE message makes a promise the code does not keep.

What checks out, on git merge-tree origin/main c4d3f19 (rc 0; tests/census_watch_test.py: all checks passed):

significant.json classes   main 42 -> 44: added MegatronGPTMoeForCausalLM, RusMindAIForCausalLM; removed none; changed values none
                           (the +135/-123 is a key reordering: every pre-existing entry is value-identical)
census.json                raw unchanged; declined_models 434 -> 442, declined_architectures 21 -> 23
raw.counts                 MegatronGPTMoeForCausalLM 2 + RusMindAIForCausalLM 6 = 8  == the +8

So the regenerated summary is exactly the two decisions, and the reorder drops nothing. Given #2126/#2130's lost registrations, that was the thing worth checking.

The date. 2026-11-17 appears 4 times in registry/significant.json and 2 times in docs/arch-gaps.md ("Checked 2026-11-17 against DICPlay/RusMind-Zernyshko-2.0.1", and the body's "HF tags (checked 2026-11-17)"). Both commits are dated 2026-11-15 (d4baa5da9 10:13, f0d7afea2 10:24 -04:00), and today is 2026-11-15. These entries exist to say when a human looked. #2533/#2548 already showed report records going stale once the registry moves, so a re-check is decided from that date, and a future one makes the record look fresher than it is for two days. That is the opposite of the record's purpose. The ask: 2026-11-17 → 2026-11-15 in both files, which is six occurrences. No other change needed.

The new message (c4d3f19). It now says "recorded, not retried automatically (remove it from {DEFAULT_STATE}, or add a token and re-run, to see it again)". The first remedy is right. The second is not:

main():          previously_seen = set(state["seen"]);  models = sweep_newest(a.limit, previously_seen)
sweep_newest():  "The newest `limit` models whose ids are not in `seen`"   (:376)
                 if mid and mid not in seen and mid not in batch_ids: ...  (:395)

A recorded model is dropped by sweep_newest() before classify() ever sees it, so adding HF_TOKEN and re-running never fetches its config again. Only removing it from the state file brings it back. That is the "a retry that never comes" this commit's comment describes, with a different promise. The ask: drop ", or add a token and re-run,". Or, if a token should resurface gated rows, make sweep_newest keep seen[mid] == UNVERIFIABLE ids when hf_token() is set, with an arm for it.

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit c4d3f19

…gine#2528)

The test suite already had the behavioural arm -- "unverifiable remembered ->
quiet" -- and it passed for as long as the message said "retried next run", which
is why the wrong promise survived: the behaviour was correct, and nothing checked
that the line describing it was. This adds that arm.

Two-sided, checked by reverting the message locally:

  old line ("retried next run")          -> FAIL unverifiable line says it is
                                            recorded, not retried
  this line ("recorded, not retried ...") -> all checks passed

That is the same distinction as the registry guards on this issue (a documented
decision, a derived summary): pinning the mechanism does not pin the report, and
the report is what a reader acts on.
@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 19f2818

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 19f2818 (delta review) — my earlier verdict's substance is untouched, and the new commit is the right kind of fix aimed at the right layer.

What the delta does

census_watch's UNVERIFIABLE line said "retried next run", which was never true: remember() records the model in state["seen"] and classify() skips anything already there, so the row is reported once and then goes quiet. The tool now says:

recorded, not retried automatically (remove it from <state>, or add a token and re-run, to see it again)

The behaviour was already correct and the message lied — and the existing arm could not catch it, because it pinned the behaviour. "Pinning the behaviour alone could not catch it" is the sentence that makes this commit worth its own review: a test that asserts what the code does will never notice a line that tells the reader something else.

The new arm is a negative and a positive

check("unverifiable line says it is recorded, not retried",
      len(line) == 1 and "retried next run" not in line[0]
      and "not retried automatically" in line[0])

Exactly one such line, the old phrase gone, the new phrase present — so it fails both on a revert and on a well-meaning rewording to something equally untrue. A single positive assertion would have passed on "will be retried eventually".

The reasoning is the part to keep

"Promising a retry that never comes is how a daily report teaches its readers to skim: they wait for a resolution, stop seeing the line, and cannot tell 'resolved' from 'remembered'."

That is the failure mode of every recurring report, stated in one sentence — and it is why naming the way back in (remove it from <state>, or add a token and re-run) is part of the fix rather than a nicety: without it the reader's only options are waiting or ignoring.

Unchanged from my earlier verdict

The registry change is still +2 classes, none removed, no existing value altered (the 258-line diffstat is re-serialisation), and the two exclusions are still evidence-shaped with dated tag checks. Nothing in this delta touches either.

Nothing to ask.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 19f2818: CHANGES

The new test is a good arm, but both asks from my c4d3f19 review are still open at this head. On git merge-tree origin/main 19f2818 (rc 0):

grep -c 2026-11-17 registry/significant.json   4      grep -c 2026-11-17 docs/arch-gaps.md   2      (today, and both commits: 2026-11-15)
tools/census_watch.py:982   "recorded, not retried automatically (remove it from {DEFAULT_STATE}, or add a token and re-run, to see it again)"
  1. The check date is still two days in the future, six times across the two files. 2026-11-17 → 2026-11-15.
  2. "or add a token and re-run" is still false. main() passes previously_seen = set(state["seen"]) to sweep_newest, which returns only ids "not in seen" (:376, :395). A recorded model is never fetched again, token or not. The new arm 7b checks that the line no longer says "retried next run" and does say "not retried automatically". It does not check the token clause, so it passes with the false promise still in the line. Either drop the clause, which leaves the state-file removal as the one true remedy, or add and "add a token" not in line[0] to 7b once it is gone. If a token is meant to bring gated rows back, sweep_newest would have to keep seen[mid] == UNVERIFIABLE ids when hf_token() is set, with an arm for that.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 19f2818: CHANGES

The two commits since my last review are good and I verified them by mutation. They are also about something else: the item I raised is untouched at this head, so this stays CHANGES on that one line.

Still open from my last review

registry/significant.json   "checked": "2026-11-17"   (MegatronGPTMoeForCausalLM, RusMindAIForCausalLM)
docs/arch-gaps.md:1233      (checked 2026-11-17)
docs/arch-gaps.md:1249      `custom_code` (checked 2026-11-17)

Today is 2026-11-15. watch: "report" is the predicate that silences a class (tools/census.py:124), so after this merges the watcher stops reporting these two and the only record of why is a claim that a human checked them two days from now. The fix is the string 2026-11-15 in four places; nothing else in my review asked for a change.

The new work, verified

c4d3f19c4 + 19f281885 replace the UNVERIFIABLE line's "retried next run" with "recorded, not retried automatically (remove it from state, or add a token and re-run, to see it again)", and the comment states the reason precisely: remember() puts the model in state["seen"] exactly like every other group, so the row is reported once and then goes quiet — the old wording promised a retry that never happens, and "goes quiet" is the one signal a daily report cannot distinguish from "resolved".

unmodified:                                             all checks passed
mutant ("retried next run" put back):  FAIL unverifiable line says it is recorded, not retried

The arm is the interesting half, and the comment says why it had to be added: the existing arm pinned the behaviour (remembered, not re-reported), which was already correct — so pinning behaviour alone could never catch a message that contradicts it. Pinning the line is what closes it. That is the same distinction I have been asking for elsewhere today, arrived at from the other end: an assertion on what the user is told, not only on what the code does.

Named way back in: "remove it from <state>, or add a token and re-run" — a message that says a thing is stuck also has to say how to unstick it, and this one does.

So

One line, four places, and the PR is done. I would rather say that plainly than hold the rest of it: the registry addition, the declined accounting, the top_unmapped exclusion, and now the UNVERIFIABLE wording are all right, and my last review confirmed the HF evidence for both classes live.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant