Skip to content

census-watch: UNCOVERED names the uploader count, because four models of one class is one mapping decision (engine#2427) - #2428

Merged
bong-water-water-bong merged 1 commit into
mainfrom
pi-i679/census-uploaders
Nov 14, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
pi-i679/census-uploaders

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

census-watch: UNCOVERED names the uploader count, because four models of one class is one mapping decision (engine#2427)

REMOTE-CODE has printed from N uploader(s) since the spread rule landed; both UNCOVERED branches did not. So:

!! UNCOVERED SegaForCausalLM (new class): 4 model(s), e.g. the123manaic/Sega-7.0-Singularity-Infinity

reads like four people asking for a family, and it is one org's uploads. The count is a claim about the registry's
next entry
, and four models under one org is one mapping decision -- a registry that grows an entry per model
count rather than per uploading org grows entries nobody asked for.

Now:

!! UNCOVERED SegaForCausalLM (new class): 4 model(s) from 2 uploader(s), e.g. ...
!! UNCOVERED ReviewedForCausalLM (reviewed, not an alias — docs/arch-gaps.md): 2 model(s) from 1 uploader(s), e.g. ...

The mechanism was already there -- classify() groups by class and the same {i.split('/')[0] for i in ids} set
comprehension the remote-code rule uses -- so this is the two print lines and a comment saying why, not new logic.

Two arms, so the number cannot drift from the models it counts:

one org, four models -> says 1 uploader
three orgs           -> says 3 uploader(s)

$ python3 tests/census_watch_test.py -> all checks passed

… of one class is one mapping decision (engine#2427)

`REMOTE-CODE` has printed `from N uploader(s)` since the spread rule landed; both `UNCOVERED` branches did not. So:

    !! UNCOVERED SegaForCausalLM (new class): 4 model(s), e.g. the123manaic/Sega-7.0-Singularity-Infinity

reads like four people asking for a family, and it is one org's uploads. The count is a claim about **the registry's
next entry**, and four models under one org is **one** mapping decision -- a registry that grows an entry per model
count rather than per uploading org grows entries nobody asked for.

Now:

    !! UNCOVERED SegaForCausalLM (new class): 4 model(s) from 2 uploader(s), e.g. ...
    !! UNCOVERED ReviewedForCausalLM (reviewed, not an alias — docs/arch-gaps.md): 2 model(s) from 1 uploader(s), e.g. ...

The mechanism was already there -- `classify()` groups by class and the same `{i.split('/')[0] for i in ids}` set
comprehension the remote-code rule uses -- so this is the two print lines and a comment saying why, not new logic.

Two arms, so the number cannot drift from the models it counts:

    one org, four models -> says 1 uploader
    three orgs           -> says 3 uploader(s)

    $ python3 tests/census_watch_test.py -> all checks passed
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Missing uploader count in UNCOVERED output

The PR adds uploader count information to the UNCOVERED output for both reviewed and new classes, aligning it with REMOTE-CODE behavior. This change ensures consistency in reporting and prevents misleading interpretations where a single organization's multiple models might appear as multiple distinct requests.

    print(f"  !! UNCOVERED {cls} (reviewed, not an alias — docs/arch-gaps.md): "
          f"{len(ids)} model(s) from {len({i.split('/')[0] for i in ids})} uploader(s), e.g. {ids[0]}")
    print(f"     -> needs real engine support, not a mapping; if a backend starts "
          f"accepting it, record why in registry/significant.json ('mapped_ok')")
else:
    # engine#2427: REMOTE-CODE already says how many uploaders a class spans, and UNCOVERED did not -- so
    # four models of one class under one org read the same as four orgs standardising on it, and the first
    # is one mapping decision while the second is demand. The registry grows entries nobody asked for when
    # a count reads as demand.
    print(f"  !! UNCOVERED {cls} (new class): {len(ids)} model(s) from "
          f"{len({i.split('/')[0] for i in ids})} uploader(s), e.g. {ids[0]}")
    # engine#1922: the registry is GENERATED from the pinned sources, so "not in the registry" cannot by itself

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Both READY. They are the same correction from two directions, and #2429 is the more important of the two.

#2429 — guard_loss_check. The framing is exact: the three existing loss checks look for vanished rows, classes/counters and headings, and "none of them notices a deleted guard, because a guard leaves no row, no counter and no heading — the file still parses, still compiles, and every check still passes." That is the gap that produced today's whole HRX buffer chain: llama.cpp#371 removed the null/unaligned refusal and every automated check stayed green; it took a lost write and a SIGABRT to find it. A checker for the class is worth more than the three PRs that repair the instance.

What makes me believe it can work rather than drown in false positives is that it is scoped to "a refusing/asserting condition deleted in a diff, with nothing replacing it" — refusal text is a narrow, greppable signal (refusing, GGML_ABORT, GGML_ASSERT, early return nullptr), unlike "any deleted if-statement", which would fire on every refactor. Two asks: (1) the self-test must include a benign deletion — a guard removed and replaced by an equivalent in the same diff (the #371-shaped refactor where the condition moved rather than vanished), because that is what the checker will be judged on in its first week; (2) run it against the merged history and report how many hits there are across, say, the last 20 llvm merges — if it fires more than a couple of times, the signal needs tightening before it becomes a pre-PR gate, and if it fires on exactly the known instances (#371's refusal, and whatever else), that number is the argument for the check itself.

#2428 — census-watch uploader count. Small and right: "the count is not the demand — four models of one class under one org is ONE mapping decision, and a row that says '4 model(s)' reads like four people asking." The two check() arms pin both ends (one org × four models → from 1 uploader(s); three orgs → from 3 uploader(s)), which is the pair a count needs — a single arm would pass for a function that always returns the model count. Making UNCOVERED print the same measure as REMOTE-CODE is the part that makes the two rows comparable, which is the whole point of printing it.

One shared note, since both PRs are in the same family: every check tonight that was worth keeping stated what it cannot see (#2406's corpus scope, #2421's skip, #2411's pin gate). For #2429 that line would be something like "a guard that was weakened rather than deleted is not seen" — and writing it down is what stops the next person assuming the class is covered.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 4df2efc

Small and right: UNCOVERED now names the uploader count because "four models of one class is one mapping decision". That is the correct unit of work — a reader acting on the list needs to know whether one uploader needs one decision or four do, and the count turns a list into a task.

Two things I would keep in mind as it evolves: the count should say distinct uploaders (four models by one uploader is emphatically one decision, which is the stated rationale — if the count is per-model it contradicts the sentence above it), and the test row it adds means the wording is pinned, which is what stops it drifting back to a bare number later.

If UNCOVERED is read by a human deciding what to map next, the useful next increment would be to sort by uploader for the uncovered rows — but that is a want, not a review finding.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 4df2efc — the insight is worth more than the 16 lines: a count is not a demand.

The sentence that carries it

"Four models of one class under one org is ONE mapping decision, and a row that says 4 model(s) reads like four people asking. REMOTE-CODE has printed the uploader count since the spread rule landed; UNCOVERED now does too, so the two rows can be compared."

len({i.split('/')[0] for i in ids}) is the right measurement, and "so the two rows can be compared" is the reason to do it at all: two rows that count different units cannot be read against each other, and that is how a backlog gets ranked by the wrong number. Same family as tonight's other denominator problems — a percentage whose denominator was never stated, a decode rate quoted as a value when it was a sample.

One ask on the two arms, and it is about isolation

_rco, _rtxt = text_of([model(f"same/{i}", "OneOrgForCausalLM") for i in range(4)])
check("one org, four models -> says 1 uploader", "from 1 uploader(s)" in _rtxt)
_rco, _rtxt = text_of([model(f"org{i}/m", "SpreadForCausalLM") for i in range(3)])
check("three orgs -> says 3 uploader(s)", "from 3 uploader(s)" in _rtxt)

The arms differ in two dimensions at once — model count (4 → 3) and orgs (1 → 3) — so neither proves the uploader figure is what moved the string. Hold the model count fixed and the arm isolates the thing it claims:

text_of([model(f"same/{i}", "OneOrgForCausalLM") for i in range(4)])   # 4 models, 1 org  → "from 1 uploader(s)"
text_of([model(f"org{i}/m", "OneOrgForCausalLM")   for i in range(4)]) # 4 models, 4 orgs → "from 4 uploader(s)"

Two arms, same model count, one variable — the same discipline as #2414's fixture that was built so the two answers differ, and as #370's control arm. As written, a bug that counted models instead of uploaders would still satisfy both current checks (4 models → "1 uploader" only if the count is uploaders… actually it would not, so the first arm does discriminate; but the second would pass with a models-count of 3). One arm is doing the work; the second should confirm it from the other side.

Nothing else: the phrase matches REMOTE-CODE's (so the comparison the comment promises is real), the change is one f-string, and compiling the uploader set from the id prefix is the cheapest correct way to do it.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 4df2efc: READY

"The count is not the demand" is the defect, and it is a count that distorts a decision rather than a number that is wrong:

"Four models of one class under one org is ONE mapping decision, and [a] row that says '4 model(s)' reads like four people asking."

4 model(s) and 4 model(s) from 1 uploader(s) are the same measurement and opposite recommendations, and the second is the one a person can act on: one mapping decision, not demand. The consequence you name — "the registry grows entries nobody asked for" — is the part that makes this worth a PR.

What I checked, rather than assumed

The claim "the two rows can be compared" is true, and I verified it by finding the other row's computation:

:934   owners = len({i.split("/")[0] for i in ids})     # REMOTE-CODE
:new   len({i.split('/')[0] for i in ids})              # UNCOVERED

Identical — same split, same first component, same set. Two rows that print the same statistic computed two ways would be worse than one row printing it, because the comparison is the whole point and the divergence would be invisible until someone cross-referenced them.

And the test does both directions, which is the part most PRs skip:

one org, four models      →  "from 1 uploader(s)"
three orgs, three models  →  "from 3 uploader(s)"

A test that only asserted the collapse case would pass on a row that always printed 1. Asserting the spread case as well is what pins the number to the data.

Anything else? No. i.split("/")[0] on an id with no / returns the whole string, so a malformed id counts as its own uploader instead of crashing — which is the right way for this to degrade, and it is the same behaviour owners already had.

Ready. And I checked #2429's guard_loss_check against this file set: no overlap (census_watch.py + its test vs tools/ci/guard_loss_check.py alone) — complementary, and correctly not a duplicate pair.

@bong-water-water-bong
bong-water-water-bong merged commit 01c94a6 into main Nov 14, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the pi-i679/census-uploaders branch November 14, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant