Skip to content

census: new HF model class uncovered by the registry #2171

Closed

Description

@github-actions

Census coverage breach (2026-11-11)

The daily new-model watcher found Hugging Face models whose architecture class is not
mapped by the engine registry (registry/architectures.json), a FastFlowLM
repo publishing a model family this engine does not serve, a Q4NX (NPU) pack for
a family only a non-NPU backend serves (engine #588), or a supporting upstream pinned
in registry/support_pins.json that moved or a new d3 weight repo:

  . REMOTE-CODE OpensciForCausalLM: 1 model(s) from 1 uploader(s), e.g. a-kemeklis/open-sci-ref-v0.01-1.7b-nemotron-hq-300B-4096-cpt-aoa10-3B
  ? UNVERIFIABLE Jar-ai/Jar-LM-tok-01 (no config / no architecture) — gated repos need a token; retried next run
  ? UNVERIFIABLE Neural-Labs/tiny-llm (no config / no architecture) — gated repos need a token; retried next run
  ? UNVERIFIABLE SadikNawazAI/sadik-tiny-storyteller (no config / no architecture) — gated repos need a token; retried next run
  ? UNVERIFIABLE Voltline/vimeml-tiny-zh-v1 (no config / no architecture) — gated repos need a token; retried next run
  ? UNVERIFIABLE ZETIC-ai/LFM3-1.7B-IQ9 (no config / no architecture) — gated repos need a token; retried next run
  ? UNVERIFIABLE aliasrobotics/alias3-web (no config / no architecture) — gated repos need a token; retried next run
  ? UNVERIFIABLE hariomlohardev/shakespeare-gpt (no config / no architecture) — gated repos need a token; retried next run
  !! NEW FLM FastFlowLM/MiniCPM-V-4.7-1B-NPU2  (2026-11-08)  q4nx 2 file(s) 2.05 GiB [model.q4nx, vision_weight.q4nx]  license apache-2.0  arch MiniCPMV4_7ForConditionalGeneration  registry: none
  !! NPU FLM FastFlowLM/Hy-Memory-30B-A3B-NPU2  (2026-11-05)  q4nx 1 file(s) 16.64 GiB [model.q4nx]  license apache-2.0  arch HYV3ForCausalLM  registry HYV3ForCausalLM -> hy_v3  npu route: no
  !! NEW FLM FastFlowLM/MiniCPM-OCR-1.0-NPU2  (2026-11-04)  q4nx 2 file(s) 1.86 GiB [model.q4nx, vision_weight.q4nx]  license apache-2.0  base openbmb/MiniCPM-V-4.6  arch MiniCPMOcrForConditionalGeneration  registry: none
  !! NEW FLM FastFlowLM/MiniCPM-V-4.7-35B-A3B-NPU2  (2026-10-30)  q4nx 2 file(s) 20.66 GiB [model.q4nx, vision_weight.q4nx]  license apache-2.0  base openbmb/MiniCPM-V-4.7-35B-A3B  arch MiniCPMV4_7ForConditionalGeneration  registry: none
  !! NPU FLM FastFlowLM/Nemotron-3.5-Bolt-E4B-DFlash-NPU2  (2026-10-30)  q4nx 1 file(s) 0.25 GiB [dflash_weight.q4nx]  license other  base nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash  arch DFlashDraftModel  registry DFlashDraftModel -> dflash  npu route: no
  !! NPU FLM FastFlowLM/Nemotron-3.5-Bolt-E4B-NPU2  (2026-10-26)  q4nx 3 file(s) 8.43 GiB [audio_weight.q4nx, model.q4nx, vision_weight.q4nx]  license other  base nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16  arch Gemma4ForConditionalGeneration  registry Gemma4ForConditionalGeneration -> gemma4  npu route: no
  !! NPU FLM FastFlowLM/Hy-MT2-1.8B-NPU2  (2026-09-02)  q4nx 1 file(s) 1.40 GiB [model.q4nx]  license apache-2.0  arch HunYuanDenseV1ForCausalLM  registry HunYuanDenseV1ForCausalLM -> hunyuan-dense  npu route: no
  !! NPU FLM FastFlowLM/Gemma4-12B-IT-NPU2  (2026-09-01)  q4nx 2 file(s) 8.68 GiB [image_audio_weights.q4nx, model.q4nx]  license apache-2.0  base google/gemma-4-E2B  arch Gemma4UnifiedForConditionalGeneration  registry Gemma4UnifiedForConditionalGeneration -> gemma4  npu route: no
  !! NEW FLM FastFlowLM/smolvla-NPU2  (2026-08-11)  no q4nx file  arch none (no config in the repo)  registry: none
  !! NPU FLM FastFlowLM/medgemma-1.5-4b-it-NPU2  (2026-08-04)  q4nx 2 file(s) 4.42 GiB [model.q4nx, vision_weight.q4nx]  license other  arch gemma3_text  registry Gemma3ForCausalLM -> gemma3  npu route: no
  !! NPU FLM FastFlowLM/LFM2-2.6B-NPU2  (2026-08-04)  q4nx 1 file(s) 1.75 GiB [model.q4nx]  arch Lfm2ForCausalLM  registry Lfm2ForCausalLM -> lfm2  npu route: no
  !! NPU FLM FastFlowLM/LFM2-1.2B-NPU2  (2026-08-04)  q4nx 1 file(s) 0.93 GiB [model.q4nx]  license other  base LiquidAI/LFM2-1.2B  arch Lfm2ForCausalLM  registry Lfm2ForCausalLM -> lfm2  npu route: no
  !! NPU FLM FastFlowLM/GPT-OSS-Safeguard-20b-NPU2  (2026-08-04)  q4nx 1 file(s) 13.45 GiB [model.q4nx]  license apache-2.0  base openai/gpt-oss-safeguard-20b  arch GptOssForCausalLM  registry GptOssForCausalLM -> gpt-oss  npu route: no
  !! NPU FLM FastFlowLM/GPT-OSS-20B-NPU2  (2026-08-04)  q4nx 1 file(s) 13.45 GiB [model.q4nx]  license apache-2.0  base openai/gpt-oss-20b  arch GptOssForCausalLM  registry GptOssForCausalLM -> gpt-oss  npu route: no
  !! NPU FLM FastFlowLM/Gemma3-4B-NPU2  (2026-08-04)  q4nx 2 file(s) 4.42 GiB [model.q4nx, vision_weight.q4nx]  license gemma  base google/gemma-3-4b-it  arch gemma3_text  registry Gemma3ForCausalLM -> gemma3  npu route: no
  !! NPU FLM FastFlowLM/Gemma3-1B-NPU2  (2026-08-04)  q4nx 1 file(s) 1.14 GiB [model.q4nx]  license gemma  base google/gemma-3-1b-it  arch Gemma3ForCausalLM  registry Gemma3ForCausalLM -> gemma3  npu route: no
  !! NPU FLM FastFlowLM/Gemma4-E4B-IT-NPU2  (2026-08-04)  q4nx 3 file(s) 8.43 GiB [audio_weight.q4nx, model.q4nx, vision_weight.q4nx]  license apache-2.0  base google/gemma-4-E4B-it  arch Gemma4ForConditionalGeneration  registry Gemma4ForConditionalGeneration -> gemma4  npu route: no
  !! NPU FLM FastFlowLM/Gemma4-E2B-IT-NPU2  (2026-08-04)  q4nx 3 file(s) 5.55 GiB [audio_weight.q4nx, model.q4nx, vision_weight.q4nx]  license apache-2.0  arch Gemma4ForConditionalGeneration  registry Gemma4ForConditionalGeneration -> gemma4  npu route: no
  !! NPU FLM FastFlowLM/Qwen3.6-35B-A3B-NPU2  (2026-07-10)  q4nx 2 file(s) 22.58 GiB [model.q4nx, vision_weight.q4nx]  license apache-2.0  base Qwen/Qwen3.6-35B-A3B  arch Qwen3_5MoeForConditionalGeneration  registry Qwen3_5MoeForConditionalGeneration -> qwen35moe  npu route: no
  !! NPU FLM FastFlowLM/medgemma-4b-it-NPU2  (2026-05-25)  q4nx 2 file(s) 4.42 GiB [model.q4nx, vision_weight.q4nx]  license other  base google/medgemma-4b-it  arch gemma3_text  registry Gemma3ForCausalLM -> gemma3  npu route: no
  !! NPU FLM FastFlowLM/Translategemma-4B-Instruct-NPU2  (2026-02-18)  q4nx 2 file(s) 4.42 GiB [model.q4nx, vision_weight.q4nx]  license gemma  arch gemma3_text  registry Gemma3ForCausalLM -> gemma3  npu route: no
  !! NPU FLM FastFlowLM/Qwen2.5-VL-3B-Instruct-NPU2  (2026-02-11)  q4nx 2 file(s) 3.74 GiB [model.q4nx, vision_weights.q4nx]  base Qwen/Qwen2.5-VL-3B-Instruct  arch Qwen2_5_VLForConditionalGeneration  registry Qwen2_5_VLForConditionalGeneration -> qwen2vl  npu route: no
  !! NPU FLM FastFlowLM/LFM2.5-1.2B-Thinking-NPU2  (2026-01-20)  q4nx 1 file(s) 0.93 GiB [model.q4nx]  license other  base LiquidAI/LFM2.5-1.2B-Base  arch Lfm2ForCausalLM  registry Lfm2ForCausalLM -> lfm2  npu route: no
  !! NPU FLM FastFlowLM/Phi4-mini-Instruct-NPU2  (2026-01-16)  q4nx 1 file(s) 3.38 GiB [model.q4nx]  license mit  arch Phi3ForCausalLM  registry Phi3ForCausalLM -> phi3  npu route: no
  !! NPU FLM FastFlowLM/LFM2-2.6B-Transcript-NPU2  (2026-01-16)  q4nx 1 file(s) 1.75 GiB [model.q4nx]  license other  base LiquidAI/LFM2-2.6B  arch Lfm2ForCausalLM  registry Lfm2ForCausalLM -> lfm2  npu route: no
  !! NPU FLM FastFlowLM/LFM2.5-1.2B-NPU2  (2026-01-16)  q4nx 1 file(s) 0.93 GiB [model.q4nx]  license other  base LiquidAI/LFM2.5-1.2B-Base  arch Lfm2ForCausalLM  registry Lfm2ForCausalLM -> lfm2  npu route: no
  !! NPU FLM FastFlowLM/LFM2-1.2B-Tool-NPU2  (2026-01-11)  q4nx 1 file(s) 0.93 GiB [model.q4nx]  arch Lfm2ForCausalLM  registry Lfm2ForCausalLM -> lfm2  npu route: no
  !! NEW FLM FastFlowLM/Whisper-V3-Turbo-NPU2  (2025-10-30)  q4nx 1 file(s) 0.61 GiB [model.q4nx]  license mit  base openai/whisper-large-v3-turbo  arch WhisperForConditionalGeneration  registry: none
  !! NPU FLM FastFlowLM/Embedding-Gemma-300M-NPU2  (2025-10-24)  q4nx 1 file(s) 0.57 GiB [model.q4nx]  license gemma  base google/embeddinggemma-300m  arch Gemma3TextModel  registry Gemma3TextModel -> gemma-embedding  npu route: no
  !! NPU FLM FastFlowLM/Gemma3-270M-NPU2  (2025-10-20)  q4nx 1 file(s) 0.47 GiB [model.q4nx]  license gemma  base google/gemma-3-270m  arch Gemma3ForCausalLM  registry Gemma3ForCausalLM -> gemma3  npu route: no
  !! NPU FLM FastFlowLM/Gemma3-4B-Text-NPU2  (2025-08-22)  q4nx 1 file(s) 3.51 GiB [model.q4nx]  license gemma  base google/gemma-3-4b-it-qat-q4_0-unquantized  arch gemma3_text  registry Gemma3ForCausalLM -> gemma3  npu route: no
  + FLM RELEASE v1.0.12  (2026-11-05)  kernel-format: q4nx  assets 13

Fix: map the class in registry/architectures.json when a pinned backend's code really
accepts the architecture, or serve the family on the NPU route when the pack asks for
it, or record why it is not an alias in registry/significant.json
and docs/arch-gaps.md (a class reviewed as significant needs real engine support, not a
mapping); for a moved supporting upstream, read the change and bump its commit and
reviewed in registry/support_pins.json, or add the new d3 repo to its list.
Re-run python3 tools/census_watch.py to confirm.

Run: https://github.com/1bit-MONSTER/engine/actions/runs/43016580831

Activity

  1. bong-water-water-bong commented on Nov 11, 2026

    @bong-water-water-bong
    Collaborator

    Fix opened (#2207), and one disposition recorded so nobody "fixes" it the wrong way: the second spelling is not a normalization bug

    #2207 adds OpensciForCausalLM to registry/significant.json with watch: "report" — the documented mechanism for "a human reviewed the class and deliberately did not map it" — after reading the model's own config rather than the class name:

    architectures: ["OpensciForCausalLM"]      model_type: "opensci"
    auto_map: AutoModelForCausalLM -> modeling_opensci.OpensciForCausalLM
    

    The auto_map is the reason it cannot be mapped: the repo ships its own modelling code, so no pinned backend accepts the architecture without vendoring or upstreaming it. Verified with tools/registry_build.py --check-gaps → "none of the 38 reviewed significant classes is mapped without a reason", rc=0.

    The disposition that matters, because the obvious next move is wrong: registry/census.json also carries OpenSciForCausalLM — the same letters, different capitalisation — with 7 models against 121. The tempting fix is to normalise the class string, and I checked before proposing it:

    tools/census_watch.py:28-29 — "there is no strip_arch either — architectures.json is keyed by the HF architecture class, which is the key."

    Exact-key matching is a stated design decision, not an oversight: the HF class string is what a converter has to recognise, so OpenSciForCausalLM and OpensciForCausalLM are two keys and a mapper that accepts one genuinely does not accept the other. Normalising here would hide a real mismatch behind a case-fold — the same shape as the "#1845 check keyed on a file name" family, where the convenient comparison silently changes what is being checked.

    So the residue is a second reviewed entry, not a code change, and it needs one input I do not have: which repos carry the OpenSciForCausalLM spelling. An HF model search for opensci returns only the open-sci/* and a-kemeklis/* family (all custom_code, all one model lineage), and census.json stores per-class counts rather than repo lists — so the 7 could be the same family with an inconsistent card, or a different one. They are two different dispositions (one entry with "same family, capitalisation variant" versus a separate review) and guessing between them is exactly what this census exists to avoid.

    Action for whoever has a census run handy: tools/census_watch.py prints the model ids it grouped per class; paste the 7 and this becomes a two-line entry. Until then the spelling stays a reported gap, which is the correct state for an unreviewed class — and REMOTE_CODE_SPREAD = 3 means a remote-code class below three uploaders is reported without alerting anyway.

  2. bong-water-water-bong commented on Nov 11, 2026

    @bong-water-water-bong
    Collaborator

    Closing: the reported breach is fixed and verified on main by content, not by PR state

    The watcher's finding was that OpensciForCausalLM was an architecture class the registry does not map. That is now a recorded review, merged in #2189:

    registry/significant.json   "OpensciForCausalLM": {model_type: opensci, watch: "report", …}   1 entry
    docs/arch-gaps.md           the dated `opensci` section                                       3 citations
    

    Checked by content rather than by "the PR merged", because a squash merge defeats merge-base --is-ancestor — the entry, the doc section and registry_build.py --check-gaps ("none of the 38 reviewed significant classes is mapped without a reason", rc=0) are all what is actually on main now.

    The disposition, for the record: the class is not mapped because the model's config carries an auto_map pointing at modeling_opensci.py — the repo ships its own modelling code, so no pinned backend can accept it without vendoring or upstreaming that code. watch: "report" makes the census count those models separately and exclude them from top_unmapped, because a recorded decision is not work remaining.

    What will look like a new breach and is not one: the census carries a second spelling, OpenSciForCausalLM (7 models against 121), from a different uploader, whose config was not reviewed. census_watch.py matches architecture classes exactly by design — "architectures.json is keyed by the HF architecture class, which is the key" — so a future run reporting it is the census working, not a regression, and it needs its own review with the repo list that run prints. A case-fold would have hidden a real converter mismatch instead of fixing one, which is why it was left alone.

    Closing. If a later sweep reports the other spelling, that is a new issue with the same shape and the same answer one config-read away.

  3. bong-water-water-bong commented on Nov 11, 2026

    @bong-water-water-bong
    Collaborator

    Three of the four registry: none lines are already answered -- the report predates the fix

    Checked each of this report's actionable lines against the tree, because the classes looked like an open gap:

    class where it is answered now
    MiniCPMV4_7ForConditionalGeneration registry/flm_answers.json — "not served: a vision-language pack whose class the pin does not register; serving it on the NPU route is engine#1788", with its docs/npu.md row
    MiniCPMOcrForConditionalGeneration registry/flm_answers.json — "not served: an OCR/vision pack whose class the pin does not register"
    WhisperForConditionalGeneration registry/flm_answers.json

    All three landed in 853f8c9c, 2026-11-11 18:04Z ("census: an answered FastFlowLM gap stays answered"), i.e. after this report was generated -- so registry: none above is stale text rather than a live gap, and the daily run should now print . NPU FLM ... <answer> for them instead of alerting.

    What is left in this report, by design rather than by defect:

    • OpensciForCausalLM (REMOTE-CODE) -- should keep alerting. It is a repo that needs trust_remote_code, so recording it as "reviewed, not an alias" would suppress the one line in this report that carries a security meaning.
    • The nine UNVERIFIABLE gated repos -- no config is readable without a token; that is a credentials question (a HF_TOKEN for the watcher), not a registry gap.
    • The npu route: no lines (hy_v3, dflash, gemma4, qwen35moe, ...) -- the registry maps these for hrx but the NPU route does not serve them; that is engine#588's scope, and census: new HF model class uncovered by the registry #2171 is not the place to fix it.

    Suggestion: re-run the watcher before the next report so the answered lines drop out; if any of the three still alerts after 853f8c9c, that is a real census_watch.py bug and the row above is the evidence to start from.

    I opened #2229 against this (adding the classes to registry/significant.json) and closed it: that is the model-census record, the pack mechanism is flm_answers.json, and the change broke the test that pins "with no answers the rows still alert".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    census-watchDaily new-model watcher alert

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions