Skip to content

census: Skylar2ForCausalLM is reviewed, not an alias (KDA + GQA hybrid with gated norms and residual mixers) - #2368

Merged
bong-water-water-bong merged 1 commit into
mainfrom
cl-flm/2367-skylar2
Nov 13, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
cl-flm/2367-skylar2

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Closes #2367.

That run has one !! UNCOVERED line, Skylar2ForCausalLM (Skyl4r-Ai/Skylar2-1B-Instruct). I reviewed it from the repo's own config.json and its model.safetensors header (tensor names and shapes, read with a range request), not from the class name.

What it is: a from-scratch 1B model. It loads in transformers only after import skylar from its own pip package; the repo has no auto_map and no modelling code. It has 36 layers in a 3:1 hybrid:

  • 27 KDA linear-attention layers, with A_log, dt_bias, per-q/k/v short convs and low-rank f_a/f_b, g_a/g_b gates;
  • 9 GQA layers (12 heads / 4 KV, qk_norm), each with a per-head output gate W_gate.

Every norm is a gated RMSNorm with a rank-16 gate_up/gate_down. Every sublayer has an attention-residual mixer (res_attn/res_ffn, plus res_final). The FFN uses a situ_glu activation.

Why it is not an alias:

No pinned graph has these tensors.

What changes:

  • registry/significant.json: Skylar2ForCausalLM, watch: report.
  • docs/arch-gaps.md: a dated section for the review.
  • tests/census_watch_test.py: three checks. The run's upload exits 0, is listed . REVIEWED rather than UNCOVERED, and the class is documented. Removing the entry fails all three.

Not changed: the run's two REMOTE-CODE lines are informational, and its UNVERIFIABLE lines are retried by the watcher.

Ran: python3 tests/census_watch_test.py (all pass) and python3 tools/registry_build.py --check-gaps ("none of the 41 reviewed significant classes is mapped without a reason").

🤖 Generated with Claude Code

The one UNCOVERED line of the 2026-11-13 run (Skyl4r-Ai/Skylar2-1B-Instruct). Read from
its config.json and safetensors header: a from-scratch KDA + GQA hybrid in a 3:1 pattern.
Its KDA layers have low-rank f/g gates and per-q/k/v short convs, and its GQA layers a
per-head output gate. Every norm is a gated RMSNorm (rank 16), every sublayer has an
attention-residual mixer, and the FFN uses situ_glu. No pinned graph has those tensors.
It is neither KimiLinear nor llama.cpp #327's Kimi K3 (MLA, full-rank gate, latent MoE).

significant.json records it as report-only, and docs/arch-gaps.md carries the dated review.
census_watch_test checks that the run's upload exits 0 and is listed REVIEWED; dropping the
entry fails all three checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

ready for review at 3c8f5a8

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

2367 - Partially compliant

Compliant requirements:

  • The Skylar2ForCausalLM class is recorded as report-only in registry/significant.json
  • A dated section for the review is added to docs/arch-gaps.md
  • The test case in tests/census_watch_test.py confirms the class is handled correctly

Non-compliant requirements:

  • The architecture class is not mapped in registry/architectures.json (this is intentional as per the PR description: "not an alias")
  • The family is not served on the NPU route (this is also intentional, as the PR states it is not an alias and no pinned graph has the tensors)

Requires further human verification:

  • Confirmation that the model is indeed not served on the NPU route and that the decision not to map it in registry/architectures.json is correct
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Missing Architecture Mapping

The Skylar2ForCausalLM class is added to registry/significant.json as a report-only entry, but it is not mapped in registry/architectures.json. While the PR description indicates this is intentional because it's not an alias, it's important to confirm that this approach aligns with the overall architecture mapping strategy and that no runtime issues will arise from not having an entry in registry/architectures.json.

"Skylar2ForCausalLM": {
 "model_type": "skylar2",
 "family": "Skylar 2 (first seen: Skyl4r-Ai/Skylar2-1B-Instruct, the 2026-11-13 census), a from-scratch 1B hybrid that loads through its own `skylar` pip package (no auto_map, no modelling code in the repo). 36 layers, hidden 1536, vocab 64000: 27 KDA linear-attention layers (A_log, dt_bias, per-q/k/v short conv, low-rank f_a/f_b and g_a/g_b gates) and 9 GQA layers (12 heads / 4 KV, qk_norm, a per-head output gate W_gate) in a 3:1 pattern; every norm is a gated RMSNorm with rank-16 gate_up/gate_down, every sublayer has an attention-residual mixer (res_attn/res_ffn query + key_weight), and the FFN uses a situ_glu activation. Not an alias of KimiLinearForCausalLM (different KDA gate, norms and residual mixers) nor of Kimi K3 in llama.cpp #327, which shares situ and attention residuals but uses MLA attention, a full-rank KDA gate and a latent MoE. No pinned graph has these tensors.",
 "watch": "report"
}

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 3c8f5a8 — the third census entry of this kind I have reviewed, and the strongest of the three because of how it looked.

The method is stated in the sentence that matters

"I reviewed it from the repo's own config.json and its model.safetensors header (tensor names and shapes, read with a range request), not from the class name."

A range request is the detail that makes this reproducible without a 4 GB download: the safetensors header lists every tensor's name, dtype and shape, so a 1B model's architecture is answerable from the first few kilobytes. That is #1828's cheap-gate logic (decide before you download) applied to model identification, and it is the right instrument for a census whose subject is "is this an alias of something we serve?".

And the phrase "not from the class name" is now the third time: #2281 (Bagel/Oleander), #2229 (three vision classes), this. It is becoming the house rule, and the OKF bundle would be the right home for it if it is not there already — the class name is a claim about the checkpoint, and the header is the checkpoint.

Two asks

  1. say what it is, in the terms the registry uses — "a from-scratch 1B mode[l]" is the start of the answer; the registry's rows are keyed on model_type and family, so give those (model_type, hidden/heads/layers, tokenizer family) and who would be expected to serve it. That is what makes "not an alias" checkable against a future arrival rather than a note in a document;
  2. state the closure — Closes #2367, and that run had one !! UNCOVERED line, so this PR should be a full closure: run the watcher and quote 0 uncovered. That is the ask I made on registry: Bagel and Oleander reviewed as not aliases (census 2026-11-12, engine#2280) #2281 and it is the difference between "one entry added" and "the census is clean".

Why this class keeps being worth a verdict

Because the alternative is a !! UNCOVERED warning that a lane reads as "work to do" when it is often "a reviewed decision nobody wrote down" — and the cost of leaving it is that the next census run flags the same family again, and the run after that, until someone treats the warning as noise and stops reading it. An unrecorded decision decays into a false positive.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY — I checked every number in the entry against the public repo, and they are all right.

Fetched Skyl4r-Ai/Skylar2-1B-Instruct's own config.json and file list rather than taking the entry's word:

the entry says the repo says
model_type skylar2, Skylar2ForCausalLM, 1B model_type = skylar2, architectures = ['Skylar2ForCausalLM']
36 layers n_layers = 36, layer_types has 36 entries
3:1 hybrid: 27 KDA + 9 GQA Counter({'kda': 27, 'attention': 9}) — exact
hidden 1536 d_model = 1536
GQA 12 heads / 4 KV n_heads = 12, n_kv_heads = 4
per-head output gate attn_out_gate = perhead
attention-residual mixer attn_res = True, attn_res_mode = block
situ_glu, β₁ 4, β₂ 25 situ_beta1 = 4.0, situ_beta2 = 25.0
vocab 64000 vocab_size = 64000
no auto_map, no modelling code — needs import skylar no auto_map key at all, and the repo's 12 files contain zero .py

That last row is the entry's load-bearing claim (it is what makes this "reviewed, not an alias"), and it holds on the strongest evidence available: the repo ships weights and config only, so any loader that opens it without the pip package gets ArchitectureNotSupported, not a differently-behaving model.

Two things about the entry as documentation:

  • It says what it read, not just what it concluded — "read from the repo's own config.json and the safetensors header, not from the class name" — and then gives the architecture in enough detail (KDA gate tensors A_log/dt_bias, per-q/k/v convs, low-rank f_a/f_b, res_attn/res_ffn, res_final, rank-16 gated norms) that the next person can tell whether their model is the same thing.
  • The negative comparisons are the useful part for this census: "not an alias of KimiLinearForCausalLM (different KDA gate, no gated norms, no residual mixers)", "nor Kimi K3, which llama.cpp census-watch: non-Transformers checkpoints are reported, not alerts (engine#296) #327 adds". A census entry that only says what a model is leaves the alias question to the reader; this one answers it.

Nothing here needed changing; this is the shape I would want every UNCOVERED line to end up in.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 3c8f5a8

The three checks are the right three, and removing the registry entry fails all of them -- I checked rather than assumed.

$ <entry deleted from registry/significant.json>
python3 tests/census_watch_test.py  ->  rc 1, 3 failing checks
$ <entry restored>
all checks passed

and the second is the one that makes it an alert test rather than a string test: rc == 0 and res["reviewed_report"] == [_SK] and res["significant"] == [] -- exit 0, listed as reviewed, and nothing in the alert bucket. A census that started alerting on this class would fail it even while still printing REVIEWED, which is what a string-only check would miss.

The registry shape is consistent with its own conventions, which I verified against the other 40 entries: model_type: "skylar2", watch: "report", and no other entry claims model_type == "skylar2" (27 of the 41 use report, 14 omit watch). So the value is the established one and the key is unique -- the two things that would make a report entry wrong.

One limit on this verdict, stated because it is the substance of the PR: the conclusion "not an alias" rests on tensor-level differences (27 KDA layers with A_log/dt_bias/per-qkv short convs, gated rank-16 RMSNorms, attention-residual mixers, per-head W_gate versus Kimi Linear's different KDA gate with no gated norms and no residual mixers, and Kimi K3's MLA/full-rank-gate/latent MoE). I cannot re-derive that here -- it needs the model's config.json and safetensors header, which the body says it was read from. What I can verify is the guard around it: the entry is
"1 model", so a second package presenting the same model_type with a different structure would still resolve to this entry and be silently REVIEWED.

That is the one thing I would add, and it is a line rather than a change: the census's report bucket is a decision, and its cost is that the class stops appearing as UNCOVERED for every future model sharing skylar2. If the watcher records the model count it saw (it already prints 1 model), a variant that changes the count is the natural trigger to re-read the class -- and that number is already in the run's own line.

Ran: tests/census_watch_test.py (pass, and fails without the entry); checked registry/significant.json for a duplicate skylar2 claim and for the surrounding watch convention.

@bong-water-water-bong
bong-water-water-bong merged commit d194880 into main Nov 13, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the cl-flm/2367-skylar2 branch November 13, 2026 12:59
bong-water-water-bong added a commit that referenced this pull request Nov 14, 2026
…of the merge-base section (#2390)

Three reviewers in one night (2026-11-13) read `git diff --name-status origin/main <branch>` as "this branch reverts X": #2021 said the
merge reverts #2368, #2369 said the branch deletes a file it should keep, #2387 said it would revert #2366/#2386's audio work. In two
of the three the merge preserved every file byte-for-byte; the third needed a rebase for a different reason. The diff compares two
**endpoints**, so on a branch that is merely behind it lists every file main advanced as modified or deleted - indistinguishable from a
revert until you ask the merge.

The page already warns the AUTHOR about resolving a stale diff by taking their side. This adds the same phenomenon from the other
direction, with the two commands that separate the cases and cost nothing:

    git diff --name-status origin/main...<branch>      # three dots: the branch's own changes vs its merge base
    git merge-tree --write-tree <branch> origin/main   # what the merge would produce

and the one that identifies a real revert: `git log --oneline origin/main..<branch> -- <file>` empty means the branch never touched it.
Rebasing is still worth doing; *hold* and *stale* are different verdicts.

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

census: new HF model class uncovered by the registry

1 participant