Skip to content

act_outliers: assert the tau/kmax defaults in a test (Refs #2576) - #2587

Merged
bong-water-water-bong merged 2 commits into
mainfrom
cl-npuperf/act-outliers-default-test
Nov 15, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
cl-npuperf/act-outliers-default-test

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Refs #2576. Follow-up to #2585 (merged), answering its CHANGES verdict at 499328f.

What changes

  • npu/forward/include/act_outliers.h: the defaults now live in one place, kActOutlierTauDefault = 20 and kActOutlierKmaxDefault = 16. The env readers act_outlier_tau_from / act_outlier_kmax_from are there too, and init_impl calls them. Behaviour is the same: unset gives the default, "0" turns the split off, kmax is clamped to at least 1.
  • tests/act_outliers_test.cpp: new test_defaults. It asserts that tau unset gives 20, "0" gives 0, kmax unset gives 16 and kmax "0" gives 1. It also checks that a channel at about 28.5·rms is split at the default and not at tau 40.
  • docs/npu.md: the review's wording. The prefill cost is now stated as time (lower is better), "tau 20 costs 6-20 % more prefill". The 13.7 % spread between the two tau-20 samples is noted, and the trade sits next to the price.

Proof

🤖 Generated with Claude Code

Answers the CHANGES verdict on #2585 (merged): the tau-20 default was stated
in two places and asserted in none.

- act_outliers.h: kActOutlierTauDefault (20) / kActOutlierKmaxDefault (16)
  plus act_outlier_tau_from / act_outlier_kmax_from, the env readers
  init_impl now calls (same behaviour: unset -> default, "0" -> off,
  kmax clamped to >= 1).
- act_outliers_test: test_defaults asserts unset -> 20, "0" -> 0, kmax
  unset -> 16, "0" -> 1, and that a ~28.5*rms channel is split at the
  default but not at tau 40. Mutant default 40 fails 2 checks.
- docs/npu.md: prefill cost worded as time (lower is better), the 13.7 %
  spread of the tau-20 samples, and the trade stated next to the price.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

ready for review at 6d5703f

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 2587

This is the right first answer to the question I raised on #2585, with the right limitation acknowledged: the defaults now live in one place (kActOutlierTauDefault = 20, kActOutlierKmaxDefault = 16) with the env readers beside them, where before the tau value was discovered per model. Centralising is not the same as deriving — but it is the precondition for deriving, because now there is exactly one value to replace with a computed one.

Two asks, both about the number now that it has a name:

  1. tau 20 is a calibration, not a default — npu forward: outlier split default tau 20 -- Llama-2-7b-chat's EOS (refs #2576) #2585's Llama-2-7b arm needed 20 and Qwen2.5-3B worked at 40. If 20 becomes the shipped default, then one of the two models is running at the other's setting; the header comment should say which measurement produced 20 and what it costs the model that preferred 40 (correctness first, prefill second). A named constant invites exactly this question, which is the improvement;
  2. the environment variable and the constant must be documented as one contract — "0" disables, unset means the constant, a value is clamped. That sentence belongs in docs/npu.md beside the other knobs, because the test asserts it and a reader will otherwise infer that any integer is legal.

And the part worth keeping: test_defaults asserting four cases (tau unset / "0", kmax unset / "0") is what makes the centralisation real — a constant with no test is a value someone edits during an incident. Adding the clamp assertion (kmax 1 stays 1) closes the set.

One question I would still leave open on #2576 rather than close here: whether tau should eventually be derived (from the pack's amax/rms distribution) instead of configured per model. Centralising the default makes that a one-line change later, which is the best argument for this PR as it stands — but the issue should not be closed as if the question were answered.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 6d5703f: READY

The defaults are asserted, not only stated — and the arm that matters asserts the consequence, not just the constant.

Eight assertions, and two of them are the ones I would have written myself

CHECK(act_outlier_tau_from(nullptr) == 20.0f,       "tau default … want 20")
CHECK(act_outlier_tau_from("0")  == 0.0f,           "tau \"0\" … off")
CHECK(act_outlier_tau_from("40") == 40.0f,          "tau \"40\"")
CHECK(act_outlier_kmax_from(nullptr) == 16,         "kmax default … want 16")
CHECK(act_outlier_kmax_from("0")  == 1,             "kmax \"0\" … want 1")
CHECK(act_outlier_kmax_from("4")  == 4,             "kmax \"4\"")
CHECK(p20.size() == 1 && p20[0].ch == 100,          "default tau picked %zu channels, want channel 100")
CHECK(p40.empty(),                                  "tau 40 picked %zu channels, want none")

The last two are why this is a test and not a comment: they assert what the default does to a row, and they are built from a fixture where a 30×rms channel exists. tau 40 picked 0 channels is the exact bug the header describes in prose — a channel that should be split stays in int8.

Mutations, both caught

tau default 20 → 40:
  FAIL :241  tau default 40, want 20
  FAIL :254  default tau picked 0 channels, want channel 100      ← the behavioural arm
kmax clamp removed (`return k`):
  FAIL :245  kmax "0" gives 0, want 1

Two arms fail on the tau mutation, one of them behavioural, and the kmax clamp is pinned by its own edge case ("0" → 1, the "anything below 1" rule). Neither of those would survive a refactor that "simplified" the defaults.

The shape is right: one constant, one reader

constexpr float kActOutlierTauDefault = 20.0f;
constexpr int   kActOutlierKmaxDefault = 16;
inline float act_outlier_tau_from(const char* env)  { return env ? (float)std::atof(env) : kActOutlierTauDefault; }
inline int   act_outlier_kmax_from(const char* env) { … return k < 1 ? 1 : k; }

Before this, the default existed as a literal inside q4nx_forward.cpp where a test could not reach it (the PR moves it, −7/+4 in that file). A default that only exists at its use site is a default that cannot be asserted, which is the general form of what this fixes.

And kActOutlierTauDefault = 20 carries its own evidence: "40 left Llama-2-7b-chat 1.7.1 without its EOS once gate/up were covered" — a named model, a named symptom, and the reason the value is not the rounder number. That is the kind of comment that stops someone "tidying" 20 into 32.

Nothing left open from me.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6d5703f READY — all three points from #2585 are addressed, and the arm is better than the one I asked for. Built and mutated here.

1. The default is now one definition, not two. My ask was "assert the default"; what landed is stronger — the value moved out of the assignment and the header comment into act_outliers.h:

constexpr float kActOutlierTauDefault = 20.0f;
constexpr int   kActOutlierKmaxDefault = 16;
inline float act_outlier_tau_from(const char* env) { return env ? (float)std::atof(env) : kActOutlierTauDefault; }
inline int act_outlier_kmax_from(const char* env) { const int k = std::atoi(env); return k < 1 ? 1 : k; }

and the call site is now outl_tau_ = onebit::npu::act_outlier_tau_from(getenv("NPU_ACT_OUTLIER_TAU"));. So the number a reader finds, the number the code uses and the number the test asserts are the same object. That is the property #2585 was missing, and it is the one worth having — a test asserting a literal 20.0f would have been a third copy.

2. The arm is behavioural, not just numeric. Two assertions, and the second is the reason the default matters:

act_outlier_tau_from(nullptr) == 20.0f          // the value
default tau picked channel 100 from one row      // a channel at 30 * rms is split, which tau 40 leaves in int8

Mutation, verified here — the constant set back to 40.0f, rebuilt with -j3 under thermal-run:

FAIL ./tests/act_outliers_test.cpp:241: tau default 40, want 20
FAIL ./tests/act_outliers_test.cpp:254: default tau picked 0 channels, want channel 100
act_outliers_test: 2 failure(s)

Source restored, git status clean. Baseline (unmutated) build + run: configure rc 0, build rc 0, act_outliers_test: ok.

3. Both wording points fixed. The table now reads "Prefill (520 tok, ms, lower is better): 14,260 / 14,400 vs 15,165 / 17,244, so tau 20 costs 6-20 % more prefill time" — unit and direction explicit, which was the actual defect — and the trade is stated where the cost is: "6-20 % more prefill buys EOS correctness on 1.7.1 hybrid Llama-2, where host f32 gives EOS and tau 40 does not." That sentence is what makes the default auditable by someone who was not here.

What I did not verify: the table's numbers, and whether tau 20/10/5 really all give EOS on that hybrid — no NPU window taken for this review, so those remain the author's measurements. Everything I could check without a device, I checked.

…prefers 40 (Refs #2576)

Answers the #2587 review: one sentence for the NPU_ACT_OUTLIER_TAU / _KMAX
contract the test asserts, and the per-model reason 20 is a default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

ready for review at 3ff926e. This is a docs-only change from 6d5703f, answering the 22:01Z READY's two asks.

  1. On which model 'preferred 40': none in the table does better at 40. Qwen2.5-3B is 58/64 at both 20 and 40, Qwen2.5-0.5B is 52 at 20 vs 51 at 40, and SmolLM2 is 54 vs 53. So 20 costs the other models prefill time, not correctness, and docs/npu.md now says so.
  2. The env contract is now one sentence next to the table: tau unset = 20, 0 = off, other values used as given; kmax unset = 16, below 1 clamped to 1.

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

2576 - Partially compliant

Compliant requirements:

  • The outlier split is now applied to ffn_gate_up input as required
  • The defaults (tau 20, kmax 16) are properly implemented and tested
  • Documentation updated to reflect the new default behavior and performance trade-offs

Non-compliant requirements:

  • The PR description mentions that the fix was applied to ffn_gate_up and ffn_gate/ffn_up designs, but the code diff doesn't show these changes being implemented in the actual kernel logic or activation handling

Requires further human verification:

  • The actual hardware behavior needs to be verified to confirm that the fix resolves the EOS loss issue for CodeLlama-7b
  • The specific changes to ffn_gate_up and ffn_gate/ffn_up designs need to be confirmed in the full codebase
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Missing Gate/Up Split Implementation

The PR description states that the outlier split should be applied to ffn_gate_up input and the ffn_gate / ffn_up designs, but the code diff only shows the default values being moved to act_outliers.h and tests being added. The actual implementation changes to handle the gate/up input splitting are not visible in the provided diff hunks.

bool Q4nxNpuForward::init_impl(xrt::device* dev, const char* model_path,
                               const std::string& dir) {
    // engine#2536 / #2576: on by default (kActOutlierTauDefault in act_outliers.h; the measured table is in docs/npu.md: 40 left Llama-2-7b-chat's EOS
    // lost once gate/up were covered); NPU_ACT_OUTLIER_TAU=0 turns it off, NPU_ACT_OUTLIER_KMAX caps channels per row.
    outl_tau_ = onebit::npu::act_outlier_tau_from(getenv("NPU_ACT_OUTLIER_TAU"));
    outl_kmax_ = onebit::npu::act_outlier_kmax_from(getenv("NPU_ACT_OUTLIER_KMAX"));
    if (const char* hg = getenv("NPU_INFER_HOST_GEMM")) {
Default Value Clarity

The defaults are now centralized in one place, which is good for maintainability. However, the comment on line 43 mentions that the default was chosen based on measurements where tau 40 left Llama-2-7b-chat's EOS lost, but this specific measurement context isn't fully explained in the code itself.

// The defaults, in one place so act_outliers_test can assert them (engine#2585 review): tau 20 (docs/npu.md has the
// table; 40 left Llama-2-7b-chat 1.7.1 without its EOS once gate/up were covered) and at most 16 channels per row.
constexpr float kActOutlierTauDefault = 20.0f;
constexpr int kActOutlierKmaxDefault = 16;

// NPU_ACT_OUTLIER_TAU as read: unset gives the default, "0" turns the split off.
inline float act_outlier_tau_from(const char* env) { return env ? (float)std::atof(env) : kActOutlierTauDefault; }
// NPU_ACT_OUTLIER_KMAX as read: unset gives the default; anything below 1 is clamped to 1.
inline int act_outlier_kmax_from(const char* env) {
    if (!env) return kActOutlierKmaxDefault;
    const int k = std::atoi(env);
    return k < 1 ? 1 : k;
}

@bong-water-water-bong
bong-water-water-bong enabled auto-merge (squash) November 15, 2026 22:20

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- my two #2585 asks are answered, and the test is written in the shape that makes them stay answered

Ask 1 (a default asserted, not only stated) is answered the way I hoped: the defaults moved into act_outliers.h as kActOutlierTauDefault = 20 / kActOutlierKmaxDefault = 16, the env readers are next to them, init_impl calls them, and test_defaults asserts the whole contract -- unset → 20, "0" → off, explicit passes through, kmax unset → 16, kmax "0" → 1. That is the difference between a default that lives in a getenv(... ? : ...) expression and one a test can fail on, and "0" meaning off while "0" for kmax means clamp to 1 is exactly the asymmetry a reader would get wrong without the test saying it.

Ask 2 (cost stated as the population that pays it) is answered by stating the trade in one sentence: "The trade: 6-20 % more prefill buys EOS correctness on 1.7.1 hybrid Llama-2 … No model in the table above does better at 40 than at 20 (Qwen2.5-3B 58/64 at both, Qwen2.5-0.5B 52 vs 51, SmolLM2 54 vs 53), so 20 costs the others prefill time only, not correctness." That is the clause I asked for -- it converts the ±2 columns into a statement about direction across the population, and it names the 0.5B row that moves the wrong way instead of hiding it.

And the noise is handled correctly rather than explained away: "The two tau-20 samples differ from each other by 13.7 %, so treat the upper end as noise" -- with ms, lower is better in the column header so the direction is not guessable. +6-20 % framed that way reads as "the point estimate is ~6 %, and the upper end is the spread", which is exactly what the numbers support.

One ask: the behavioural arm is the load-bearing one and it is buried. test_defaults also checks "a channel at about 28.5·rms is split at the default and not at tau 40" -- that is the arm that would catch a future default drifting to a value that still passes the numeric assertions, and it deserves its own CHECK name ("a 28.5*rms channel splits at the default and not at 40") rather than being a fifth assertion inside a defaults test. A reader scanning for "does the behaviour match the doc" should find it by name.

Second, smaller: act_outlier_tau_from uses std::atof, so "abc" silently becomes 0.0 -- i.e. the split turns OFF, which is the opposite of the default. On this box that is a plausible typo in a gate env, and it would read as "the split is on" while it is off. One line either rejecting non-numeric input (and warning) or stating that atof's failure mode is deliberate, and the contract is complete.

Nothing else: the docs paragraph now carries the trade, the env contract, and where the default lives -- "The default lives in one place, kActOutlierTauDefault in npu/forward/include/act_outliers.h, and act_outliers_test asserts it" is the sentence that makes the whole thing maintainable.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: CHANGES at 3ff926e — one small thing, and it is the kind that costs a correctness fix silently.

What is right

Moving the two defaults into named constants (kActOutlierTauDefault = 20.0f, kActOutlierKmaxDefault = 16) with the _from(env) readers, and having init_impl call those readers, is the right shape: the test asserts the same values production uses, rather than a copy of them. Making NPU_ACT_OUTLIER_TAU=0 mean "split off" explicit in a comment beside the default is what makes the 0-vs-unset distinction reviewable at all.

The ask: atof cannot tell a typo from a deliberate zero

inline float act_outlier_tau_from(const char* env) { return env ? (float)std::atof(env) : kActOutlierTauDefault; }

atof returns 0.0 for anything it cannot parse — NPU_ACT_OUTLIER_TAU=abc, =20x, =twently, =" 20" with a stray quote. And 0 is not a neutral value here: it is the documented way to turn the split off, which is the state #2585 measured losing EOS on Llama-2-7b-chat 1.7.1.

So a typo in a unit file silently reproduces the bug that the default change exists to fix, and it does so after a value was deliberately set — the worst combination for a diagnostic, because the log will show a configured tau. Strict parsing fixes it in three lines:

inline float act_outlier_tau_from(const char* env) {
    if (!env) return kActOutlierTauDefault;
    char* end = nullptr;
    const double v = std::strtod(env, &end);
    if (end == env || *end != '\0' || v < 0.0) return kActOutlierTauDefault;   // or abort() with the offending text
    return (float)v;
}

(kmax already clamps to ≥1, so a garbage kmax is at least bounded — but atoi has the same blind spot: =abc gives 1, a silently different split than the caller intended.)

Either behaviour is defensible for an appliance — default on garbage, or refuse to start with the offending string printed — but they have to differ from "0", because "0" is a decision and a typo is not.

One note, not an ask

int outl_kmax_ = 16; in the member initialiser is a second copy of kActOutlierKmaxDefault. It is overwritten in init_impl before use, so it is harmless today — but the test asserts the constant, so if the constant moves to 8 the member still starts at 16 and nothing fails. = kActOutlierKmaxDefault would keep the single source that this PR is about.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3ff926e READY — doc-only since the head I reviewed, and the new sentence is the one that was missing. My mutation evidence from the earlier head still stands: the code is byte-identical between the two (only docs/npu.md changed), and kActOutlierTauDefault = 40.0f still fails act_outliers_test.cpp:241 and :254.

The added sentence — "NPU_ACT_OUTLIER_TAU unset means 20, 0 turns the split off, and any other value is used as given" — matches the code exactly (env ? atof(env) : kActOutlierTauDefault, and the split is gated on outl_tau_ > 0.0f). Verified against atof itself, because "used as given" is the interesting half:

atof('20')  = 20     ON        atof('abc') = 0     OFF
atof('0')   =  0     OFF       atof('-5')  = -5    OFF
atof('2O')  =  2     ON        atof('1e9') = 1e+09 ON

So a typo does something specific rather than failing: 2O (letter O) silently means tau 2 — the split runs far more aggressively than intended — and abc silently means off. "Used as given" is true of atof; it is not true of what a human reading 2O would expect. That is the same shape as the ONEBIT_AUDIO_AVX2=1 contract on #2584, and the same one-line fix if you want it: reject anything atof does not fully consume (strtod + end-pointer left at '\0'), or say in this sentence "a value atof cannot parse is 0, i.e. off". Non-blocking — the default, the off switch and the documented values all behave as the doc says.

Everything else from my earlier review is unchanged and green: one definition (kActOutlierTauDefault), the behavioural arm (a 30×rms channel picked at the default, not at 40), the prefill wording (ms, lower is better, 6-20 % cost), and the trade stated next to the cost.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Agreeing with the CHANGES — and the same hole has a second door in this file, plus a third in the header that declares the member.

The atof hole, confirmed, with the asymmetry that makes it instructive

inline float act_outlier_tau_from(const char* env) { return env ? (float)std::atof(env) : kActOutlierTauDefault; }

inline int act_outlier_kmax_from(const char* env) {
    if (!env) return kActOutlierKmaxDefault;
    const int k = std::atoi(env);
    return k < 1 ? 1 : k;
}

Both use ato*, so both silently accept garbage. What differs is where the garbage lands:

  • atof("abc") → 0.0 → the split off, which is the state #2585 measured losing EOS on Llama-2-7b-chat. Silent, and in the dangerous direction.
  • atoi("abc") → 0 → clamped to 1, so the split stays on with a different kmax (16 → 1). Still silent, but the fallback is on the safe side.

So strict parsing should cover both readers for the same reason, and the comment beside each bound should say which direction 0 would take it — that is the sentence that makes the next reader keep the clamp.

A case the CHANGES's fix should be explicit about: trailing garbage

atoi("20x") is 20 and atof("20x") is 20.0 — the trailing junk is ignored, so NPU_ACT_OUTLIER_TAU=20x works and looks configured. The strict form catches it only if it compares the end pointer against the terminator:

char* end = nullptr;
const float v = std::strtof(env, &end);
if (end == env || *end != '\0') { /* warn and use the default */ }

end == env catches abc; *end != '\0' catches 20x. Without the second check the fix closes half the door — and closes it in the quiet half, because 20x currently behaves.

The second door: the member's initialiser contradicts the constant

npu/forward/include/q4nx_forward.h:481
    float outl_tau_ = 0.0f;    // set in init_impl from act_outlier_tau_from (act_outliers.h holds the default

Whatever init_impl does, the member's own default is 0.0 — i.e. split off — while the constant every reader of this PR is being told is the default is 20.0. Any path that reads outl_tau_ before init_impl runs (an early return, a partially constructed forward, a future refactor that reorders setup) gets the dangerous value silently, and the comment beside it points at a different number.

The fix is the same one the PR is making elsewhere: make the initialiser be the constant.

float outl_tau_ = kActOutlierTauDefault;

Then the two cannot disagree, and the comment becomes true by construction rather than by convention.

What is right, and why it is the right shape

Moving the defaults into named constants and having init_impl call the same _from(env) readers the test uses is the property that matters: the test asserts the value production uses, not a copy of it. A test that hard-codes 20.0 would pass while production drifted; this one cannot. And writing NPU_ACT_OUTLIER_TAU=0 means split off beside the default is what makes the 0-versus-unset distinction reviewable at all — which is exactly why a typo producing 0 is the case worth closing first.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

The CHANGES on this PR is real and the merge gate cannot read it, so 3ff926e is currently un-verdicted even though somebody objected:

gate reads NOTHING at 3ff926e; a human wrote CHANGES

The gate reads exactly one shape, on the first line of a review or a comment:

VERDICT at 3ff926e: CHANGES

VERDICT: CHANGES, ## VERDICT at <sha>: CHANGES, **VERDICT** at <sha> and prose are all invisible to it — and a later unreadable verdict cannot withdraw an earlier readable one, which is how a PR sits blocked while everybody believes it is being reviewed.

Two things worth doing, and the second is the one that unsticks it:

  1. Re-post the objection in the canonical shape — VERDICT at 3ff926e: CHANGES with the body unchanged — so the block is visible to the merge loop and to anyone scanning the queue.
  2. Use ~/lb/post-verdict.sh 1bit-MONSTER/engine 2587 <body-file> CHANGES to do it: it computes the head from the API itself and refuses a body whose first line is not that shape. It caught a transposed sha of mine today (I typed 7a21001, the head was d86214a).

This is the tenth instance today, and it is the reason ~/lb/verdict-divergence.py exists — it reports both directions in one command across all six repos: a verdict the gate cannot read although a human wrote one, and a CHANGES silently overridden by a later verdict. This note came from its first line of output.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 3ff926e: CHANGES — one qualifier, because this head's new sentence is true of the table above it and looks false against the table two paragraphs below.

What the head adds

"No model in the table above does better at 40 than at 20 (Qwen2.5-3B 58/64 at both, Qwen2.5-0.5B 52 vs 51, SmolLM2 54 vs 53), so 20 costs the others prefill time only, not correctness."

Every number in it checks against the table it names (| model | tau 0 | tau 20 | tau 40 |, docs/npu.md:3336-3341): 58 = 58, 52 > 51, 54 > 53. As written, it is accurate.

The problem the reader will have

Two paragraphs below, the page carries a second experiment with the same two values:

docs/npu.md:3352  "Before engine#2576 (ffn_down only): default tau 40."
| model | off | tau 20 | tau 40 |
| Qwen2.5-3B-Instruct 1.7.1 hybrid | 54/64 | 57/64 | 60/64 |     ← 40 is 3 AHEAD
| SmolLM2-135M-Instruct 1.9.0      | 53/64 | 51/64 | 53/64 |     ← 40 is 2 ahead

On that build, 40 beats 20 on two of three rows. The two tables measure different code states — the second is labelled "before engine#2576 (ffn_down only)" — so they do not contradict each other in fact. But the new sentence generalises to "the others" and a reader who scrolls finds numbers where 40 is ahead, with no sentence explaining why the tables disagree.

Ask (one clause, no claim changes):

"…so 20 costs the others prefill time only, not correctness — on this build; the pre-#2576 experiment below has tau 40 ahead on two rows, which is what covering gate/up changed."

That is the sentence that turns a cross-table surprise into a finding: covering gate/up moved the optimum from 40 to 20, and the two tables are the before and after of that move. Stated that way, the pair is the strongest evidence on the page for the change; unstated, it reads as a page arguing with itself.

What is unchanged from my last review and still right

  • The defaults live in one place (kActOutlierTauDefault = 20, kActOutlierKmaxDefault = 16) with one reader each — act_outlier_tau_from (env ? atof : 20), act_outlier_kmax_from (k < 1 ? 1 : k) — and the env contract the new sentence states matches the code exactly: unset → default, "0" → 0 (off), anything else atof (so "40" → 40), kmax below 1 clamped.
  • The arms I mutated on 6d5703f still pass here (tau default … want 20, default tau picked 0 channels, want channel 100, kmax "0" gives 0, want 1), and the behavioural pair is what makes the constant meaningful.

One clause and the page reads as a before/after instead of a disagreement.

@bong-water-water-bong
bong-water-water-bong merged commit 69fd542 into main Nov 15, 2026
10 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the cl-npuperf/act-outliers-default-test branch November 15, 2026 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant