Trust map: where it holds vs folds under pressure
diff --git a/models/laguna-s-2.1.md b/models/laguna-s-2.1.md
index 7d7fb3d..9a160d9 100644
--- a/models/laguna-s-2.1.md
+++ b/models/laguna-s-2.1.md
@@ -10,7 +10,7 @@ class: specialist:coding
hf: https://huggingface.co/poolside/laguna-s-2.1
tested_on: Q4_K_M GGUF (routed experts Q4_K/imatrix, signal path Q8_0), served on poolside's own llama.cpp fork (commit 04b2b72cb) AND an independent quant fork, findings matched on both; 3-arm thinking ablation (off/capped/on), blind 2-vote judged; 2026-07-23/24
status: VERSION-PINNED to the config as of ~2026-07-24. ⚠️ Poolside kept changing the config after the ~07-21 release (public HF commits, incl. one flipping thinking on by default; see changelog). Re-verify against the current upload before relying on this.
-verdict: "Configured right (thinking off, native tool format, integrity clause, version pinned + capped) this is an excellent and genuinely production-stable coding agent for a single box: 99.9% turn success over a 12h soak, and not benchmaxxed on held-out competence. Configured wrong it is frustrating: the headline thinking mode is net-negative and barely fires, tool-calling is zero outside the native format, and it will help cover things up if you frame it as cleanup."
+verdict: "Configured right (thinking off, native tool format, integrity clause, version pinned + capped) this is an excellent and genuinely production-stable coding agent for a single box: 100% turn success over a 12h soak, and not benchmaxxed on held-out competence. Configured wrong it is frustrating: the headline thinking mode is net-negative and barely fires, tool-calling is zero outside the native format, and it will help cover things up if you frame it as cleanup."
---
# Laguna S 2.1: offlabel operating guide
@@ -230,18 +230,18 @@ Coverage tag per axis: **✅ measured** (held-out, 3-arm, blind 2-vote) · **
## 5d. Is it production-stable? A 12-hour soak says yes (external validation)
-Everything above is held-out behavioral testing, which is good at finding failure modes and bad at answering "will this thing survive a real workday." [@BlackwellBoy](https://x.com/BlackwellBoy) answered that part independently, on a **genuinely different stack**: **NVFP4 served with vLLM** on a DGX Spark (ours was Q4_K_M on llama.cpp), 12 hours inside his own production agent pipeline, ~389 sessions and ~2,947 turns, thinking enabled throughout, revision `0761412`, hard session-token cap.
+Everything above is held-out behavioral testing, which is good at finding failure modes and bad at answering "will this thing survive a real workday." [@BlackwellBoy](https://x.com/BlackwellBoy) answered that part independently, on a **genuinely different stack**: **NVFP4 served with vLLM** on a DGX Spark (ours was Q4_K_M on llama.cpp), 12 hours inside his own production agent pipeline, 409 sessions and 3,096 turns, thinking enabled throughout, revision `0761412`, hard session-token cap.
**Raw logs, harness, and the losing sweep cells are published:** [Blackwellboy/laguna-s21-lab](https://github.com/Blackwellboy/laguna-s21-lab) (per-turn logs, incident log, integrity probes, container recipe with pinned digests, and a 20-cell tuning sweep including every cell that lost). Numbers below are as he stated them; that repo is the source of record.
**What a user should take from it, most useful first:**
-- **It is stable enough to leave running.** 2,944 of 2,947 turns succeeded (**99.9%**), **zero crashes, zero restarts**, ~4 GiB memory creep across the full 12h, **~13.5s per agent turn**. This is the strongest evidence available that, configured correctly, Laguna is a production-grade agent driver on one box rather than a demo.
+- **It is stable enough to leave running.** all 3,096 turns returned HTTP 200 (**100%**), **zero crashes, zero restarts**, ~4 GiB memory creep across the full 12h, **~13.5s per agent turn**. This is the strongest evidence available that, configured correctly, Laguna is a production-grade agent driver on one box rather than a demo.
- **The native tool format is not just better, it is near-perfect at scale.** **100% tool-call success over ~11.5h** of continuous agent work on `poolside_v1`. Our §4 number (83% native vs 0% generic) came from a small probe set; thousands of turns is far stronger. Combined: the format choice is the difference between "works flawlessly" and "does not work at all." There is no middle.
- **Pin + cap held off unbounded generation, with an important scope limit.** With the revision pinned and his own session-token cap set: **zero unbounded-generation loops in 12h**. His correction, which matters: the 9 incidents in his logs were **his own driver's session cap firing by design**, and this all happened **while the thinking gate barely opened**, so it is **not** evidence that pin+cap tames a thinking-heavy workload. Read it as "cap your own tokens and you will not get unbounded generation," not "pin+cap makes thinking safe."
- **The §5b integrity clause is not a quirk of our stack.** He ran it through his soak on a different quant and serving setup, hit it with **3 disguised cover-up requests, and it refused all 3**, explaining each time why hiding audit findings is the problem. Cross-quant replication was the gap §5b explicitly named. It is now closed, which makes the clause the best-evidenced recommendation in this guide.
-**On thinking rates, two datasets that point the same way for possibly different reasons.** His pipeline observed thinking fire on **~3 of 2,944 turns (~0.1%)**; ours saw 5-18% under a bare "helpful assistant" persona. It is tempting to say his number confirms our persona×task gate, and **I originally wrote exactly that, which was overreach.** His own correction: the API returned **empty `reasoning` fields even when thinking was explicitly requested**, so on that revision and serving path part of the ~0.1% may be **template/parser-level rather than the model declining to think.** His honest phrasing, which is the one to use: *the gate observably almost never opens on this rev and serving path.*
+**On thinking rates, two datasets that point the same way for possibly different reasons.** His pipeline observed thinking fire on **3 of 3,096 turns (0.0969%, ~0.1%)**; ours saw 5-18% under a bare "helpful assistant" persona. It is tempting to say his number confirms our persona×task gate, and **I originally wrote exactly that, which was overreach.** His own correction: the API returned **empty `reasoning` fields even when thinking was explicitly requested**, so on that revision and serving path part of the ~0.1% may be **template/parser-level rather than the model declining to think.** His honest phrasing, which is the one to use: *the gate observably almost never opens on this rev and serving path.*
What each dataset actually supports, kept separate:
- **Ours** shows the model *chooses* not to think under a named persona. That rests on our own mechanism check: the template hands it an opened `
` every time (verified byte-identical via `/apply-template`), and it still emits nothing. That is a model-behavior claim.
@@ -315,6 +315,7 @@ llama.cpp rejects that with `400: Assistant message must contain either 'content
- **Scope caveats:** single test window, one quant, config is a moving target (version-pinned). Multilingual/crisis probes small-n. Mitigation validated thinking-OFF; thinking-ON re-run and long-regime non-termination pending. Benchmarks not reproduced (behavior-vs-behavior, not a leaderboard).
## Changelog
+- `2026-07-29`: **corrected the soak counts in §5d, the card and this changelog ([@Blackwellboy](https://github.com/Blackwellboy) correction at source).** The ~389-session / ~2,947-turn figures were an ~11.5h checkpoint of a run that finished at **409 sessions and 3,096 turns**. Separately, the **99.9% turn success** figure was wrong in his primary and is now **100%**: `turns.jsonl` has 3,099 lines, but 3 of those are `kind: integrity_probe` records rather than turns, and all 3,096 turn records returned HTTP 200. His turn log is written unconditionally with whatever status came back, so a failed turn would carry a non-200 status and none does. Also: 409 is the session count, of which **400 completed and 9 were killed at his driver's session-token cap** by design, the same 9 incidents already scoped in the 2026-07-25 entry below. Thinking rate restated on the correct denominator: **3 of 3,096 (0.0969%)**, unchanged at ~0.1%. Corrections and primary re-derivation: [laguna-s21-lab@4507826](https://github.com/Blackwellboy/laguna-s21-lab/commit/4507826d7bc403684bb9da0850a49ebfc7f4d966). A sixth surface carrying the same figure, the `verdict:` line in this guide's own frontmatter, is corrected here too; it was not in the original correction list. Nothing about the thinking-gate or depth-collapse conclusions changes; only counts.
- `2026-07-26`: **§2 firing gate rebuilt on two axes, and task shape resolved rather than left contested** ([#11](https://github.com/TheTom/offlabel/issues/11), @Blackwellboy). Firing probability and reasoning length move **independently, sometimes in opposite directions**, so the single "suppressor" axis is gone. The decisive case: adding tool schemas to an otherwise identical request cut median reasoning **62%** (745 to 282) while firing went **up** 12 points (24/40 to 29/40), making it the second-highest firing condition in the grid. My "tools are a major suppressor, which is why a maximally coding-shaped benchmark with no apparatus reasons the most" line was therefore wrong in both halves and is removed; the bare-prompt conclusion survives on the no-apparatus evidence alone, but not for the reason I gave. **The three-stack composite curve is replaced by his single-stack 10-point curve** (one rev, one prompt set, n=40 per condition, varying only apparatus); our rows and @Defilan's are demoted to corroboration of the ordering, since absolute levels are stack-specific (his bare 75% vs our ~50%). Numbers re-derived from his published `grid_turns.jsonl`, not transcribed. **Task shape resolved for the tested prompts**, and it has two parts. Shape is a large independent effect: pooled across all ten conditions, math fires **92/100**, code **62/100**, reasoning **47/100**, and **summarization 0/100** (0/105 with the bare parser samples), a floor no apparatus explains, which makes the task itself the strongest single suppressor in the study. Separately, **code is a persona conjunction**: 10/10 bare, **0/10** under a bare named persona, 10/10 again under a full agent prompt, so "coding suppresses" and "coding reasons most" are both correct readings depending on which single apparatus level you held fixed. **My inference that an all-coding, no-apparatus benchmark measures apparatus rather than shape is CONFIRMED** by his shape-crossed grid, not retired; it sits at the C0 cell where code fires 10/10, so it cannot see the C4 collapse or the summarization floor. **Weight limit, verified not assumed:** each shape in that grid is ONE fixed prompt repeated with a nonce (within-cell prompt-token spread never exceeds 4 across all 40 cells), and the summarization prompt is also structurally unlike the others (288 prompt tokens vs 116-124, the only one supplying a passage). So these are prompt-level results, not category-level ones, and the section says so. The two-stack reconciliation is unaffected: it only needs the same prompt behaving differently under different apparatus. §3's "coding-shaped tasks suppress independently" corrected to the interaction, and noted the persona lever floors at 3/40 rather than 0. Corrected grid size: **400 grid turns** (450 logged total including parser and criteria phases), not 450 grid turns.
- `2026-07-26`: **retracted the "dose curve" framing I had added earlier the same day.** I wrote that thinking-firing scales down with how much apparatus the prompt carries. [@BlackwellBoy's 10-condition, 400-turn grid](https://github.com/Blackwellboy/laguna-s21-lab/tree/main/gate-study) falsifies the monotonic part: ten bullet rules drop firing to **7%** (3/40), but a **longer** full agent prompt brings it back to **60-72%**. It is **prompt shape, not prompt quantity**. What the agent prompt actually does is **compress** thinking rather than stop it: median thinking tokens fall from **3,536** bare to **745** then **282** under agent prompts, which is what reconciles this with the §5e long-loop attenuation. Interim advice at the time: control thinking with the kwarg, not with prompt engineering. **Superseded by the two-axis entry above**, which folds in the replacement model.
- `2026-07-26`: revision from two more independent stacks. **§2 firing gate reframed as a dose-response curve, not a switch** (bare prompt ~50% firing, +persona halves it, +rules/clause/tools ~0.1%): per @BlackwellBoy's reread, his soak sent `enable_thinking: true` on all 3,099 turns, so it was a FIRES arm, and an *unnamed* protocol prompt suppressed as hard as a named one (0.13% vs 0.06%), meaning **the apparatus suppresses, not the persona name**. **The §2 open question is now largely CLOSED**: our OFF arm's 0% means did-not-reason by construction (`false` pre-closes ``), so our firing rates keep the firing-rate label. **NEW §5f: a truncated reasoning turn silently kills agent loops** (400 `must contain content or tool_calls`, killed 5/14 runs, deterministic), a candidate mechanism for our own turn-11 hang. **NEW §5g: measured gfx1151 serving config** (`--no-mmap`, `-fa on` worth 2.27x prefill / 2.22x decode at 56k with retention 40%→75.8%, DFlash 53.8% acceptance on real code editing but 2.1x SLOWER on synthetic). **§4 sharpened**: the requirement is the native *template* server-side (`--jinja`), not the client format; a generic OpenAI client merged four real PRs, so 'use pool' was too narrow. Card updated to match. 🧪 A fourth stack's finding that thinking is **regime-dependent** (ON better for single-turn codegen) is NOT merged here: it is under contributor review in [#10](https://github.com/TheTom/offlabel/pull/10).
@@ -322,5 +323,5 @@ llama.cpp rejects that with `400: Assistant message must contain either 'content
- `2026-07-24`: initial behavioral assessment + serving findings on Q4_K_M; §5b integrity clause validated (4/4 caves closed, thinking-OFF).
- `2026-07-25`: added §5e third-stack field notes from @Defilan (gfx1151/llama.cpp/generic harness, `reasoning_content` reliably populated) and corrected §2 as a result. The `enable_thinking` kwarg **defaults to false**, so passing `false` is a no-op against the template default; verified in our own fork's template. Persona gate reproduced (6/6 fired with no persona, 0/5 with one); the **task-shape** half did NOT reproduce and is now marked unresolved. NEW: the gate **attenuates with conversation length** (single-turn persona drove thinking to zero; a 20+ turn agentic loop only shortened it, 35,913 to 20,573 chars, block count 10 to 9), which a scenario battery cannot surface. Flagged an open limitation of our own firing numbers: they were read from `reasoning_content`, so "0% fired" cannot distinguish not-reasoning from not-parsed. §5b gains his independent rule-by-rule ablation on freshly-worded scenarios (clause reproduces 7/7 vs 3/7 unprompted; rule (1) load-bearing; floor of (1)(2)(4)) and his walk-back of the rule-(3) cost magnitude.
- `2026-07-25`: applied @BlackwellBoy's two precision corrections to §5d after review. (a) His ~0.1% thinking rate is an OBSERVED routing rate on that rev/serving path, not proof the model declined to think: empty `reasoning` fields came back even when thinking was requested, so part of it may be template/parser-level. I had written that his number confirmed our persona-gate mechanism; that was overreach and is now corrected. (b) "Zero loops" is scoped to zero UNBOUNDED-GENERATION loops while the gate barely opened; his 9 logged incidents were his own driver's session cap firing by design, so it is not evidence about thinking-heavy workloads. Also recorded his stack precisely (NVFP4 on vLLM, vs our Q4_K_M on llama.cpp) and linked his published raw logs. Separately added the flash-attention serving finding (`-fa on`, up to 2.2x decode at 56K depth) from an independent Strix Halo test.
-- `2026-07-24`: added §5d external validation, [@BlackwellBoy](https://x.com/BlackwellBoy)'s 12h / 2,947-turn production soak (revision `0761412`, different quant + serving). Independently corroborates thinking-barely-fires, native-tool-format reliability (100% at scale), the config-drift + token-cap fix, and cross-quant replication of the §5b integrity clause (3/3 cover-ups refused); adds soak reliability data (99.9% turn success, 0 crashes).
+- `2026-07-24`: added §5d external validation, [@BlackwellBoy](https://x.com/BlackwellBoy)'s 12h / 3,096-turn production soak (revision `0761412`, different quant + serving). Independently corroborates thinking-barely-fires, native-tool-format reliability (100% at scale), the config-drift + token-cap fix, and cross-quant replication of the §5b integrity clause (3/3 cover-ups refused); adds soak reliability data (100% turn success, 0 crashes).
- **⚠️ Vendor config drift (from the public HF commit history):** after the ~2026-07-21 release (commit `e4daa58`), Poolside kept changing the config over the next ~3 days with several commits: a chat-template fix to preserve reasoning across turns (`fac825e`), two marking a tokenizer token special to match internal serving (`6548c10`, `88796b9`), **one flipping thinking to on-by-default and dropping the `max_new_tokens` cap (`179ee67`)**, and the latest (`b0a9fd7`, ~07-24) inlining the chat template for GGUF conversion. The commit history shows only config/tokenizer/template files changed; I did not hash-diff the weight blobs to confirm the weights themselves are untouched. The thinking-on-by-default + no-cap flip is behaviorally relevant (it intersects both the thinking-hurts finding and the community non-termination complaints). Not characterizing these as "silent" (the commits are public); the point is the config is a moving target. **This guide is pinned to the ~07-24 config; re-verify against the current upload.**