[grunion-ai]

What changed

New benchmarks/authoring/: five frozen prompts, the semantic keys each is checked against, and benchmark.mjs check|score to score an agent's frozen candidates.

Why

Closes #208. Decklet's tests prove the engine. Nothing measured whether an agent following SKILL.md gets to FINALIZE PASS on its first try, so a SKILL.md edit shipped with no number before or after. This follows archify's benchmarks/ordinary-model-floor pattern, sized for decklet.

• Cases: a pitch, a QBR, a one-page document-letter, a carousel-4x5, and a twelve-vendor landscape. The landscape's twelve SVG logos sit in inputs/landscape/, which the runner copies to the agent.
• Semantic keys are checked on model.json: format, slide range, required deck.entities, and minimum chart / logo / donut / img counts.
• Scoring: score <run-dir> runs finalize on each candidate and writes one row per case, with pass, failed stage, misses, top three diagnostic codes (from Coded validate diagnostics #214 and Coded verify diagnostics #222), and repairs and minutes from the runner's run.json.
• No model launch. The runner owns models, credentials, timing and transcripts.
• References: five hand-built candidates prove the wiring. Two come from bin/new.mjs; three were hand-built, and the one-pager took one repair, found through verify --json (parity/text-over-text on row r2). run.json marks them "evidence": false, and the CLI says so.
• Protocol: the README sets the fair-run rules (same commit and prompts; an installed decklet outside this repo; attempt 1 frozen) and where results go (results/).
• Packaging: benchmarks/ is not in the npm files list, so an installed decklet never contains the cases. .gitignore covers the decks and receipts that scoring writes beside each candidate.

How it was verified

• test/benchmark.test.mjs: 7 pass. It covers:
  • the shipped manifest via the API and the CLI, and a broken manifest naming each problem;
  • semantic misses listed in order;
  • stubbed scoring of every reference (5/5, evidence: false);
  • a missing candidate and a broken one (fit/nowrap-width in its top codes);
  • score --out;
  • a live Chromium run of all five references to FINALIZE PASS.
• npm run bench:check prints manifest ok: 5 cases.
• npm test with Chromium only: 647 pass, 19 fail, the same 19 that fail on untouched main in this container (17 WebKit cases with no WebKit installed, two sub-pixel toolbar overlaps from container fonts). CI runs all three engines.
• deck.html / library.html rebuild: not needed, the engine is untouched.

---

Generated by Claude Code