Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion qa/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "qa",
"version": "3.4.1",
"version": "3.5.0",
"description": "QA Quincey, the manual-QA persona. Verifies one defined flow against the spec or mockup and records a QA posture. qa:browser drives the real running app through the user's persistent agent-browser session (click/pixel level), walks the spec, seeds and tears down tagged data via the repo's recipe, and ends with the QA_STATUS contract that satisfies the build-time Stop hook and the PR qa-gate CI. qa:headless does the same for backend features with no UI (capturing side effects). qa:qa-plan authors the two-phase QA plan and writes the approval stamp. The plugin also SHIPS Quincey's enforcement hooks (hooks/hooks.json): the QA-plan gates (presentation, build, PR) and the QA-status Stop gate, active in opted-in repos (.qa-plan-gate.json) under ~/dev. Skills: qa:browser, qa:headless, qa:qa-plan.",
"author": {
"name": "Mujtaba Badat",
Expand Down
13 changes: 13 additions & 0 deletions qa/skills/qa-plan/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
# qa:plan CHANGELOG

## v2.1.0

Story-first companion artifact.

- The artifact template now opens with three context blocks above the
Development section: **The story** (the change's user story plus the observed
problem), **The solution being built**, and **How this plan proves it**
(numbered bullets mapping the plan's rows to the story's outcome), each with
its own FILL marker (`STORY` / `SOLUTION` / `PROOF`). Step 4b documents how to
fill them from the Step 2 success criteria (never invented). Driven by
Mujtaba's 2026-07-20 feedback: a QA artifact should say what problem is being
solved and what the solution is before showing any test rows.

## v2.0.0

Readability overhaul + a pointable companion artifact.
Expand Down
11 changes: 9 additions & 2 deletions qa/skills/qa-plan/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: qa-plan
version: 2.0.0
version: 2.1.0
description: |
QA Quincey's planning skill: turn a change's success criteria into a two-phase
QA plan written into the PR body, BEFORE the PR is reviewed or merged. Produces a
Expand Down Expand Up @@ -160,7 +160,14 @@ Mechanics:

Publish a rendered, always-linked view of the same plan as a Claude **artifact**, so there is one pretty page to point at. The artifact is a **companion**, not the source of truth: the gates only ever read the PR-body `## QA` section (Step 4). The artifact can carry the full mechanics that would bloat the table.

1. **Build the HTML.** Copy `references/artifact-template.html` (this skill's base directory) to the session scratchpad and swap the content between the `FILL:` markers: the **title**, the **Development** ELI5 + rows, the **Production** ELI5 + rows, the **Production artifacts**, and the **Definition of Done**. The artifact is the leaner companion view: it deliberately drops the QA-driver line, the `Standard (all green)` line, and the QA-posture line (those live in the PR body, the source of truth). The template is self-contained (inline CSS + data-URI images, theme-aware, no external assets, which the artifact CSP requires) and uses display checkbox glyphs (`☐` / `☑`), NEVER square-bracket checkboxes (the artifact is never gate-parsed, so glyphs are free here).
1. **Build the HTML.** Copy `references/artifact-template.html` (this skill's base directory) to the session scratchpad and swap the content between the `FILL:` markers: the **title**, the story-first **context blocks** (STORY / SOLUTION / PROOF, below), the **Development** ELI5 + rows, the **Production** ELI5 + rows, the **Production artifacts**, and the **Definition of Done**. The artifact is the leaner companion view: it deliberately drops the QA-driver line, the `Standard (all green)` line, and the QA-posture line (those live in the PR body, the source of truth).

**Story-first context blocks (required).** The page opens with three blocks ABOVE the Development section, so a reader gets what problem is being solved, what is being built, and how the plan proves it before any table:
- `FILL: STORY` - the change's user story in one sentence (`As <user>, when <situation>, I want <capability>, so <outcome>`), then the observed problem in a muted line (the incident, gap, or pain that motivated the change, with date/PR when one exists). Derive it from the same success criteria Step 2 pulled (spec, issue, mockup, or the user's own words); never invent it.
- `FILL: SOLUTION` - one paragraph naming what is being built to deliver that outcome, concrete enough that the QA rows below visibly test it.
- `FILL: PROOF` - 2-4 numbered bullets mapping the plan to the story: each states one thing the plan establishes and ends with a muted pointer to the rows that establish it (for example "(Dev rows 1-3.)", "(Prod row 2.)"). This is the bridge that lets the reader see the QA verifies the solution actually solves the story's problem.

The template is self-contained (inline CSS + data-URI images, theme-aware, no external assets, which the artifact CSP requires) and uses display checkbox glyphs (`☐` / `☑`), NEVER square-bracket checkboxes (the artifact is never gate-parsed, so glyphs are free here).

**Driver avatars.** The `Tester` cell shows a logo-only avatar that carries the driver's identity for assistive tech via `role="img"` + `aria-label` (the `title` is the hover tooltip), with the decorative inner `.pic` marked `aria-hidden`: `<span class="av" title="<id>" role="img" aria-label="<id>"><span class="pic <id>" aria-hidden="true"></span></span>`, where `<id>` is a roster id with a built-in avatar (`claude` `mutwo` `muthree` `mufour` `mujtaba`). For any driver without one, use the initials fallback: `<span class="av" title="<id>" role="img" aria-label="<id>"><span class="pic generic" aria-hidden="true">M5</span></span>`. The template's header comment documents how to add a new avatar (inline its `github.com/<handle>.png` as a data URI in a new `.pic.<id>` rule; `claude` is a hand-drawn inline-SVG burst on Anthropic clay).

Expand Down
40 changes: 38 additions & 2 deletions qa/skills/qa-plan/references/artifact-template.html
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@
<!--
/qa:plan companion-artifact template. COMPANION view, never gate-parsed.
Clone this file to the scratchpad and swap the content between the FILL markers:
FILL: TITLE / DEV ELI5 / DEV ROWS / PROD ELI5 / PROD ROWS / ARTIFACTS / DOD
FILL: TITLE / STORY / SOLUTION / PROOF / DEV ELI5 / DEV ROWS / PROD ELI5 / PROD ROWS / ARTIFACTS / DOD
The page is story-first: the reader gets the user story (problem + outcome), the
solution being built, and how the plan proves it BEFORE any QA table.
Self-contained (inline CSS + data-URI images) so it satisfies the artifact CSP.
Theme-aware (light + dark). The default content is a worked example so it renders standalone.

Expand Down Expand Up @@ -40,6 +42,17 @@
header.plan{margin-bottom:8px;}
.eyebrow{font-size:12px;letter-spacing:.14em;text-transform:uppercase;color:var(--accent);font-weight:650;margin:0 0 10px;}
h1.title{font-size:clamp(24px,3.4vw,34px);line-height:1.12;margin:0 0 8px;letter-spacing:-.02em;text-wrap:balance;font-weight:750;}
/* story-first context blocks (The story / The solution / How this plan proves it) */
.context{margin-top:22px;display:grid;gap:20px;}
.block.story{border-left:3px solid var(--accent);}
.block.solution{border-left:3px solid var(--live);}
.story-quote{font-size:15.5px;font-weight:600;line-height:1.5;margin:0 0 12px;text-wrap:pretty;}
.story-quote .role{color:var(--accent);}
.incident{font-size:13.5px;color:var(--muted);}
ul.proof{list-style:none;padding:0;margin:0;display:grid;gap:10px;}
ul.proof li{display:flex;gap:10px;align-items:flex-start;font-size:14px;max-width:78ch;}
ul.proof .tick{color:var(--live);font-weight:700;line-height:1.4;flex:0 0 auto;}
ul.proof .where{color:var(--muted);font-size:12.5px;}
section.phase{margin-top:34px;}
.phase-head{display:flex;align-items:center;gap:11px;margin:0 0 12px;}
.phase-head h2{font-size:18px;margin:0;font-weight:700;letter-spacing:-.01em;}
Expand All @@ -65,7 +78,7 @@
.pic.generic{background:linear-gradient(135deg,#5b6675,#3d4756);}
.aux{margin-top:34px;display:grid;gap:20px;}
.block{background:var(--panel);border:1px solid var(--hair);border-radius:14px;box-shadow:var(--shadow);padding:18px 20px;}
.block h3{font-size:12px;letter-spacing:.1em;text-transform:uppercase;color:var(--muted);margin:0 0 12px;font-weight:700;}
.block h3,.context .block h2{font-size:12px;letter-spacing:.1em;text-transform:uppercase;color:var(--muted);margin:0 0 12px;font-weight:700;}
.block.artifact{border-left:3px solid var(--live);}
.block p{margin:0 0 9px;}
.block p:last-child{margin-bottom:0;}
Expand All @@ -82,6 +95,29 @@
<h1 class="title">email-hero: LLM-failure health check</h1>
</header>

<div class="context">
<div class="block story">
<h2>The story</h2>
<!-- FILL: STORY. The user story (As <user>, when <situation>, I want <capability>, so <outcome>), then the observed problem in muted text. Pull it from the change's success criteria / issue / PR summary; never invent it. -->
<p class="story-quote"><span class="role">As Mujtaba,</span> when email-hero's LLM starts failing in production, I want the health check to notice and flag it, so silent LLM breakage surfaces in the mini's triage instead of going unnoticed for days.</p>
<p class="incident">Observed problem: LLM calls were failing quietly inside the daemon; <code>email-hero health</code> stayed green because nothing recorded the failures where it could see them.</p>
</div>
<div class="block solution">
<h2>The solution being built</h2>
<!-- FILL: SOLUTION. One paragraph: what is being built to deliver the story's outcome. -->
<p>A per-failure sink records every LLM failure to <code>llm-failures.jsonl</code>, and <code>email-hero health</code> counts the failures inside its window, exiting non-zero past <code>--llm-max-failures</code> so mini check #8 turns red when the LLM is breaking.</p>
</div>
<div class="block">
<h2>How this plan proves it</h2>
<!-- FILL: PROOF. 2-4 numbered bullets mapping the plan's rows to the story's outcome; each ends with a muted (Dev/Prod row N) pointer. -->
<ul class="proof">
<li><span class="tick">1.</span><span>The counting rules are right in isolation: unit tests cover in-window counting, torn lines, and the cap. <span class="where">(Dev row 1.)</span></span></li>
<li><span class="tick">2.</span><span>A forced failure on a real copy is noticed end to end: the sink records it and <code>health</code> exits non-zero at the threshold. <span class="where">(Dev row 2.)</span></span></li>
<li><span class="tick">3.</span><span>The live daemon on the mini behaves the same after deploy, which is the story's outcome verified in production. <span class="where">(Prod row 1.)</span></span></li>
</ul>
</div>
</div>

Comment thread
coderabbitai[bot] marked this conversation as resolved.
<section class="phase dev">
<div class="phase-head"><h2>🖥️ Development</h2></div>
<!-- FILL: DEV ELI5 (one plain-language line, no jargon) -->
Expand Down
Loading