Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions TODOS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# TODOS

gstack-extensions TODOs: grouped by plugin/component, `**Priority:**` P0-P4, completed items move to the bottom.

## Completed
2 changes: 1 addition & 1 deletion qa/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "qa",
"version": "3.5.0",
"version": "3.6.0",
"description": "QA Quincey, the manual-QA persona. Verifies one defined flow against the spec or mockup and records a QA posture. qa:browser drives the real running app through the user's persistent agent-browser session (click/pixel level), walks the spec, seeds and tears down tagged data via the repo's recipe, and ends with the QA_STATUS contract that satisfies the build-time Stop hook and the PR qa-gate CI. qa:headless does the same for backend features with no UI (capturing side effects). qa:qa-plan authors the two-phase QA plan and writes the approval stamp. The plugin also SHIPS Quincey's enforcement hooks (hooks/hooks.json): the QA-plan gates (presentation, build, PR) and the QA-status Stop gate, active in opted-in repos (.qa-plan-gate.json) under ~/dev. Skills: qa:browser, qa:headless, qa:qa-plan.",
"author": {
"name": "Mujtaba Badat",
Expand Down
12 changes: 12 additions & 0 deletions qa/skills/qa-plan/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# qa:plan CHANGELOG

## v2.2.0

Checkbox-free, skimmable companion artifact.

- Template restructure (Mujtaba's 2026-07-22 feedback): terse "Story" /
"Solution" headings; the Solution and Production-artifacts blocks are bullet
lists, never paragraphs; the Story block ends with a muted "Linked issue:"
line (the GitHub issue when one exists, else "none" + provenance); and the
artifact carries NO checkboxes at all (no Dev-table column, plain-bullet
Definition of Done). Checkbox state lives only in the PR body's ## QA
section, which the merge gates read. Step 4b instructions updated to match.

## v2.1.0

Story-first companion artifact.
Expand Down
14 changes: 8 additions & 6 deletions qa/skills/qa-plan/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: qa-plan
version: 2.1.0
version: 2.2.0
description: |
QA Quincey's planning skill: turn a change's success criteria into a two-phase
QA plan written into the PR body, BEFORE the PR is reviewed or merged. Produces a
Expand Down Expand Up @@ -162,20 +162,22 @@ Publish a rendered, always-linked view of the same plan as a Claude **artifact**

1. **Build the HTML.** Copy `references/artifact-template.html` (this skill's base directory) to the session scratchpad and swap the content between the `FILL:` markers: the **title**, the story-first **context blocks** (STORY / SOLUTION / PROOF, below), the **Development** ELI5 + rows, the **Production** ELI5 + rows, the **Production artifacts**, and the **Definition of Done**. The artifact is the leaner companion view: it deliberately drops the QA-driver line, the `Standard (all green)` line, and the QA-posture line (those live in the PR body, the source of truth).

**Story-first context blocks (required).** The page opens with three blocks ABOVE the Development section, so a reader gets what problem is being solved, what is being built, and how the plan proves it before any table:
- `FILL: STORY` - the change's user story in one sentence (`As <user>, when <situation>, I want <capability>, so <outcome>`), then the observed problem in a muted line (the incident, gap, or pain that motivated the change, with date/PR when one exists). Derive it from the same success criteria Step 2 pulled (spec, issue, mockup, or the user's own words); never invent it.
- `FILL: SOLUTION` - one paragraph naming what is being built to deliver that outcome, concrete enough that the QA rows below visibly test it.
**Story-first context blocks (required).** The page opens with three blocks ABOVE the Development section, headed tersely **"Story"**, **"Solution"**, and **"How this plan proves it"**, so a reader gets what problem is being solved, what is being built, and how the plan proves it before any table:
- `FILL: STORY` - the change's user story in one sentence (`As <user>, when <situation>, I want <capability>, so <outcome>`), then the observed problem in a muted line (the incident, gap, or pain that motivated the change, with date/PR when one exists), then a muted **`Linked issue:`** line: link the GitHub issue when one exists; otherwise write `none` plus where the change originated (e.g. "requested in-session, <date>"). Derive it all from the same success criteria Step 2 pulled (spec, issue, mockup, or the user's own words); never invent it.
- `FILL: SOLUTION` - a short **bullet list** (2-5 bullets, `ul.proof` markup), never a paragraph, naming what is being built to deliver that outcome, concrete enough that the QA rows below visibly test it.
- `FILL: PROOF` - 2-4 numbered bullets mapping the plan to the story: each states one thing the plan establishes and ends with a muted pointer to the rows that establish it (for example "(Dev rows 1-3.)", "(Prod row 2.)"). This is the bridge that lets the reader see the QA verifies the solution actually solves the story's problem.

The template is self-contained (inline CSS + data-URI images, theme-aware, no external assets, which the artifact CSP requires) and uses display checkbox glyphs (`☐` / `☑`), NEVER square-bracket checkboxes (the artifact is never gate-parsed, so glyphs are free here).
The **Production artifacts** block is likewise a **bullet list** (one bullet per artifact fact), never a paragraph.

The template is self-contained (inline CSS + data-URI images, theme-aware, no external assets, which the artifact CSP requires) and carries **NO checkboxes of any kind**: no checkbox column in the Development table, plain bullets in the Definition of Done, and never square-bracket boxes. Checkbox state lives ONLY in the PR body's `## QA` section, the source of truth the merge gates read; the artifact is the readable view of the plan, not a tracker.

**Driver avatars.** The `Tester` cell shows a logo-only avatar that carries the driver's identity for assistive tech via `role="img"` + `aria-label` (the `title` is the hover tooltip), with the decorative inner `.pic` marked `aria-hidden`: `<span class="av" title="<id>" role="img" aria-label="<id>"><span class="pic <id>" aria-hidden="true"></span></span>`, where `<id>` is a roster id with a built-in avatar (`claude` `mutwo` `muthree` `mufour` `mujtaba`). For any driver without one, use the initials fallback: `<span class="av" title="<id>" role="img" aria-label="<id>"><span class="pic generic" aria-hidden="true">M5</span></span>`. The template's header comment documents how to add a new avatar (inline its `github.com/<handle>.png` as a data URI in a new `.pic.<id>` rule; `claude` is a hand-drawn inline-SVG burst on Anthropic clay).

2. **Publish, idempotently.**
- **Re-run (URL already in the PR body):** read the existing `📄 Plan view:` URL from the body and call `Artifact` with that same `file_path` AND `url: <existing-url>` so it updates in place and the link never changes.
- **First run (no URL yet):** call `Artifact` with the `file_path` (no `url`). Use a **stable** `title` and `favicon` (🧪) so redeploys stay one artifact. Take the returned URL and write it into the `📄 Plan view:` line of the PR body (re-edit the body via `gh pr edit`). If there is no PR yet, hold the URL and include the `📄 Plan view:` line when `/ship` folds the section in (or hand the user the URL to paste).

3. **Tell the user it is private.** Artifacts are private by default. If a reviewer or teammate needs to open the plan view, the user shares it from the artifact page's share menu; the PR body link works for anyone who can see the PR regardless. Note that the artifact is a **snapshot at plan-authoring time** (boxes unchecked); the live checkbox state is in the PR body. Re-running `/qa:plan` refreshes the artifact.
3. **Tell the user it is private.** Artifacts are private by default. If a reviewer or teammate needs to open the plan view, the user shares it from the artifact page's share menu; the PR body link works for anyone who can see the PR regardless. Note that the artifact is a **snapshot at plan-authoring time** and deliberately carries no checkboxes; the live checkbox state is in the PR body's `## QA` section. Re-running `/qa:plan` refreshes the artifact.

If the `Artifact` tool is unavailable (non-interactive / headless run), skip this step, keep the PR-body section (the source of truth) intact, and note that the companion artifact was not published.

Expand Down
56 changes: 33 additions & 23 deletions qa/skills/qa-plan/references/artifact-template.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,18 @@
/qa:plan companion-artifact template. COMPANION view, never gate-parsed.
Clone this file to the scratchpad and swap the content between the FILL markers:
FILL: TITLE / STORY / SOLUTION / PROOF / DEV ELI5 / DEV ROWS / PROD ELI5 / PROD ROWS / ARTIFACTS / DOD
(DEV ROWS have NO checkbox cell; DOD is plain bullets; SOLUTION + ARTIFACTS are bullet lists.)
The page is story-first: the reader gets the user story (problem + outcome), the
solution being built, and how the plan proves it BEFORE any QA table.
solution, and how the plan proves it BEFORE any QA table. Headings are terse:
"Story" / "Solution" / "How this plan proves it". The Story block ends with a
muted "Linked issue:" line pointing at the GitHub issue when one exists (write
"none" plus provenance when it does not). The Solution and Production-artifacts
blocks are BULLET LISTS (ul.proof), not paragraphs. The artifact carries NO
checkboxes of any kind (no column in the Dev table, no glyphs in the DoD);
checkbox state lives only in the PR body's ## QA section, which the gates read.
Self-contained (inline CSS + data-URI images) so it satisfies the artifact CSP.
Theme-aware (light + dark). The default content is a worked example so it renders standalone.
(Restructured 2026-07-22: terse headings, bulleted Solution/Artifacts, checkbox-free.)

DRIVER AVATARS: the Tester column shows a logo-only avatar. It carries the driver's
identity for assistive tech via role="img" + aria-label (title= is the hover tooltip);
Expand Down Expand Up @@ -42,7 +50,7 @@
header.plan{margin-bottom:8px;}
.eyebrow{font-size:12px;letter-spacing:.14em;text-transform:uppercase;color:var(--accent);font-weight:650;margin:0 0 10px;}
h1.title{font-size:clamp(24px,3.4vw,34px);line-height:1.12;margin:0 0 8px;letter-spacing:-.02em;text-wrap:balance;font-weight:750;}
/* story-first context blocks (The story / The solution / How this plan proves it) */
/* story-first context blocks (Story / Solution / How this plan proves it) */
.context{margin-top:22px;display:grid;gap:20px;}
.block.story{border-left:3px solid var(--accent);}
.block.solution{border-left:3px solid var(--live);}
Expand All @@ -63,7 +71,6 @@
thead th{text-align:left;font-weight:650;font-size:12px;letter-spacing:.03em;text-transform:uppercase;color:var(--muted);padding:12px 16px;border-bottom:1px solid var(--hair-strong);white-space:nowrap;}
tbody td{padding:13px 16px;border-bottom:1px solid var(--hair);vertical-align:top;}
tbody tr:last-child td{border-bottom:none;}
td.check{width:34px;text-align:center;font-size:18px;color:var(--accent);line-height:1;}
td.tester{white-space:nowrap;}
td.expect{color:var(--muted);}
td.notes{color:var(--muted);font-size:13px;}
Expand All @@ -83,9 +90,6 @@
.block p{margin:0 0 9px;}
.block p:last-child{margin-bottom:0;}
.block .mono{font-size:13px;word-break:break-word;}
ul.dod{list-style:none;padding:0;margin:0;display:grid;gap:9px;}
ul.dod li{display:flex;gap:10px;align-items:flex-start;font-size:14px;}
ul.dod .box{color:var(--accent);font-size:17px;line-height:1.35;}
</style>

<div class="wrap">
Expand All @@ -97,15 +101,20 @@ <h1 class="title">email-hero: LLM-failure health check</h1>

<div class="context">
<div class="block story">
<h2>The story</h2>
<!-- FILL: STORY. The user story (As <user>, when <situation>, I want <capability>, so <outcome>), then the observed problem in muted text. Pull it from the change's success criteria / issue / PR summary; never invent it. -->
<h2>Story</h2>
<!-- FILL: STORY. The user story (As <user>, when <situation>, I want <capability>, so <outcome>), then the observed problem in muted text, then a muted 'Linked issue:' line: an <a> to the GitHub issue when one exists, else 'none' plus where the change originated. Pull it from the change's success criteria / issue / PR summary; never invent it. -->
<p class="story-quote"><span class="role">As Mujtaba,</span> when email-hero's LLM starts failing in production, I want the health check to notice and flag it, so silent LLM breakage surfaces in the mini's triage instead of going unnoticed for days.</p>
<p class="incident">Observed problem: LLM calls were failing quietly inside the daemon; <code>email-hero health</code> stayed green because nothing recorded the failures where it could see them.</p>
<p class="incident">Linked issue: none (change originated in-session).</p>
</div>
<div class="block solution">
<h2>The solution being built</h2>
<!-- FILL: SOLUTION. One paragraph: what is being built to deliver the story's outcome. -->
<p>A per-failure sink records every LLM failure to <code>llm-failures.jsonl</code>, and <code>email-hero health</code> counts the failures inside its window, exiting non-zero past <code>--llm-max-failures</code> so mini check #8 turns red when the LLM is breaking.</p>
<h2>Solution</h2>
<!-- FILL: SOLUTION. Bullet list (ul.proof, bullet tick spans): 2-5 short bullets naming what is being built to deliver the story's outcome. Never a paragraph. -->
<ul class="proof">
<li><span class="tick">&#8226;</span><span>A per-failure sink records every LLM failure to <code>llm-failures.jsonl</code>.</span></li>
<li><span class="tick">&#8226;</span><span><code>email-hero health</code> counts failures inside its window and exits non-zero past <code>--llm-max-failures</code>.</span></li>
<li><span class="tick">&#8226;</span><span>Mini check #8 turns red when the LLM is breaking, surfacing silent failures in triage.</span></li>
</ul>
</div>
<div class="block">
<h2>How this plan proves it</h2>
Expand All @@ -124,18 +133,16 @@ <h2>How this plan proves it</h2>
<p class="eli5">On a throwaway copy we fake an LLM failure and check the health command notices and flags it, all before this can merge.</p>
<div class="card"><div class="tablewrap">
<table>
<thead><tr><th>✓</th><th>Tester</th><th>Check</th><th>Expect</th><th>Notes</th></tr></thead>
<thead><tr><th>Tester</th><th>Check</th><th>Expect</th><th>Notes</th></tr></thead>
<tbody>
<!-- FILL: DEV ROWS. Use ☐ (open) / ☑ (passed) in the check cell, never square-bracket checkboxes. Tester = avatar span. -->
<!-- FILL: DEV ROWS. NO checkbox cell (pass/fail state lives in the PR body, the source of truth the gates read). Tester = avatar span. -->
<tr>
<td class="check">☐</td>
<td class="tester"><span class="av" title="claude" role="img" aria-label="claude"><span class="pic claude" aria-hidden="true"></span></span></td>
<td>Unit tests for <code>llm_health</code></td>
<td class="expect">Assertions pass; sink fires once per failure; fail-safes never raise</td>
<td class="notes">in-window counting, torn-line skip, cap trim</td>
</tr>
<tr>
<td class="check">☐</td>
<td class="tester"><span class="av" title="mutwo" role="img" aria-label="mutwo"><span class="pic mutwo" aria-hidden="true"></span></span></td>
<td>E2E on mini: force an LLM failure, run <code>health --llm-max-failures 1</code></td>
<td class="expect">Row in <code>llm-failures.jsonl</code>; exits <code>1</code> at threshold 1, <code>0</code> at high</td>
Expand Down Expand Up @@ -169,17 +176,20 @@ <h2>How this plan proves it</h2>
<div class="aux">
<div class="block artifact">
<h3>Production artifacts</h3>
<!-- FILL: ARTIFACTS. The exact thing production runs (image digest/tag, bundle id, deploy id) + host + how exercised. -->
<p><span class="mono">email-hero</span> on <code>mutwos-mac-mini</code> at the merged <code>main</code> commit, exercised by <code>email-hero health</code> (check #8, every 600s) + the <code>com.emailhero.watch</code> daemon.</p>
<!-- FILL: ARTIFACTS. Bullet list (ul.proof): one bullet per artifact fact: the exact thing production runs (image digest/tag, bundle id, deploy id), the host, how it is exercised. Never a paragraph. -->
<ul class="proof">
<li><span class="tick">&#8226;</span><span><span class="mono">email-hero</span> on <code>mutwos-mac-mini</code> at the merged <code>main</code> commit.</span></li>
<li><span class="tick">&#8226;</span><span>Exercised by <code>email-hero health</code> (check #8, every 600s) + the <code>com.emailhero.watch</code> daemon.</span></li>
</ul>
</div>
<div class="block">
<h3>Definition of Done</h3>
<ul class="dod">
<!-- FILL: DOD. Use ☐ / ☑ glyphs. -->
<li><span class="box">☐</span><span>Tests written and green</span></li>
<li><span class="box">☐</span><span>Independent local review clear (<code>/eng:cr</code>) + CodeRabbit addressed</span></li>
<li><span class="box">☐</span><span>Docs updated where user-facing</span></li>
<li><span class="box">☐</span><span><code>where-things-run.json</code> bumped if the deploy changed hosts</span></li>
<ul class="proof">
<!-- FILL: DOD. Plain bullets, no checkbox glyphs (checkbox state lives in the PR body). -->
<li><span class="tick">&#8226;</span><span>Tests written and green</span></li>
<li><span class="tick">&#8226;</span><span>Independent local review clear (<code>/eng:cr</code>) + CodeRabbit addressed</span></li>
<li><span class="tick">&#8226;</span><span>Docs updated where user-facing</span></li>
<li><span class="tick">&#8226;</span><span><code>where-things-run.json</code> bumped if the deploy changed hosts</span></li>
</ul>
</div>
</div>
Expand Down
Loading