| Field | Value |
|---|---|
| Status | Canonical demo playbook |
| Last updated | 2026-08-09 |
| Product surface | Sample agent project → mutiny init → mutiny run (via OpenAI Agents SDK adapter); Hosted UI secondary for lineage |
Do not invent a different demo narrative in README or slides without updating this file.
Honesty: Live demos may use the bundled sample harness. Say so explicitly (“sample / reference agent”) — do not pitch “our vulnerable demo agent” as the product. Do not pitch Mutiny as “an OpenAI Agents SDK testing tool.”
Opening pitch: Mutiny is a behavioral fuzz-testing engine for AI agents. Adapter #1 ships with support for OpenAI Agents SDK projects.
- Sample agent project ready (OpenAI Agents SDK; or bundled demo labeled as sample)
-
mutiny init/mutiny runavailable or Hosted backup path rehearsed and labeled -
GET /api/healthgreen if showing Hosted (API, DB, model probe) - Smoke:
scripts/smoke_reliability.py→SMOKE GATE PASSED(≥2/3 on pin) when using harness - Pinned config matches
config/demo_pin.jsonwhen applicable - Browser ready for Hosted lineage (optional beat)
- Backup path verified:
scripts/backup_fixture_demo.py→BACKUP FIXTURE PATH OK - Agent “fix” patch ready (remove APR trust / enforce approval server-side)
- Authorization / safety messaging known
Nuclear options: guided/template pin; smaller N/G; Hosted against sample; fixture backup / video.
Spine: clone sample → init → run → prove → Hosted lineage → regress.
| Time | Say | Show |
|---|---|---|
| 0:00–0:15 | “Agents don’t just chat—they call tools that move money. Prompts aren’t tests.” | Sample project tree |
| 0:15–0:30 | “Mutiny is a behavioral fuzz-testing engine. You install it into your agent project.” | From-source install (uv sync) → mutiny init artifacts (PyPI planned) |
| 0:30–0:45 | “You connect via an adapter and declare invariants—e.g. refunds over $200 need approval. Adapter #1: OpenAI Agents SDK.” | .mutiny/adapter.py + policy.yaml / refund_limit |
| 0:45–1:05 | “mutiny run fuzzes your agent against those invariants.” |
Campaign progress (CLI or Hosted) |
| 1:05–1:20 | “Violation—deterministic proof from the tool call.” | issue_refund(amount=…, approved=false) + rule id |
| 1:20–1:35 | “Lineage from seed to exploit.” | Hosted graph or CLI parent trail |
| 1:35–1:50 | “Minimize, save regression. Before fix: fail. After fix: pass.” | Minimize → FAIL → patch → PASS |
| 1:50–2:00 | “Mutiny doesn’t ask if it looks safe—it proves where your policy breaks and locks the fix.” | Closing frame |
Timing checklist
- Rehearsal 1 ≤2:00
- Rehearsal 2 ≤2:00
- Rehearsal 3 ≤2:00
Use the 2-minute spine, then deepen:
- Problem (45s): tool-call failures as software defects in shipped agents.
- Install path (45s): engine + adapter layer; Adapter #1 = OpenAI Agents SDK; future adapters same Core.
- Principles (30s): AI proposes; code proves.
- Live campaign (2:00): init → run → violation.
- Hosted lineage (30s): optional.
- Related work (30s): credit Promptfoo/Garak; state wedge (engine, not framework lock-in).
- Q&A buffer.
Audience: engineers who will poke holes.
- Show adapter boundary: Core never imports the app or framework SDK; OpenAI Agents SDK adapter maps tool calls.
- Show PolicyEvaluator unit test or pure function on a fixture trace.
- Show a near-miss vs violation fitness distinction.
- Open raw ExecutionTrace JSON.
- Run minimize; show a rejected shortening that fails re-exec.
- Explain trust boundaries: local project / localhost; authorized use.
- Honest competitor answer (see §6).
- Optional: CLI
mutiny testas same Core replay path.
Avoid: claiming LangGraph/CrewAI support today; MCP setup; cloud multi-tenant claims; “the product is our demo agent”; “Mutiny is an OpenAI Agents SDK tool.”
If live search or CLI path fails:
- Play screen recording of a green campaign (same thesis).
- State clearly it is a recording; then switch live to minimize/replay if possible on a stored violator.
- If no stored violator, walk fixture trace + unit oracle + regression replay against a known failing conversation checked into
examples/. - Never paste a fake violation row.
- If only Hosted+demo harness works: call it a sample reference agent, then pivot narrative to “same engine loop on your agent project via adapter.”
| Symptom | Action |
|---|---|
| No violation by gen 3 | Guided mode / soften sample agent; restart |
mutiny init not shipped |
Show hand-authored sample artifacts; label WIP |
| Model errors | Fallback Ollama/templates; or recording |
| UI SSE stall | Refresh; rely on candidates snapshot |
| Demo too slow | Lower N/G; jump to pre-warmed campaign |
| Judge says “hardcoded” | Show lineage + near-misses + seed ≠ final |
| Judge says “just your demo agent?” | Agree sample is a harness; product = engine + adapter into their project |
| Judge says “Promptfoo?” | Agree on overlap; show search+oracle+minimize + install path |
| Judge says “only OpenAI Agents SDK?” | Agree Adapter #1 is OpenAI Agents SDK; Core is framework-neutral; more adapters on roadmap |
| Question | Answer |
|---|---|
| vs Promptfoo? | Strong overlap on policies/trajectories/CI. Mutiny productizes evolutionary search + deterministic tool oracle + minimize→regress as an installable engine. |
| vs Garak Agent Breaker? | They attack tools well. We center user invariants, deterministic acceptance, lineage, regression artifacts, local install. |
| Why no LLM judge? | Tool-arg violations are predicates; judges add flake. |
| Why only OpenAI Agents SDK? | Adapter #1 is OpenAI Agents SDK. LangGraph/CrewAI/PydanticAI/AutoGen/HTTP are Beta/v1 adapters—same Core. |
| Isn’t the sample agent weak on purpose? | Yes—like a fuzz harness. Integrity = real execution + oracle. Product is the engine testing your agent. |
| Production zero-days? | Mutiny finds breaks of defined policies; search is incomplete; false negatives expected. |
| Safety? | Authorized use; local/localhost; mock tools in demos; no open proxy. |
| Fitness without logits? | Heuristic for search only; acceptance is binary violation. |
Closing line (canonical):
Mutiny doesn’t ask whether your agent looks safe. It keeps attacking until it proves where your policy breaks—then makes sure that failure never comes back.
Do: Lead with engine + install-into-your-project; show tool JSON; show re-exec minimize; FAIL→PASS; call demo a sample/harness; name OpenAI Agents SDK as first adapter.
Don’t: Pitch bundled demo as the product; claim multi-framework support today; claim Mutiny is an OpenAI Agents SDK tool; claim novelty over Promptfoo/Garak; fake DB violations; open arbitrary URLs.