DEVOP-573: author 2026 Q1 tabletop scenario doc (eliza-allora-plugin)

[srt0422]

Summary

Documents the inaugural Shai-Hulud-class tabletop exercise: an injected "eliza-allora-plugin was published with a postinstall payload yesterday at 4pm" scenario that walks the team end-to-end through SECURITY-RUNBOOK.md (DEVOP-571).

The doc itself is an operational artifact — it's the script the facilitator runs on the day of the exercise. The exercise is a team activity and is NOT considered complete until the run + debrief have actually happened.

What's in the doc

• Pre-exercise setup checklist (facilitator).
• The injected scenario with specific exfil mechanics, IOC discovery timeline, and T+0 trigger.
• Pre-assigned roles (incident lead, communicator, executor, BE rep, FE rep, founder observer-only) with explicit "if someone is missing, postpone" rule.
• Six phases keyed to runbook sections, each with a target elapsed time and explicit success/failure modes.
• The 30-minute time-to-clean-republish target broken into 4 milestones (T+5 / T+10 / T+20 / T+30).
• Debrief script (6 questions, verbatim) that produces ticket inputs from the team's own language.
• Output checklist (Linear tickets, runbook PR, lessons-learned update, next-year calendar invite).
• Notes-from-runbook-author identifying the three seams the exercise should specifically stress.

Why this PR is opened now (before DEVOP-571 merges)

Per the project plan: "the actual exercise requires the team — note that on the ticket." This PR satisfies the authoring requirement so the facilitator has a complete artifact to schedule against. The scenario doc cross-references the runbook by relative path; once both PRs merge the links resolve correctly.

Linear

https://linear.app/alloralabs/issue/DEVOP-573

Status note

This ticket stays in In Review after the doc is merged. It moves to Done only after the live exercise has run and the debrief outputs (Linear tickets, runbook PR) have been filed.

Test plan

• Facilitator schedules a 90-minute slot in 2026 Q1 within 2 weeks of doc landing.
• Pre-exercise setup checklist walked once dry as a smoke test (Slack channel creation, role confirmations).
• After the live run, "Lessons learned" section updated in a follow-up PR.

🤖 Generated with Claude Code

---

Summary by cubic

Adds the 2026 Q1 tabletop exercise script for a poisoned publish of eliza-allora-plugin, aligned to SECURITY-RUNBOOK.md with a <30‑minute clean‑republish target. Meets DEVOP-573 by mapping roles, phases, and outputs to the runbook.

• New Features

  • Scenario doc at tabletop/2026-Q1-shai-hulud-eliza.md with exfil mechanics, IOC timeline, and T+0 trigger.
  • Wall-clock layout and a single sim‑vs‑real rule for execution clarity.
  • Debrief script that produces Linear tickets, plus an output checklist.

• Bug Fixes

  • Authority and phase corrections: founder observer is silent; incident lead owns per-version npm unpublish; full-package delete gate defined once in §3 (publisher + on-call agreement in #security-alerts); removed dead-letter PyPI step; moved blast‑radius work to Phase 3.
  • Clarity and consistency: kept the injected-scenario blockquote in-character; replaced the exfil URL with an RFC‑reserved placeholder; clarified post‑mortem timeline math; fixed references to Runbook §5 Scenario C step 1; used the §3 role name “founder observer” in setup; aligned the duration parenthetical with the wall‑clock layout; prefixed §8 section references with “Runbook”; noted the runbook link resolves after DEVOP‑571 merges.

^{Written for commit fd6a03b. Summary will update on new commits.}

[cubic-dev-ai]

cubic analysis

1 issue found across 1 file

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="tabletop/2026-Q1-shai-hulud-eliza.md">

<violation number="1" location="tabletop/2026-Q1-shai-hulud-eliza.md:63">
P2: The founder role is defined as non-participating, but Phase 2 requires founder buy-in for the unpublish decision. Clarify the role so the success criteria are achievable without contradiction.</violation>
</file>

Linked issue analysis

Linked issue: DEVOP-573: Tabletop exercise: simulate eliza-allora-plugin poisoned publish

Status	Acceptance criteria	Notes
✅	Scenario: assume `eliza-allora-plugin@` was published with a postinstall payload yesterday at 4pm.	The PR includes a detailed injected scenario with the 04:00 PM yesterday timestamp, exfil steps, IOC timeline, and T+0 trigger.
✅	Roles assigned: incident lead, communicator, executor (plus reps/observer).	The doc contains a roles table with assigned roles and explicit instructions for each role.
✅	Walk through every runbook section: detection → triage → containment → rotation → republish → post-mortem.	The PR defines six timed phases keyed to the runbook sections and describes success/failure modes for each phase.
✅	Time-to-clean-republish target: <30 min (with milestones T+5 / T+10 / T+20 / T+30).	The document explicitly specifies the 30-minute target and breaks it into the stated milestones and phase targets.
✅	Debrief script that produces ticket inputs (so gaps found can be documented later).	The PR includes a debrief section with six verbatim questions and instructions to turn answers into Linear tickets.

Architecture diagram

sequenceDiagram
    participant Dev as DevOps Engineer
    participant GH as GitHub
    participant Runner as GHA Runner
    participant NPM as npm Registry
    participant CF as CloudFlare Worker
    participant Socket as Socket.dev
    participant Sweep as Daily IOC Sweep
    participant Alert as #security-alerts
    participant Slack as #tabletop-2026-q1
    participant Facil as Facilitator
    participant Obs as Founder Observer

    Note over Dev,Obs: DEVOP-573 Tabletop Exercise - Injected Scenario Flow

    alt T-16 hours: Poisoned Publish
        Dev->>GH: Push code to allora-network/eliza-allora-plugin
        GH->>Runner: Trigger release workflow
        Runner->>Runner: Publish eliza-allora-plugin@<latest>
        Runner->>NPM: npm publish
        NPM-->>Dev: Published version
        Note over NPM: Tarball includes postinstall payload
        NPM->>Runner: Install triggers postinstall
        Runner->>CF: Exfil ~/.npmrc _authToken
        Runner->>CF: Exfil .git/config credentials
        CF-->>Runner: Confirmation
        Runner->>NPM: Republish with bumped version <latest>.1
    end

    Note over Socket,Alert: T-13.5 hours: Detection

    Socket->>Socket: Scan npm for malicious packages
    Socket->>Socket: Flag eliza-allora-plugin@<latest> & @<latest>.1
    Socket-->>Sweep: Advisory feed

    Note over Sweep,Alert: T+0: Exercise Start

    Sweep->>Sweep: Org-wide IOC sweep workflow
    Sweep->>Alert: Open incident-response ticket
    Alert->>Alert: Post alert in #security-alerts

    Facil->>Slack: Paste the injected scenario alert (T+0)
    Note over Facil: Exercise clock starts on first "ack"

    alt Phase 1: Detection + Triage (T+5 target)
        Dev->>Slack: Acknowledge alert (ack)
        Dev->>Dev: Walk triage decision tree
        Dev->>Dev: IOC match → did we publish? → yes → Scenario C
        Dev->>Slack: Open timeline thread
        Note over Dev,Slack: Success: Scenario C identified within 5 minutes
    else Failure: Skip IOC cross-check
        Dev->>Dev: Assume worst without verification
    end

    alt Phase 2: Stop the Bleed (T+10 target)
        Dev->>Dev: Instruct executor to deprecate npm versions
        Dev->>NPM: npm deprecate (simulated)
        Dev->>Dev: Decide on unpublish (consult founder)
        Dev->>Dev: Search for consumer repos via gh
    else Failure: Try to delete package entirely
        Dev->>NPM: Attempt full deletion (runbook violation)
    end

    alt Phase 3: Audit Blast Radius (T+20 target)
        Dev->>Dev: List secrets the publish workflow could read
        Dev->>Dev: gh search for consumer repos
        Dev->>Dev: Draft GitHub security advisory
        Dev->>Dev: Draft downstream-consumer notification
        Dev->>Dev: Backend/Frontend reps review notification
    else Failure: Rotate before listing secrets
        Dev->>Dev: Skip secret discovery step
    end

    alt Phase 4: Clean Republish (T+30 target)
        Dev->>GH: Use clean GHA-hosted runner (not local)
        Dev->>Dev: Read release.yml workflow
        Dev->>Dev: Cut fresh minor bump tag
        Dev->>GH: Trigger release workflow
        GH->>Runner: Run clean publish
        Runner->>NPM: Publish clean version (simulated)
        Dev->>Dev: Send advisory + notification (simulated)
    else Failure: Use local machine
        Dev->>Dev: Attempt republish from local (worst failure mode)
    end

    alt Phase 5: Token Rotation (post-clock)
        Dev->>Dev: Walk rotation list from Phase 3
        Dev->>Dev: Note OIDC migration opportunities
    end

    alt Phase 6: Post-Mortem
        Dev->>Dev: Draft timeline from channel transcript
        Dev->>Dev: Identify 16-hour detection blind spot
        Dev->>Dev: File action items as Linear tickets
    end

    Note over Facil,Obs: Debrief (30 minutes after clock stops)

    Facil->>Slack: Run 6 debrief questions
    Slak->>Facil: Team provides verbatim responses
    Facil->>Facil: Convert responses to ticket descriptions
    Facil->>Facil: Generate output checklist (Linear, runbook PR, lessons learned)

Loading

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.}

[srt0422]

Needs-human: Phase 5 / debrief personnel overlap

Phase 5 runs in parallel with the debrief (wall-clock: T+30 → T+60) and this checklist assigns the rotation walk to the lead, while the §5 debrief assumes the lead's participation — debrief question 2 asks about steps where "the lead and executor had to negotiate what was meant." The doc does not state who attends the debrief while the rotation walk is running.

Two resolutions, depending on the intended exercise design:

1. Lead + communicator peel off for the rotation walk and rejoin the debrief once the rotation list is exhausted (the wall-clock already notes Phase 5 "concludes whenever the rotation list is exhausted") — state that allocation explicitly here.
2. The lead delegates the rotation walk (e.g. to the executor) and attends the full debrief.

Which allocation matches the intended design? I can apply the clarifying note once confirmed.

[srt0422]

Final cross-provider review (agent-code-review, 3-pass)
verdict: Ready with fixes
counts: P0=0 P1=0 P2=2 P3=1
coverage: Pass 1 — correctness+security: kimi-k3 (orchestrator inline, user-mandated substitute for Claude Opus 4.7); style: z-ai/glm-5.2 (substitute for Claude Sonnet 5). Pass 2 — correctness: google/gemini-2.5-pro; security: deepseek/deepseek-v4-pro (NO FINDINGS); style: meta-llama/llama-4-maverick. Pass 3 — correctness: meta-llama/llama-4-maverick; security: z-ai/glm-5.2 (substitute for Claude Sonnet 5); style: meta-llama/llama-4-scout. Anthropic tier was replaced per user directive (no claude-code dispatch). All Pass 2/3 findings source-verified by the orchestrator.

severity	file:line	category	reviewer	summary	suggested_fix
P2	tabletop/2026-Q1-shai-hulud-eliza.md:3	correctness	persona	The doc schedules the exercise for "2026 Q1" (line 3) while its own status line is dated 2026-05-13 (line 330), after Q1 2026 ends — the facilitator cannot tell whether the quarter or the date is authoritative. (Confirmed by Gemini 2.5 Pro + all three Pass 3 providers.)	Reconcile the quarter reference with the status date — either the exercise targets the next Q1 (2027 Q1, matching the annual cadence in §6 and debrief q6) or the status date/quarter is stale; this turns on ticket scope for DEVOP-573.
P2	tabletop/2026-Q1-shai-hulud-eliza.md:10	correctness	persona	Duration allocates "30 min Phase 6 post-mortem" inside the 90-minute slot (lines 10–11), but the wall-clock starts Phase 6 "after T+60" AND "once Phase 5 is done" while Phase 5 "concludes whenever the rotation list is exhausted" (lines 99–102) — a long rotation list pushes Phase 6 past T+90 and overruns the slot. (Confirmed by Gemini 2.5 Pro + all three Pass 3 providers.)	Either cap the Phase 5 rotation walk at T+60 (deferring remaining tokens to follow-up tickets) or state explicitly that Phase 6 slips when the rotation list runs long.
P3	tabletop/2026-Q1-shai-hulud-eliza.md:13	style	persona	Link text for the same runbook target differs between line 13 (`SECURITY-RUNBOOK.md` in backticks, with extension) and line 89 (SECURITY-RUNBOOK, plain, no extension); a reader could wonder whether the two links point at different documents. (GLM 5.2, source-verified.)	Standardize both link texts to [`SECURITY-RUNBOOK.md`](../SECURITY-RUNBOOK.md).

Refuted (source-verified false positives, not findings):

• "publisher" not defined in the §3 role table (Gemini, P1; Llama Scout re-raised as P0): refuted by all three Pass 3 providers and the Pass 1 review — the authority paragraph describes the runbook's gate using runbook vocabulary, and Phase 2 states the gate must not fire in this exercise, so the role is never simulated.
• Llama Maverick style pass (4 findings): voice mixing (line 234 area — second person appears only in facilitator asides, a distinct textual function), § symbol inconsistency (usage is consistent: bare for internal sections, Runbook §N for runbook refs), lines exceeding 72 cols (the long lines are Markdown table rows), <latest> vs @<latest> placeholder inconsistency (@<latest> is npm package-spec syntax, intentional).
• "Communicator files DEVOP-578 follow-up" role inconsistency (Gemini, P3): filing a comms-adjacent follow-up ticket is within the communicator role's spirit; no concrete consequence.