diff --git a/docs/dd-338-marketplace-refresh.md b/docs/dd-338-marketplace-refresh.md new file mode 100644 index 0000000..e56a7c3 --- /dev/null +++ b/docs/dd-338-marketplace-refresh.md @@ -0,0 +1,126 @@ +# DD-338 Marketplace Refresh Runbook + +After running `stallari-conformance verify --live` (Phase D.2) and +`tier_issuance.py --apply` (Phase D.3) to flip tool declarations, the +marketplace tile state is refreshed via this five-step chain. + +The script `scripts/dd-338-refresh-catalog.mjs` is a thin orchestrator: it +wraps the existing `scripts/build-catalog.js`, snapshots the prior +`dist/catalog.json`, regenerates the catalog, computes a structured diff, +and emits `dist/dd-338-refresh-summary.json` for review. + +## Chain + +### 1. Regenerate the catalog + +From the `stallari-plugins/` repo root: + +```bash +node scripts/dd-338-refresh-catalog.mjs +``` + +This will: + +1. Move any existing `dist/catalog.json` to `dist/catalog.json.prev` (the + diff baseline). +2. Run `node scripts/build-catalog.js`, which regenerates + `dist/catalog.json`, `dist/services.json`, `dist/add-ons.json`, + `dist/pack-details.json`, and per-pack manifests under + `dist/packs///manifest.json`. AJV schema validation + (DD-333 Phase A.1) + cross-field gates (DD-333 F.1 + F.4) run as part + of `build-catalog.js`. If anything fails the chain exits non-zero and + the prior catalog snapshot is preserved at `dist/catalog.json.prev`. +3. Compute a structured diff between prev and new and write + `dist/dd-338-refresh-summary.json`. +4. Print a brief human-readable summary to stdout. + +**First-run behaviour:** when there is no prior `dist/catalog.json` the +script treats the run as a baseline. The summary reports every entry as +an addition and notes that no diff was computed. + +### 2. Review the diff + +Open `dist/dd-338-refresh-summary.json`. The shape is: + +```json +{ + "generated": "2026-05-24T03:14:15.000Z", + "dd": "DD-338", + "phase": "D.5", + "catalog_meta": { "version": "1.1.0", "generated": "2026-05-24", "total": 73, "plugins": 65, "packs": 8 }, + "diff": { + "baseline": false, + "added": ["..."], + "removed": ["..."], + "changed": [ + { + "name": "cloudflare-blade", + "type": "plugin", + "tier_change": null, + "readiness_change": null, + "tool_changes": [ + { "tool": "cf_d1_query", "field": "deterministic_ordering", "before": "unstable", "after": "stable" } + ] + } + ], + "summary": { "added": 0, "removed": 0, "changed": 1, "tool_flips_total": 1 } + } +} +``` + +Per-tool granularity flips should match the proposals D.3 generated. If +they do not, investigate — something edited `plugins/tools/*.json` between +the D.3 `--apply` and this refresh. + +Diff axes tracked per tool: `scope_filtering`, `field_projection`, +`deterministic_ordering`, `audit_surface`, `domain_scope`, plus +`presence` (tools added or removed from a catalog entry). Entry-level +changes track `tier` and `readiness`. + +### 3. Commit and push + +The regenerated `dist/catalog.json` is **gitignored** — it is rebuilt by +CI on every push to `main`. What you commit is the source change in +`plugins/tools/*.json` (the D.3 `--apply` output). The downstream +`stallari-registry-infra` deploy workflow picks up the catalog on next +push to `main`, or you can manually `wrangler deploy` from the +registry-infra repo. + +If `dist/dd-338-refresh-summary.json` is useful as a review artefact for +a PR description, paste the `diff.summary` block + `changed[]` rows into +the PR body — do **not** commit the summary file itself (it sits inside +the gitignored `dist/`). + +### 4. Daemon refresh + +Running Stallari daemons poll the registry HTTP endpoint at a +configurable interval (default 5 min per `RegistryClient` config). +Force-refresh via the daemon CLI: + +```bash +stallari-cli registry refresh +``` + +### 5. Verify + +Restart the Stallari app OR wait for the next refresh tick. Marketplace +tile badges should reflect the new tier state. Cross-check a known +flipped tool against the in-app marketplace surface and confirm the +declared granularity row matches the summary. + +## Failure modes + +| Symptom | Cause | Resolution | +|---|---|---| +| `scripts/build-catalog.js exited with code 1` | Schema validation or cross-field gate failed | Stderr from build-catalog includes the offending entry + field; fix the source `plugins/tools/.json` | +| Empty diff after D.3 `--apply` | Source files not actually changed | Re-run `tier_issuance.py --apply` and confirm git status shows mutations in `plugins/tools/` | +| Summary missing | Build-catalog failed mid-run | `dist/catalog.json.prev` is preserved as the baseline; rerun the wrapper after fixing the source | +| Daemon still shows old badges after `stallari-cli registry refresh` | Cloudflare Worker cache or R2 not yet redeployed | Check the `stallari-registry-infra` deploy workflow; the catalog is fronted by a CF Worker that has its own deploy cadence | + +## Related + +- [[DD-338]] Phase D § "D.5 marketplace tile state refresh" +- Phase D.2 — `stallari-conformance verify --live` +- Phase D.3 — `tier_issuance.py --apply` +- `scripts/build-catalog.js` — the existing catalog builder this script wraps +- `stallari-registry-infra/` — out-of-scope cross-repo deploy pipeline that ultimately serves `dist/catalog.json` to running daemons diff --git a/scripts/dd-338-refresh-catalog.mjs b/scripts/dd-338-refresh-catalog.mjs new file mode 100644 index 0000000..0105906 --- /dev/null +++ b/scripts/dd-338-refresh-catalog.mjs @@ -0,0 +1,316 @@ +#!/usr/bin/env node +/** + * DD-338 Phase D.5 — marketplace tile state refresh wrapper. + * + * Thin orchestrator around the existing `scripts/build-catalog.js`. Chains: + * + * 1. Snapshot existing dist/catalog.json → dist/catalog.json.prev (if any) + * 2. Run `node scripts/build-catalog.js` (regenerates dist/catalog.json + * + per-pack manifests + services.json + add-ons.json) + * 3. Diff prev vs new catalog (per-tool granularity declaration flips + + * per-entry tier/readiness changes + add/remove counts) + * 4. Write dist/dd-338-refresh-summary.json (structured) + * 5. Print brief human-readable summary to stdout + * + * On first run (no prior catalog) emits a baseline summary and exits 0. + * On build-catalog failure, exits non-zero and propagates stderr. + * + * Usage: + * node scripts/dd-338-refresh-catalog.mjs + * + * Operational runbook: docs/dd-338-marketplace-refresh.md + */ + +import { spawn } from "node:child_process"; +import { readFile, writeFile, mkdir, rename, stat } from "node:fs/promises"; +import { join, resolve } from "node:path"; + +const ROOT = resolve(import.meta.dirname, ".."); +const DIST_DIR = join(ROOT, "dist"); +const CATALOG_PATH = join(DIST_DIR, "catalog.json"); +const PREV_PATH = join(DIST_DIR, "catalog.json.prev"); +const SUMMARY_PATH = join(DIST_DIR, "dd-338-refresh-summary.json"); +const BUILD_SCRIPT = join(ROOT, "scripts", "build-catalog.js"); + +/** Test whether a filesystem path exists (file or dir). */ +async function pathExists(p) { + try { + await stat(p); + return true; + } catch { + return false; + } +} + +/** + * Compute a structured diff between a prior catalog and a new catalog. + * + * Diff shape: + * { + * baseline: bool, // true when no prior catalog existed + * added: [name, ...], // entries present in new but not prev + * removed: [name, ...], // entries present in prev but not new + * changed: [{ name, type, tier_change, readiness_change, tool_changes }], + * summary: { added, removed, changed, tool_flips_total } + * } + * + * `tool_changes` is a list of `{ tool, field, before, after }` rows for any + * tools[].granularity declaration flip OR tools[] add/remove inside an entry + * present in both catalogs. The four fields tracked are the granularity axes: + * scope_filtering, field_projection, deterministic_ordering, audit_surface, + * plus domain_scope when present (DD-333 F.4). + * + * Exported for testing. + */ +export function diffCatalogs(prevCatalog, newCatalog) { + const granularityAxes = [ + "scope_filtering", + "field_projection", + "deterministic_ordering", + "audit_surface", + "domain_scope", + ]; + + // First-run baseline: no prior catalog. Every entry is a fresh add. + if (prevCatalog === null) { + const data = Array.isArray(newCatalog?.data) ? newCatalog.data : []; + return { + baseline: true, + added: data.map((e) => e.name).sort(), + removed: [], + changed: [], + summary: { + added: data.length, + removed: 0, + changed: 0, + tool_flips_total: 0, + }, + }; + } + + const prevData = Array.isArray(prevCatalog?.data) ? prevCatalog.data : []; + const newData = Array.isArray(newCatalog?.data) ? newCatalog.data : []; + + const prevByName = new Map(prevData.map((e) => [e.name, e])); + const newByName = new Map(newData.map((e) => [e.name, e])); + + const added = []; + const removed = []; + const changed = []; + let toolFlipsTotal = 0; + + for (const [name, _entry] of newByName) { + if (!prevByName.has(name)) added.push(name); + } + for (const [name, _entry] of prevByName) { + if (!newByName.has(name)) removed.push(name); + } + + for (const [name, newEntry] of newByName) { + const prevEntry = prevByName.get(name); + if (!prevEntry) continue; + + const tierChange = + prevEntry.tier !== newEntry.tier + ? { before: prevEntry.tier ?? null, after: newEntry.tier ?? null } + : null; + const readinessChange = + prevEntry.readiness !== newEntry.readiness + ? { + before: prevEntry.readiness ?? null, + after: newEntry.readiness ?? null, + } + : null; + + // Per-tool granularity diff. Tools live on tools[]; not all catalog + // entries declare them (packs don't), so default to empty arrays. + const prevTools = new Map( + (Array.isArray(prevEntry.tools) ? prevEntry.tools : []) + .filter((t) => t && typeof t.name === "string") + .map((t) => [t.name, t]), + ); + const newTools = new Map( + (Array.isArray(newEntry.tools) ? newEntry.tools : []) + .filter((t) => t && typeof t.name === "string") + .map((t) => [t.name, t]), + ); + + const toolChanges = []; + + for (const [toolName, newTool] of newTools) { + const prevTool = prevTools.get(toolName); + if (!prevTool) { + toolChanges.push({ + tool: toolName, + field: "presence", + before: null, + after: "added", + }); + toolFlipsTotal += 1; + continue; + } + const prevGran = prevTool.granularity || {}; + const newGran = newTool.granularity || {}; + for (const axis of granularityAxes) { + const before = prevGran[axis] ?? null; + const after = newGran[axis] ?? null; + if (before !== after) { + toolChanges.push({ tool: toolName, field: axis, before, after }); + toolFlipsTotal += 1; + } + } + } + + for (const [toolName, _prevTool] of prevTools) { + if (!newTools.has(toolName)) { + toolChanges.push({ + tool: toolName, + field: "presence", + before: "present", + after: "removed", + }); + toolFlipsTotal += 1; + } + } + + if (tierChange || readinessChange || toolChanges.length > 0) { + changed.push({ + name, + type: newEntry.type ?? null, + tier_change: tierChange, + readiness_change: readinessChange, + tool_changes: toolChanges, + }); + } + } + + added.sort(); + removed.sort(); + changed.sort((a, b) => a.name.localeCompare(b.name)); + + return { + baseline: false, + added, + removed, + changed, + summary: { + added: added.length, + removed: removed.length, + changed: changed.length, + tool_flips_total: toolFlipsTotal, + }, + }; +} + +/** Render a brief human-readable summary string from a diff object. */ +export function formatSummary(diff, { generatedAt } = {}) { + const lines = []; + if (diff.baseline) { + lines.push( + `Baseline run — no prior catalog. ${diff.summary.added} entries indexed as initial state.`, + ); + } else { + const { added, removed, changed, tool_flips_total } = diff.summary; + if (added === 0 && removed === 0 && changed === 0) { + lines.push("No catalog changes detected."); + } else { + lines.push( + `Catalog diff: +${added} added, -${removed} removed, ~${changed} changed (${tool_flips_total} tool-level flip(s)).`, + ); + for (const name of diff.added) lines.push(` + ${name}`); + for (const name of diff.removed) lines.push(` - ${name}`); + for (const entry of diff.changed) { + const bits = []; + if (entry.tier_change) + bits.push(`tier ${entry.tier_change.before}→${entry.tier_change.after}`); + if (entry.readiness_change) + bits.push( + `readiness ${entry.readiness_change.before}→${entry.readiness_change.after}`, + ); + if (entry.tool_changes.length > 0) + bits.push(`${entry.tool_changes.length} tool flip(s)`); + lines.push(` ~ ${entry.name}: ${bits.join(", ")}`); + } + } + } + if (generatedAt) lines.push(`Generated: ${generatedAt}`); + lines.push(`Summary written: ${SUMMARY_PATH}`); + return lines.join("\n"); +} + +/** Run `node scripts/build-catalog.js` as a child process, inheriting stdio. */ +function runBuildCatalog() { + return new Promise((resolveBuild, rejectBuild) => { + const child = spawn(process.execPath, [BUILD_SCRIPT], { + cwd: ROOT, + stdio: "inherit", + }); + child.on("error", rejectBuild); + child.on("close", (code) => { + if (code === 0) resolveBuild(); + else + rejectBuild( + new Error(`scripts/build-catalog.js exited with code ${code}`), + ); + }); + }); +} + +async function readCatalogIfExists(path) { + if (!(await pathExists(path))) return null; + const raw = await readFile(path, "utf-8"); + return JSON.parse(raw); +} + +async function main() { + await mkdir(DIST_DIR, { recursive: true }); + + // Step 1 — snapshot existing catalog (if any). We move-rename rather than + // copy so the next build-catalog run produces a clean new file. The prev + // file is what we diff against; the new file is the post-build artefact. + const hadPrev = await pathExists(CATALOG_PATH); + if (hadPrev) { + await rename(CATALOG_PATH, PREV_PATH); + } + + // Step 2 — regenerate. build-catalog.js handles its own logging. + await runBuildCatalog(); + + // Step 3 — read both catalogs and diff. prev may not exist on first run. + const prevCatalog = await readCatalogIfExists(PREV_PATH); + const newCatalog = await readCatalogIfExists(CATALOG_PATH); + if (!newCatalog) { + throw new Error( + `build-catalog.js completed but ${CATALOG_PATH} is missing. Cannot diff.`, + ); + } + + const diff = diffCatalogs(prevCatalog, newCatalog); + + // Step 4 — emit summary JSON. + const generatedAt = new Date().toISOString(); + const summary = { + generated: generatedAt, + dd: "DD-338", + phase: "D.5", + catalog_meta: newCatalog.meta ?? null, + diff, + }; + await writeFile(SUMMARY_PATH, JSON.stringify(summary, null, 2) + "\n"); + + // Step 5 — human readable. + console.log(""); + console.log("--- DD-338 D.5 refresh summary ---"); + console.log(formatSummary(diff, { generatedAt })); +} + +// Run main() only when executed directly (not imported as a module). +const isMain = + process.argv[1] && import.meta.url === `file://${process.argv[1]}`; + +if (isMain) { + main().catch((err) => { + console.error(err.stack || err.message || String(err)); + process.exit(1); + }); +} diff --git a/scripts/dd-338-refresh-catalog.test.js b/scripts/dd-338-refresh-catalog.test.js new file mode 100644 index 0000000..1657903 --- /dev/null +++ b/scripts/dd-338-refresh-catalog.test.js @@ -0,0 +1,278 @@ +import { describe, it } from "node:test"; +import assert from "node:assert/strict"; + +import { diffCatalogs, formatSummary } from "./dd-338-refresh-catalog.mjs"; + +// --------------------------------------------------------------------------- +// diffCatalogs — baseline (no prior catalog) +// --------------------------------------------------------------------------- + +describe("diffCatalogs — first-run baseline", () => { + it("treats every entry as an add when prev is null", () => { + const newCatalog = { + meta: { total: 2 }, + data: [ + { name: "alpha", type: "plugin", tier: "certified" }, + { name: "beta", type: "pack", tier: "community" }, + ], + }; + const diff = diffCatalogs(null, newCatalog); + assert.equal(diff.baseline, true); + assert.deepEqual(diff.added, ["alpha", "beta"]); + assert.deepEqual(diff.removed, []); + assert.deepEqual(diff.changed, []); + assert.equal(diff.summary.added, 2); + assert.equal(diff.summary.removed, 0); + assert.equal(diff.summary.changed, 0); + assert.equal(diff.summary.tool_flips_total, 0); + }); + + it("handles empty new catalog on baseline run", () => { + const diff = diffCatalogs(null, { data: [] }); + assert.equal(diff.baseline, true); + assert.deepEqual(diff.added, []); + assert.equal(diff.summary.added, 0); + }); +}); + +// --------------------------------------------------------------------------- +// diffCatalogs — identical catalogs (empty diff) +// --------------------------------------------------------------------------- + +describe("diffCatalogs — identical catalogs", () => { + it("returns zero deltas when prev and new are deep-equal", () => { + const catalog = { + meta: { total: 1 }, + data: [ + { + name: "alpha", + type: "plugin", + tier: "certified", + readiness: "production", + tools: [ + { + name: "alpha_list", + granularity: { + scope_filtering: "server-side", + field_projection: "explicit", + deterministic_ordering: "stable", + audit_surface: "complete", + }, + }, + ], + }, + ], + }; + // Clone via JSON to make sure object identity doesn't fool the diff. + const prev = JSON.parse(JSON.stringify(catalog)); + const next = JSON.parse(JSON.stringify(catalog)); + const diff = diffCatalogs(prev, next); + assert.equal(diff.baseline, false); + assert.deepEqual(diff.added, []); + assert.deepEqual(diff.removed, []); + assert.deepEqual(diff.changed, []); + assert.equal(diff.summary.tool_flips_total, 0); + }); +}); + +// --------------------------------------------------------------------------- +// diffCatalogs — single tool granularity flip +// --------------------------------------------------------------------------- + +describe("diffCatalogs — single tool flip", () => { + it("reports one tool change when deterministic_ordering flips", () => { + const prev = { + data: [ + { + name: "alpha", + type: "plugin", + tier: "certified", + tools: [ + { + name: "alpha_query", + granularity: { + scope_filtering: "server-side", + field_projection: "explicit", + deterministic_ordering: "unstable", + audit_surface: "complete", + }, + }, + ], + }, + ], + }; + const next = { + data: [ + { + name: "alpha", + type: "plugin", + tier: "certified", + tools: [ + { + name: "alpha_query", + granularity: { + scope_filtering: "server-side", + field_projection: "explicit", + deterministic_ordering: "stable", + audit_surface: "complete", + }, + }, + ], + }, + ], + }; + const diff = diffCatalogs(prev, next); + assert.equal(diff.baseline, false); + assert.deepEqual(diff.added, []); + assert.deepEqual(diff.removed, []); + assert.equal(diff.changed.length, 1); + assert.equal(diff.changed[0].name, "alpha"); + assert.equal(diff.changed[0].tool_changes.length, 1); + assert.deepEqual(diff.changed[0].tool_changes[0], { + tool: "alpha_query", + field: "deterministic_ordering", + before: "unstable", + after: "stable", + }); + assert.equal(diff.summary.tool_flips_total, 1); + assert.equal(diff.changed[0].tier_change, null); + assert.equal(diff.changed[0].readiness_change, null); + }); + + it("reports tier change when entry tier flips", () => { + const prev = { + data: [{ name: "beta", type: "plugin", tier: "community" }], + }; + const next = { + data: [{ name: "beta", type: "plugin", tier: "verified" }], + }; + const diff = diffCatalogs(prev, next); + assert.equal(diff.changed.length, 1); + assert.deepEqual(diff.changed[0].tier_change, { + before: "community", + after: "verified", + }); + }); +}); + +// --------------------------------------------------------------------------- +// diffCatalogs — adds, removes, mixed deltas +// --------------------------------------------------------------------------- + +describe("diffCatalogs — adds and removes", () => { + it("captures added and removed entries by name", () => { + const prev = { + data: [ + { name: "alpha", type: "plugin" }, + { name: "beta", type: "plugin" }, + ], + }; + const next = { + data: [ + { name: "alpha", type: "plugin" }, + { name: "gamma", type: "pack" }, + ], + }; + const diff = diffCatalogs(prev, next); + assert.deepEqual(diff.added, ["gamma"]); + assert.deepEqual(diff.removed, ["beta"]); + assert.deepEqual(diff.changed, []); + }); + + it("captures tool add and tool remove within the same entry", () => { + const prev = { + data: [ + { + name: "alpha", + type: "plugin", + tools: [{ name: "tool_a" }, { name: "tool_b" }], + }, + ], + }; + const next = { + data: [ + { + name: "alpha", + type: "plugin", + tools: [{ name: "tool_a" }, { name: "tool_c" }], + }, + ], + }; + const diff = diffCatalogs(prev, next); + assert.equal(diff.changed.length, 1); + const flips = diff.changed[0].tool_changes; + assert.equal(flips.length, 2); + const added = flips.find((f) => f.tool === "tool_c"); + const removed = flips.find((f) => f.tool === "tool_b"); + assert.deepEqual(added, { + tool: "tool_c", + field: "presence", + before: null, + after: "added", + }); + assert.deepEqual(removed, { + tool: "tool_b", + field: "presence", + before: "present", + after: "removed", + }); + assert.equal(diff.summary.tool_flips_total, 2); + }); +}); + +// --------------------------------------------------------------------------- +// formatSummary — human-readable rendering +// --------------------------------------------------------------------------- + +describe("formatSummary", () => { + it("renders baseline label on first run", () => { + const text = formatSummary({ + baseline: true, + added: ["a", "b"], + removed: [], + changed: [], + summary: { added: 2, removed: 0, changed: 0, tool_flips_total: 0 }, + }); + assert.match(text, /Baseline run/); + assert.match(text, /2 entries indexed/); + }); + + it("renders empty-diff label when no changes", () => { + const text = formatSummary({ + baseline: false, + added: [], + removed: [], + changed: [], + summary: { added: 0, removed: 0, changed: 0, tool_flips_total: 0 }, + }); + assert.match(text, /No catalog changes detected/); + }); + + it("renders tool flip counts when changes present", () => { + const text = formatSummary({ + baseline: false, + added: [], + removed: [], + changed: [ + { + name: "alpha", + type: "plugin", + tier_change: null, + readiness_change: null, + tool_changes: [ + { + tool: "alpha_q", + field: "deterministic_ordering", + before: "unstable", + after: "stable", + }, + ], + }, + ], + summary: { added: 0, removed: 0, changed: 1, tool_flips_total: 1 }, + }); + assert.match(text, /\+0 added, -0 removed, ~1 changed/); + assert.match(text, /1 tool-level flip/); + assert.match(text, /~ alpha:/); + }); +});