diff --git a/README.md b/README.md index d9bac72..31bb154 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,7 @@ Use it when hardcoded `if/else` logic is too rigid, but a heavyweight workflow o - AI-readable docs: [`docs/ai-coding-assistants.md`](docs/ai-coding-assistants.md), [`docs/public/llms.txt`](docs/public/llms.txt), and the official [`neuron-js` AI skill](docs/public/skills/neuron-js/SKILL.md) - Comparison and migration guides: [`docs/comparisons/`](docs/comparisons/) for json-rules-engine, JsonLogic, node-rules, and if/else migrations - Workflow automation recipes: [`docs/integrations/`](docs/integrations/) for n8n deterministic routing and LangGraph decision nodes +- Proof assets: [`docs/benchmarks/methodology.md`](docs/benchmarks/methodology.md) for the benchmark contract and [`docs/benchmarks/assets/generated/explainability-trace-diagram.md`](docs/benchmarks/assets/generated/explainability-trace-diagram.md) for the inspectable trace diagram --- @@ -40,6 +41,19 @@ Use `neuron-js` when: Do not use `neuron-js` when a simple hardcoded condition is clearer and rarely changes, when arbitrary user code execution is required, or when you need a full BPMN/process orchestration platform. +## Public proof assets + +Neuron-JS proof material is published as methodology and inspectability artifacts first. Benchmark result claims remain blocked until the benchmark harness emits real measured output with `claims_allowed: true`. + +

+ Neuron-JS diagram showing rule JSON and business input flowing through schema validation, developer registry, deterministic Synapse evaluation, result output, explanation trace, and audit-ready decision. +

+ +- Benchmark methodology and result contract: [`docs/benchmarks/methodology.md`](docs/benchmarks/methodology.md) +- Explainability proof metadata and alt text: [`docs/benchmarks/assets/generated/explainability-trace-diagram.md`](docs/benchmarks/assets/generated/explainability-trace-diagram.md) +- Visual proof system and publication guardrails: [`docs/benchmarks/visual-proof-system.md`](docs/benchmarks/visual-proof-system.md) +- Playground README demo capture spec: [`docs/playground/readme-demo-capture.md`](docs/playground/readme-demo-capture.md) + --- ## ✨ Features @@ -160,6 +174,7 @@ Available adoption assets: - Runnable examples: [`examples/`](examples/) including n8n and LangGraph workflow automation recipes - JSON Schemas, validation, and explain output: [`docs/schemas-validation-explainability.md`](docs/schemas-validation-explainability.md) +- Benchmark methodology, visual proof, and playground capture specifications: [`docs/benchmarks/`](docs/benchmarks/) and [`docs/playground/readme-demo-capture.md`](docs/playground/readme-demo-capture.md) - Comparison and migration guides: [`docs/comparisons/`](docs/comparisons/) for choosing and migrating from json-rules-engine, JsonLogic, node-rules, and hand-written if/else - AI-readable docs: [`docs/ai-coding-assistants.md`](docs/ai-coding-assistants.md), [`docs/public/llms.txt`](docs/public/llms.txt), [`docs/public/llms-full.txt`](docs/public/llms-full.txt), and [`docs/public/skills/neuron-js/SKILL.md`](docs/public/skills/neuron-js/SKILL.md) diff --git a/docs/.vitepress/config.ts b/docs/.vitepress/config.ts index 5d43ce1..e7fca94 100644 --- a/docs/.vitepress/config.ts +++ b/docs/.vitepress/config.ts @@ -49,6 +49,8 @@ export default defineConfig({ { text: 'Benchmarks & Visual Proof', link: '/benchmarks/' }, { text: 'Benchmarks', link: '/benchmarks/methodology' }, { text: 'Visual Proof System', link: '/benchmarks/visual-proof-system' }, + { text: 'Benchmark Visual Pack', link: '/benchmarks/benchmark-visual-pack' }, + { text: 'Playground Capture Spec', link: '/playground/readme-demo-capture' }, { text: 'Prompt Kit', link: '/benchmarks/prompt-kit' }, { text: 'Asset Folder', link: '/benchmarks/assets/' } ] diff --git a/docs/benchmarks/assets/README.md b/docs/benchmarks/assets/README.md index acbe572..8f0f4c3 100644 --- a/docs/benchmarks/assets/README.md +++ b/docs/benchmarks/assets/README.md @@ -16,6 +16,7 @@ docs/benchmarks/assets/ readme-proof-strip.md storyboard/ playground-readme-gif.md + ai-rule-safety-carousel.md generated/ benchmark-chart-throughput.svg benchmark-chart-cold-start.svg @@ -29,7 +30,7 @@ docs/benchmarks/assets/ ## Folder rules - `source-data/` stores benchmark input data and schemas only. Generated assets must cite these files when they include measurements. -- `prompts/` stores frozen prompts copied or split from `docs/benchmarks/prompt-kit.md` for repeatable generation. +- `prompts/` stores frozen prompts copied or split from `docs/benchmarks/prompt-kit.md` for repeatable generation. The benchmark chart prompt set lives at `prompts/benchmark-chart-prompts.md`; the methodology card prompt lives at `prompts/methodology-infographic.md`. - `storyboard/` stores GIF/clip scripts, frame lists, and capture notes. - `generated/` stores exported SVG/PNG/GIF/MP4 assets. @@ -40,6 +41,15 @@ docs/benchmarks/assets/ - Real benchmark files must include `node_version`, `package_version`, `commit_sha`, `warmup_iterations`, and `measured_iterations`. - Any generated benchmark chart must name its source data file in adjacent metadata or in the SVG metadata block. +## Generated assets + +Current generated proof assets: + +- `generated/methodology-card.svg`: 16:9 methodology card explaining no-fabricated-benchmark rules without performance numbers. +- `generated/explainability-trace-diagram.svg`: 16:9 SVG diagram showing business input -> schema validation -> registry -> rule evaluation -> match/fail trace -> deterministic decision. +- `generated/explainability-trace-diagram.html`: browser preview wrapper with caption and alt text. +- `generated/explainability-trace-diagram.md`: source grounding, caption, alt text, integrity notes, and export notes. + ## Accessibility rules Every final asset needs: diff --git a/docs/benchmarks/assets/asset-manifest.md b/docs/benchmarks/assets/asset-manifest.md new file mode 100644 index 0000000..e9c662d --- /dev/null +++ b/docs/benchmarks/assets/asset-manifest.md @@ -0,0 +1,27 @@ +# Benchmark visual asset manifest + +Status: structure-only draft prompts. Final performance charts are blocked until measured benchmark output exists. + +## Draft prompt assets + +| Asset | Path | Claim status | Source dependency | +| --- | --- | --- | --- | +| Throughput chart prompt | `prompts/benchmark-chart-prompts.md#throughput-comparison-chart` | No numeric claim | actual benchmark output required | +| Cold-start chart prompt | `prompts/benchmark-chart-prompts.md#cold-start-comparison-chart` | No numeric claim | actual benchmark output required | +| Bundle-size chart prompt | `prompts/benchmark-chart-prompts.md#bundle-size-comparison-chart` | No numeric claim | actual benchmark output required | +| Validation-overhead chart prompt | `prompts/benchmark-chart-prompts.md#validation-overhead-chart` | No numeric claim | actual benchmark output required | +| Explanation-overhead chart prompt | `prompts/benchmark-chart-prompts.md#explanation-overhead-chart` | No numeric claim | actual benchmark output required | +| Methodology card prompt | `prompts/methodology-infographic.md` | Methodology only, no performance claim | `docs/benchmarks/methodology.md` | + +## Intended generated output paths + +- `generated/benchmark-chart-throughput.svg` +- `generated/benchmark-chart-cold-start.svg` +- `generated/benchmark-chart-bundle-size.svg` +- `generated/benchmark-chart-validation-overhead.svg` +- `generated/benchmark-chart-explanation-overhead.svg` +- `generated/methodology-card.svg` + +## Publication blocker + +Do not publish generated benchmark charts until the source file satisfies `result_kind: "actual_benchmark"`, `is_placeholder: false`, and `claims_allowed: true`. diff --git a/docs/benchmarks/assets/generated/explainability-trace-diagram.html b/docs/benchmarks/assets/generated/explainability-trace-diagram.html new file mode 100644 index 0000000..4d86745 --- /dev/null +++ b/docs/benchmarks/assets/generated/explainability-trace-diagram.html @@ -0,0 +1,46 @@ + + + + + + Neuron-JS explainability trace diagram + + + +
+
+
+ Diagram showing business input and rule JSON flowing through schema validation, developer registry, deterministic Synapse evaluation, result output, match/fail explanation trace, and audit-ready decision; a dashed branch shows validation errors returning to a repair loop. +
+
Caption: Neuron-JS turns serializable business-rule JSON into deterministic decisions by validating inputs, constraining execution through a developer-owned registry, and exposing a match/fail explanation trace for review.
+

Source note: docs/schemas-validation-explainability.md and docs/benchmarks/visual-proof-system.md. No benchmark or performance claims are made.

+
+
+ + diff --git a/docs/benchmarks/assets/generated/explainability-trace-diagram.md b/docs/benchmarks/assets/generated/explainability-trace-diagram.md new file mode 100644 index 0000000..46ea3fb --- /dev/null +++ b/docs/benchmarks/assets/generated/explainability-trace-diagram.md @@ -0,0 +1,37 @@ +# Explainability trace diagram metadata + +Asset: `explainability-trace-diagram` + +Files: + +- `docs/benchmarks/assets/generated/explainability-trace-diagram.svg` +- `docs/benchmarks/assets/generated/explainability-trace-diagram.html` + +Caption: + +Neuron-JS turns serializable business-rule JSON into deterministic decisions by validating inputs, constraining execution through a developer-owned registry, and exposing a match/fail explanation trace for review. + +Alt text: + +Diagram showing business input and rule JSON flowing through schema validation, developer registry, deterministic Synapse evaluation, result output, match/fail explanation trace, and audit-ready decision; a dashed branch shows validation errors returning to a repair loop. + +Source grounding: + +- `chaos-vault/50-research/neuron-js-growth-plan.md`, NJS-GROWTH-07 lines 294-318: proof assets objective and README visual proof baseline. +- `chaos-vault/50-research/neuron-js-marketing-assets-benchmark.md`, lines 87-100 and 160-174: honest benchmark proof, playground, trace panel, README GIF pattern. +- `chaos-vault/50-research/neuron-js-social-demand-gap.md`, lines 184-210: target pains and wedge around schema validation, deterministic guardrails, and explanation traces. +- `docs/benchmarks/visual-proof-system.md`: palette, typography, diagram rules, brand guardrails, and canonical trace flow. +- `docs/benchmarks/prompt-kit.md`: explainability trace diagram prompt, labels, branches, and negative constraints. +- `docs/schemas-validation-explainability.md`: validation, deterministic execution, normalized output, and explanation trace contracts. + +Integrity notes: + +- This asset contains no benchmark numbers, winner claims, speed claims, or unsupported performance claims. +- The diagram does not show arbitrary code execution or unsafe business-user publication. +- The registry is visible as the control boundary for approved actions, conditions, parameters, and rules. + +Export notes: + +- SVG canvas: 1600 x 900, 16:9. +- The SVG is directly embeddable in docs/README and can be exported to PNG with any SVG-capable browser or renderer. +- The HTML file previews the SVG with caption and alt text. diff --git a/docs/benchmarks/assets/generated/explainability-trace-diagram.svg b/docs/benchmarks/assets/generated/explainability-trace-diagram.svg new file mode 100644 index 0000000..c3c7c7c --- /dev/null +++ b/docs/benchmarks/assets/generated/explainability-trace-diagram.svg @@ -0,0 +1,30 @@ + + From JSON rule to explainable decision + Neuron-JS inspectability diagram showing business input and serializable rule JSON passing through schema validation, developer-owned registry constraints, deterministic Synapse evaluation, result output, match/fail trace, and audit-ready decision. A dashed error branch shows validation errors returning to a repair loop. + Asset: NJS-GROWTH-07 explainability-trace-diagram. Package: @sebasoft/neuron-js. Sources: docs/schemas-validation-explainability.md, docs/benchmarks/visual-proof-system.md, docs/benchmarks/prompt-kit.md, chaos-vault NJS-GROWTH-07 evidence notes. Integrity: no benchmark or performance claims. + + + + + + + + + + From JSON rule to explainable decision@sebasoft/neuron-js: validate inputs, constrain execution vocabulary, run deterministically, then inspect why a rule matched or failed. + Integrity noteNo benchmark numbers or speed claims in this asset. + + + Business inputRule JSON + datascript.idpricing-rulescart.total125.00 + Schema validationvalidateScript()known shapeInvalid paths return$.rules[0].actions + Registrydeveloper-ownedConditionsActionsNo arbitrary code + Synapsedeterministic runsame input -> same result + Result objectnormalized outputok: truerulesExecuted: 1messages: [...]stable for tests/logs + Match/fail tracewhy it happenedscript_receivedrule_availableexecution_completed + Validation errorsexact JSON paths guide repairRepair loopfix input before execution + Control boundaryOnly approved actions, conditions, parameters, and rules can execute. + Audit-ready decisiondeterministic result + explanation tracereview, snapshot, log, or support ticket + Inspectability path:Business input -> schema validation -> registry -> rule evaluation -> match/fail trace -> deterministic decision + diff --git a/docs/benchmarks/assets/generated/methodology-card.svg b/docs/benchmarks/assets/generated/methodology-card.svg new file mode 100644 index 0000000..4afe271 --- /dev/null +++ b/docs/benchmarks/assets/generated/methodology-card.svg @@ -0,0 +1,31 @@ + + Benchmark methodology card for @sebasoft/neuron-js + Methodology infographic with no performance numbers. It explains that public claims require actual benchmark output and placeholder data is blocked. + + + + + + + + Benchmark proof without fake numbers + @sebasoft/neuron-js publishes methodology before claims. + + + Same scenarioCompare like-for-like decision tasks. + + Same input sizeName workload profile before charting. + + Warmup before timingSeparate setup from measured run. + + Real versionsRecord package and Node versions. + + Commit attachedTie every chart to source code state. + + Claims gatePublish only actual benchmark output. + + + Blocked placeholder rule + Public claims require result_kind=actual_benchmark, is_placeholder=false, claims_allowed=true. + Methodology: docs/benchmarks/methodology.md · Placeholder data is for wiring only. + diff --git a/docs/benchmarks/assets/prompts/ai-rule-safety-carousel.md b/docs/benchmarks/assets/prompts/ai-rule-safety-carousel.md new file mode 100644 index 0000000..b93cd0d --- /dev/null +++ b/docs/benchmarks/assets/prompts/ai-rule-safety-carousel.md @@ -0,0 +1,55 @@ +# AI-generated-rule safety carousel prompt + +Source storyboard: `docs/benchmarks/assets/storyboard/ai-rule-safety-carousel.md` +Source system: `docs/benchmarks/visual-proof-system.md` +Prompt kit source: `docs/benchmarks/prompt-kit.md`, section 4. + +```text +Create a 5-slide technical comic carousel for @sebasoft/neuron-js about safe AI-generated JSON business rules. + +Objective: +Explain that LLMs can draft JSON rules, but production logic needs schema validation, developer-owned registry boundaries, tests, approvals, rollback, deterministic execution, and explanation traces. The tone should be credible and developer-specific, not fearful and not generic AI hype. + +Format: +5 vertical carousel slides, 4:5 aspect ratio, designed for LinkedIn and X image threads. Keep each headline under 8 words and body text under 35 words. Include footer on every slide: "@sebasoft/neuron-js — TypeScript-first JSON business rules". + +Visual system: +Dark technical comic with safety gates. Use slate background, raised dark panels, faint grid, Inter/system UI captions, JetBrains Mono/SFMono JSON labels. Use cyan for the Neuron-JS execution path, emerald for validation/pass, violet for trace/audit, amber for pending LLM draft, rose for blocked invalid paths. Avoid mascots, magic wand imagery, robot savior imagery, generic SaaS gradients, and fearmongering visuals. + +Slide 1: +Headline: "AI can draft rules." +Body: "But a plausible JSON rule can still encode the wrong assumption, wrong field, or wrong action." +Scene: Developer inspects an amber LLM output card containing a short JSON-like rule draft. One field is subtly suspicious: customer.segment == "enterprise" routes to auto_approve_discount. Add labels "LLM draft", "Looks valid? Maybe.", and "Not production-ready." Rose warning marker only; no panic imagery. +Alt text: "A developer inspects an amber LLM-generated JSON rule card. A small rose warning marker highlights a risky business assumption, with text saying the draft is not production-ready." + +Slide 2: +Headline: "Validate before runtime." +Body: "Schema-first checks reject malformed scripts before they reach execution." +Scene: Draft rule enters an emerald-and-rose validation gate. One invalid path is blocked with exact label "blocked: conditions[0].operator". A corrected rule exits as "schema: pass". Show repair loop, not dead-end failure. +Alt text: "A JSON rule passes through a schema validation gate. One invalid path is blocked in rose with an exact JSON path, while a corrected rule exits with an emerald pass marker." + +Slide 3: +Headline: "Constrain what can run." +Body: "A developer-owned registry defines approved actions, conditions, parameters, and rules." +Scene: Validated rule reaches a registry boundary. Approved condition/action tiles are cyan and connected. An unapproved action tile, "auto_refund_all_orders", is outside the boundary and blocked by a rose dashed line. Include labels "developer registry", "approved vocabulary only", and "no arbitrary code". +Alt text: "A validated rule enters a developer-owned registry boundary. Approved action and condition tiles connect in cyan, while an unapproved action is blocked outside the boundary." + +Slide 4: +Headline: "Review like code." +Body: "Generated rules still need tests, approval, ownership, and rollback before production use." +Scene: Checklist beside the rule card: "test cases", "owner approval", "rollback ready". Approval stamp is developer/owner review, not automatic AI approval. Add small note "sample structure only". Do not show fake percentages, fake pass rates, or benchmark numbers. +Alt text: "A generated rule card sits beside a review checklist showing tests, owner approval, and rollback snapshot. The visual emphasizes governance before production use." + +Slide 5: +Headline: "Then execute deterministically." +Body: "Synapse runs the approved rule path, and the trace explains why the decision matched or failed." +Scene: Approved rule enters a cyan Synapse execution node. Output splits into a result card and violet trace rows showing input field, condition, matched/failed status, and action triggered. Final caption: "AI drafts. Neuron-JS verifies." +Alt text: "An approved JSON rule flows into a Synapse execution node. A result card and violet explanation trace rows show why the rule matched and what action ran." + +Negative constraints: +Do not claim AI-generated rules are safe automatically. Do not depict non-technical users publishing generated rules without review. Do not make benchmark, speed, superiority, or competitor-negative claims. Do not show arbitrary code execution. Do not include fake metrics or lorem ipsum. +``` + +## Per-slide prompt split + +Use the full prompt above when the image tool supports multi-image carousel generation. If it only supports one image at a time, generate each slide with the same `Objective`, `Visual system`, `Format`, and `Negative constraints`, replacing the slide section with the target slide only. diff --git a/docs/benchmarks/assets/prompts/benchmark-chart-prompts.md b/docs/benchmarks/assets/prompts/benchmark-chart-prompts.md new file mode 100644 index 0000000..684abf8 --- /dev/null +++ b/docs/benchmarks/assets/prompts/benchmark-chart-prompts.md @@ -0,0 +1,137 @@ +# Benchmark chart prompts + +Use these prompts only with real measured benchmark output. If real output is missing, render a non-numeric empty state labeled `pending measured data` and do not show values from `benchmarks/sample-results.placeholder.json`. + +Required source gates for every metric chart: + +- `result_kind: "actual_benchmark"` +- `is_placeholder: false` +- `claims_allowed: true` +- exact source path included in the asset metadata +- Node version, package versions, commit SHA, benchmark date, warmup iterations, and measured iterations included in the chart or adjacent metadata + +Shared visual rules: + +- Use dark background `#020617`, raised panels `#0f172a`, and faint grid `#1e293b`. +- Use Neuron-JS cyan `#22d3ee`, json-rules-engine violet `#a78bfa`, JsonLogic amber `#fbbf24`, hand-coded TypeScript emerald `#34d399`, and rule-engine-js slate `#94a3b8`. +- Keep Neuron-JS identifiable but not visually oversized. +- Do not use 3D charts, cropped axes, trophy icons, rank badges, `fastest` language, or decorative data marks. +- Every visible number must exist in the source benchmark artifact. + +## Throughput comparison chart + +Metric: `throughput_decisions_per_second` + +Unit: decisions per second + +Interpretation note: Higher throughput may be better only within the named scenario and input size. + +Final path: `docs/benchmarks/assets/generated/benchmark-chart-throughput.svg` + +```text +Create a dark technical benchmark chart for @sebasoft/neuron-js. + +Title: "Throughput comparison" +Metric: throughput_decisions_per_second +Unit: decisions per second + +Compare @sebasoft/neuron-js, json-rules-engine, json-logic-js, hand-coded-typescript, and rule-engine-js from the supplied benchmark source file only. Show scenario and input size in the subtitle. Include source path, Node version, package versions, commit SHA, benchmark date, warmup iterations, and measured iterations. + +If the source artifact fails the required source gates, render no bars and show: "Pending measured data — structure only." +``` + +## Cold-start comparison chart + +Metric: `cold_start_ms` + +Unit: milliseconds + +Interpretation note: Lower cold start may be better only for the named runtime setup. + +Final path: `docs/benchmarks/assets/generated/benchmark-chart-cold-start.svg` + +```text +Create a dark technical benchmark chart for @sebasoft/neuron-js. + +Title: "Cold-start comparison" +Metric: cold_start_ms +Unit: milliseconds + +Compare the same engine set from the supplied benchmark source file only. Keep axis labels explicit that the metric is measured in milliseconds. Include methodology metadata and source path. Do not imply browser, serverless, or runtime conclusions beyond the source artifact. + +If the source artifact fails the required source gates, render no bars and show: "Pending measured data — structure only." +``` + +## Bundle-size comparison chart + +Metric: `bundle_size_minified_bytes` + +Unit: minified bytes + +Interpretation note: Lower bundle size may help browser/playground use, but does not measure runtime behavior. + +Final path: `docs/benchmarks/assets/generated/benchmark-chart-bundle-size.svg` + +```text +Create a dark technical bundle-size chart for @sebasoft/neuron-js. + +Title: "Bundle-size comparison" +Metric: bundle_size_minified_bytes +Unit: minified bytes + +Compare the same engine set from the supplied benchmark source file only. Label the bundler/minification context if the source artifact provides it. Do not convert units unless the chart also preserves the exact source value in metadata. + +If the source artifact fails the required source gates, render no bars and show: "Pending measured data — structure only." +``` + +## Validation-overhead chart + +Metric: `validation_overhead_ms` + +Unit: milliseconds + +Interpretation note: Shows the cost of schema validation as a delta against equivalent non-validation execution. + +Final path: `docs/benchmarks/assets/generated/benchmark-chart-validation-overhead.svg` + +```text +Create a dark technical validation-overhead chart for @sebasoft/neuron-js. + +Title: "Validation overhead" +Metric: validation_overhead_ms +Unit: milliseconds + +Compare the same engine set from the supplied benchmark source file only. Make the subtitle clear that this is overhead, not total runtime. Include scenario, input size, source path, and methodology metadata. + +If the source artifact fails the required source gates, render no bars and show: "Pending measured data — structure only." +``` + +## Explanation-overhead chart + +Metric: `explanation_overhead_ms` + +Unit: milliseconds + +Interpretation note: Shows the cost of trace/explanation collection as a delta against trace-disabled execution. + +Final path: `docs/benchmarks/assets/generated/benchmark-chart-explanation-overhead.svg` + +```text +Create a dark technical explanation-overhead chart for @sebasoft/neuron-js. + +Title: "Explanation overhead" +Metric: explanation_overhead_ms +Unit: milliseconds + +Compare the same engine set from the supplied benchmark source file only. Make the subtitle clear that this is trace/explanation overhead, not total runtime. Include scenario, input size, source path, and methodology metadata. + +If the source artifact fails the required source gates, render no bars and show: "Pending measured data — structure only." +``` + +## Verification checklist + +- Every visible number appears in the source benchmark artifact. +- Axis labels match the metric and unit. +- Chart metadata includes source path, scenario, input size, Node version, package versions, commit SHA, benchmark date, warmup iterations, and measured iterations. +- Alt text names the metric, scenario, input size, and no-fabrication policy. +- Any conclusion is limited to the named scenario and input size. diff --git a/docs/benchmarks/assets/prompts/methodology-infographic.md b/docs/benchmarks/assets/prompts/methodology-infographic.md new file mode 100644 index 0000000..3354825 --- /dev/null +++ b/docs/benchmarks/assets/prompts/methodology-infographic.md @@ -0,0 +1,44 @@ +# Methodology infographic prompt + +This prompt is safe to use before benchmark data exists because it contains no performance numbers. + +Final draft path: + +- `docs/benchmarks/assets/generated/methodology-card.svg` + +```text +Create a dark technical methodology infographic card for @sebasoft/neuron-js benchmarks. + +Objective: +Explain the honest benchmark rules before any performance claims are published. + +Headline: +"Benchmark proof without fake numbers" + +Layout: +Six evidence gates in a 16:9 dashboard card: +1. Same scenario +2. Same input size +3. Warmup before measurement +4. Real package versions +5. Commit SHA attached +6. Claims allowed only from actual benchmark output + +Text constraints: +- Do not include performance numbers. +- Include this footnote: "Placeholder data is for wiring only. Public claims require result_kind=actual_benchmark, is_placeholder=false, claims_allowed=true." +- Include source note: "Methodology: docs/benchmarks/methodology.md" + +Visual style: +Dark technical schematic, slate panels, faint grid, amber methodology accents, cyan source path highlight, emerald pass gates, rose blocked placeholder gate. No trophy icon, no fastest claim, no generic robot imagery. + +Export: +16:9 docs/social card, plus editable SVG source. +``` + +Verification checklist: + +- Contains no benchmark performance numbers. +- Shows the placeholder rejection rule. +- References `docs/benchmarks/methodology.md`. +- Works as a docs card and social explainer. diff --git a/docs/benchmarks/assets/source-data/README.md b/docs/benchmarks/assets/source-data/README.md new file mode 100644 index 0000000..23d5838 --- /dev/null +++ b/docs/benchmarks/assets/source-data/README.md @@ -0,0 +1,19 @@ +# Benchmark visual source data + +This folder is reserved for benchmark source artifacts consumed by generated visuals. + +Current status: no publishable benchmark result artifact exists. + +Allowed files: + +- `*.actual.json`: measured benchmark output that matches `../../results.schema.json` and sets `result_kind: "actual_benchmark"`, `is_placeholder: false`, and `claims_allowed: true`. +- `*.metadata.md`: adjacent methodology notes for generated assets. +- `*.example.json`: schema examples only, never public performance claims. + +Rejected for final chart claims: + +- `benchmarks/sample-results.placeholder.json` +- any file with `result_kind: "placeholder_sample"` +- any file with `is_placeholder: true` +- any file with `claims_allowed: false` +- any row whose notes contain `synthetic placeholder` diff --git a/docs/benchmarks/assets/storyboard/ai-rule-safety-carousel.md b/docs/benchmarks/assets/storyboard/ai-rule-safety-carousel.md new file mode 100644 index 0000000..8b34458 --- /dev/null +++ b/docs/benchmarks/assets/storyboard/ai-rule-safety-carousel.md @@ -0,0 +1,236 @@ +# AI-generated rule safety carousel/comic + +Asset taxonomy: `ai-rule-safety-carousel` +Status: script and generation prompts for review; no generated image asset yet. +Recommended surfaces: LinkedIn carousel, X image thread, GitHub discussion image set, docs proof-assets page. +Recommended format: 5 slides, 4:5 vertical carousel, 1080x1350 or 1200x1500. + +## Source grounding + +Source of record: + +- `chaos-vault/50-research/neuron-js-growth-plan.md`, NJS-GROWTH-07 lines 294-318: proof assets must support benchmarks, playground, and visual explanation; README proof asset waits for stable proof. +- `chaos-vault/50-research/neuron-js-marketing-assets-benchmark.md`, lines 87-100 and 160-174: publish honest proof, live demos, trace panels, and README visual proof; do not overclaim without measured evidence. +- `chaos-vault/50-research/neuron-js-social-demand-gap.md`, lines 184-210: target the pain that LLM-generated rules are dangerous without validation/schemas and that developers need deterministic guardrails and explanations. +- `docs/benchmarks/visual-proof-system.md`, lines 13-23, 110-118, and 179-189: visual assets should communicate credibility, inspectability, determinism, and AI safety; AI-rule-safety carousel uses a bridge/funnel technical comic with safety gates; social slides need short copy and `@sebasoft/neuron-js`. +- `docs/benchmarks/prompt-kit.md`, lines 213-258: base prompt for the AI-generated-rule safety carousel. + +Hindsight memory helped: no. Hindsight recall returned no relevant stored memories; this asset is grounded in the vault and repository files above. + +## Positioning guardrails + +Core message: + +> AI can draft rule JSON, but production logic still needs schema validation, developer-owned vocabulary, tests, approvals, rollback, deterministic execution, and explanation traces. + +Say: + +- `AI drafts. Neuron-JS verifies.` +- `Rules as data, not arbitrary code.` +- `Validate -> constrain -> test -> approve -> execute -> explain.` +- `Deterministic workflow logic with auditability.` + +Do not say: + +- `AI-generated rules are safe automatically.` +- `Business users can publish arbitrary logic without developer review.` +- `Neuron-JS prevents every possible business-policy error.` +- `Faster`, `best`, or performance claims without measured benchmark output. +- Negative claims against competitors. + +## Visual system + +- Style: technical comic, dark proof-system UI, restrained cyberpunk glow, no childish mascot. +- Palette: slate background, cyan Neuron-JS path, emerald validation pass, violet trace/audit, amber pending LLM draft, rose blocked unsafe path. +- Typography: Inter/system UI for captions, JetBrains Mono/SFMono for JSON, trace labels, and schema paths. +- Composition: bridge/funnel sequence. Each slide moves a generated draft through one safety gate. +- Recurring artifacts: LLM draft card, JSON rule card, schema gate, registry boundary, test/approval checklist, Synapse execution node, explanation trace panel. +- Footer on every slide: `@sebasoft/neuron-js — TypeScript-first JSON business rules`. +- Accessibility: keep headline under 8 words, body under 35 words, high contrast, alt text per slide. + +## 5-slide carousel script + +### Slide 1 — AI drafts the rule + +Headline: `AI can draft rules.` + +Body copy: `But a plausible JSON rule can still encode the wrong assumption, wrong field, or wrong action.` + +On-slide labels: + +- `LLM draft` +- `Looks valid? Maybe.` +- `Not production-ready.` + +Visual action: + +A developer reviews an amber LLM output card containing a short JSON-like rule draft. One field is subtly suspicious: `customer.segment == "enterprise"` routes to `auto_approve_discount`. A rose warning dot marks the risky assumption, but the frame avoids panic imagery. + +Purpose: + +Introduce the risk without fearmongering: the issue is not AI itself; the issue is trusting generated business logic without controls. + +Alt text: + +`A developer inspects an amber LLM-generated JSON rule card. A small rose warning marker highlights a risky business assumption, with text saying the draft is not production-ready.` + +### Slide 2 — Schema validation catches shape errors + +Headline: `Validate before runtime.` + +Body copy: `Schema-first checks reject malformed scripts before they reach execution.` + +On-slide labels: + +- `Schema validation` +- `blocked: unknown field` +- `repair path: conditions[0].operator` + +Visual action: + +The draft rule enters an emerald-and-rose validation gate. A malformed or unknown operator is blocked with a precise JSON path. A second corrected rule exits the gate as `schema: pass`. + +Purpose: + +Show that validation gives a concrete repair loop, not a vague failure state. + +Alt text: + +`A JSON rule passes through a schema validation gate. One invalid path is blocked in rose with an exact JSON path, while a corrected rule exits with an emerald pass marker.` + +### Slide 3 — Registry limits executable vocabulary + +Headline: `Constrain what can run.` + +Body copy: `A developer-owned registry defines approved actions, conditions, parameters, and rules.` + +On-slide labels: + +- `developer registry` +- `approved vocabulary only` +- `no arbitrary code` + +Visual action: + +The validated rule reaches a registry boundary. Approved condition/action tiles are cyan and connected; an unapproved `auto_refund_all_orders` tile is outside the boundary and blocked by a rose dashed line. + +Purpose: + +Clarify Neuron-JS as a controlled execution layer, not a magic AI agent or arbitrary-code runtime. + +Alt text: + +`A validated rule enters a developer-owned registry boundary. Approved action and condition tiles connect in cyan, while an unapproved action is blocked outside the boundary.` + +### Slide 4 — Tests, approval, and rollback + +Headline: `Review like code.` + +Body copy: `Generated rules still need tests, approval, ownership, and rollback before production use.` + +On-slide labels: + +- `test cases` +- `owner approval` +- `rollback ready` +- `sample structure only` + +Visual action: + +A checklist overlays the rule card: fixture tests pass, owner approval is pending then stamped approved, rollback snapshot is saved. No benchmark numbers or success-rate metrics appear. + +Purpose: + +Prevent overclaiming. Validation is necessary but not sufficient; governance completes the safety story. + +Alt text: + +`A generated rule card sits beside a review checklist showing tests, owner approval, and rollback snapshot. The visual emphasizes governance before production use.` + +### Slide 5 — Deterministic execution with trace + +Headline: `Then execute deterministically.` + +Body copy: `Synapse runs the approved rule path, and the trace explains why the decision matched or failed.` + +On-slide labels: + +- `Synapse execution` +- `result: routed` +- `trace: condition matched` +- `audit-ready explanation` + +Visual action: + +The approved rule enters a cyan Synapse execution node. Output splits into a result card and violet trace rows: input field, condition, matched/failed status, action triggered. The final caption reads `AI drafts. Neuron-JS verifies.` + +Purpose: + +End on the product wedge: deterministic workflow logic with auditability, not probabilistic decisioning. + +Alt text: + +`An approved JSON rule flows into a Synapse execution node. A result card and violet explanation trace rows show why the rule matched and what action ran.` + +## Single-master image generation prompt + +```text +Create a 5-slide technical comic carousel for @sebasoft/neuron-js about safe AI-generated JSON business rules. + +Objective: +Explain that LLMs can draft JSON rules, but production logic needs schema validation, developer-owned registry boundaries, tests, approvals, rollback, deterministic execution, and explanation traces. The tone should be credible and developer-specific, not fearful and not generic AI hype. + +Format: +5 vertical carousel slides, 4:5 aspect ratio, designed for LinkedIn and X image threads. Keep each headline under 8 words and body text under 35 words. Include footer on every slide: "@sebasoft/neuron-js — TypeScript-first JSON business rules". + +Visual system: +Dark technical comic with safety gates. Use slate background, raised dark panels, faint grid, Inter/system UI captions, JetBrains Mono/SFMono JSON labels. Use cyan for the Neuron-JS execution path, emerald for validation/pass, violet for trace/audit, amber for pending LLM draft, rose for blocked invalid paths. Avoid mascots, magic wand imagery, robot savior imagery, generic SaaS gradients, and fearmongering visuals. + +Slide 1: +Headline: "AI can draft rules." +Body: "But a plausible JSON rule can still encode the wrong assumption, wrong field, or wrong action." +Scene: Developer inspects an amber LLM output card containing a short JSON-like rule draft. One field is subtly suspicious: customer.segment == "enterprise" routes to auto_approve_discount. Add labels "LLM draft", "Looks valid? Maybe.", and "Not production-ready." Rose warning marker only; no panic imagery. +Alt text: "A developer inspects an amber LLM-generated JSON rule card. A small rose warning marker highlights a risky business assumption, with text saying the draft is not production-ready." + +Slide 2: +Headline: "Validate before runtime." +Body: "Schema-first checks reject malformed scripts before they reach execution." +Scene: Draft rule enters an emerald-and-rose validation gate. One invalid path is blocked with exact label "blocked: conditions[0].operator". A corrected rule exits as "schema: pass". Show repair loop, not dead-end failure. +Alt text: "A JSON rule passes through a schema validation gate. One invalid path is blocked in rose with an exact JSON path, while a corrected rule exits with an emerald pass marker." + +Slide 3: +Headline: "Constrain what can run." +Body: "A developer-owned registry defines approved actions, conditions, parameters, and rules." +Scene: Validated rule reaches a registry boundary. Approved condition/action tiles are cyan and connected. An unapproved action tile, "auto_refund_all_orders", is outside the boundary and blocked by a rose dashed line. Include labels "developer registry", "approved vocabulary only", and "no arbitrary code". +Alt text: "A validated rule enters a developer-owned registry boundary. Approved action and condition tiles connect in cyan, while an unapproved action is blocked outside the boundary." + +Slide 4: +Headline: "Review like code." +Body: "Generated rules still need tests, approval, ownership, and rollback before production use." +Scene: Checklist beside the rule card: "test cases", "owner approval", "rollback ready". Approval stamp is developer/owner review, not automatic AI approval. Add small note "sample structure only". Do not show fake percentages, fake pass rates, or benchmark numbers. +Alt text: "A generated rule card sits beside a review checklist showing tests, owner approval, and rollback snapshot. The visual emphasizes governance before production use." + +Slide 5: +Headline: "Then execute deterministically." +Body: "Synapse runs the approved rule path, and the trace explains why the decision matched or failed." +Scene: Approved rule enters a cyan Synapse execution node. Output splits into a result card and violet trace rows showing input field, condition, matched/failed status, and action triggered. Final caption: "AI drafts. Neuron-JS verifies." +Alt text: "An approved JSON rule flows into a Synapse execution node. A result card and violet explanation trace rows show why the rule matched and what action ran." + +Negative constraints: +Do not claim AI-generated rules are safe automatically. Do not depict non-technical users publishing generated rules without review. Do not make benchmark, speed, superiority, or competitor-negative claims. Do not show arbitrary code execution. Do not include fake metrics or lorem ipsum. +``` + +## Channel fit + +- LinkedIn: strongest fit. Use as a 5-slide carousel for engineering leads, developer tooling audiences, and AI-workflow builders. Caption angle: `LLMs can draft business rules. The production question is: what validates, constrains, tests, executes, and explains them?` +- X: good fit as a 5-image thread. Lead with Slide 1 or Slide 5. Keep post copy concrete and avoid broad AI-safety rhetoric. +- GitHub discussion/docs: useful as a proof-system storyboard before generating final images. Pair with `docs/benchmarks/visual-proof-system.md` and `docs/benchmarks/prompt-kit.md`. + +## Review checklist + +- [x] Communicates the risk of LLM-generated rules without fearmongering. +- [x] Shows Neuron-JS guardrails: schema validation, registry constraints, tests, approvals, rollback, deterministic execution, trace. +- [x] Avoids benchmark numbers and performance claims. +- [x] Avoids competitor-negative claims. +- [x] Includes text, visual action, style constraints, and alt text for each slide. +- [x] Uses source-grounded product language: TypeScript-first, explainable, schema-validatable JSON business rules. diff --git a/docs/benchmarks/assets/storyboard/playground-readme-gif.md b/docs/benchmarks/assets/storyboard/playground-readme-gif.md new file mode 100644 index 0000000..1eee60b --- /dev/null +++ b/docs/benchmarks/assets/storyboard/playground-readme-gif.md @@ -0,0 +1,56 @@ +# Playground README GIF storyboard + +Primary spec: `docs/playground/readme-demo-capture.md`. + +Status: blocked on playground implementation. This storyboard is ready for capture automation once `/playground/?example=pricing-rules&capture=readme` exists. + +## Fixture lock + +Use only the runnable pricing fixture: + +- rules: `examples/pricing-rules/rules.json` +- input: `examples/pricing-rules/input.json` +- expected output: `examples/pricing-rules/expected-output.json` +- runnable verification: `node examples/pricing-rules/run.ts` after `yarn build` + +Canonical visible result: + +```json +{ + "ok": true, + "rulesExecuted": 1, + "finalTotal": 105, + "discountAmount": 20, + "messages": ["Applied 16% discount: -20"] +} +``` + +## Shot list + +| Step | Duration | Frame | Required selector/assertion | +| --- | ---: | --- | --- | +| 1 | 1.0s | Playground shell loads with Pricing Rules selected. | `[data-testid="example-select"]` contains `pricing-rules`. | +| 2 | 1.5s | Rule JSON panel highlights `vip-order-discount`. | `[data-testid="rule-json-panel"]` contains `vip-order-discount`. | +| 3 | 1.5s | Input JSON panel highlights `cart.subtotal = 125`. | `[data-testid="input-json-panel"]` contains `125`. | +| 4 | 1.5s | Validation state shows passed and zero errors. | `[data-testid="validation-status"]` contains `passed`; `[data-testid="validation-errors"]` contains `No validation errors`. | +| 5 | 2.0s | Trace highlights the matched rule and condition. | `[data-testid="trace-row-vip-order-discount"]` has `.trace-row--matched`. | +| 6 | 2.0s | Result panel displays `finalTotal: 105` and discount. | `[data-testid="result-panel"]` contains `finalTotal: 105` and `discountAmount: 20`. | +| 7 | 1.5s | Share URL appears with encoded state. | `[data-testid="share-url"]` contains `example=pricing-rules`. | + +## Capture command contract + +Future command: + +```sh +yarn capture:playground-readme --example pricing-rules --format mp4,gif +``` + +Future outputs: + +```text +docs/benchmarks/assets/generated/playground-readme-demo.mp4 +docs/benchmarks/assets/generated/playground-readme-demo.gif +docs/benchmarks/assets/generated/playground-readme-demo.metadata.json +``` + +The asset must not include benchmark numbers, rank claims, or performance language. diff --git a/docs/benchmarks/benchmark-visual-pack.md b/docs/benchmarks/benchmark-visual-pack.md new file mode 100644 index 0000000..5767bee --- /dev/null +++ b/docs/benchmarks/benchmark-visual-pack.md @@ -0,0 +1,112 @@ +# NJS-GROWTH-07 benchmark visual pack plan + +Status: draft asset pack and prompt set. Final benchmark charts are blocked until measured benchmark output exists. + +Source of record: + +- `chaos-vault/50-research/neuron-js-growth-plan.md`, `NJS-GROWTH-07` lines 294-318: proof assets must cover benchmarks, playground, and visual explanation. +- `chaos-vault/50-research/neuron-js-marketing-assets-benchmark.md`, lines 87-100 and 160-174: publish honest benchmarks, compare the required competitor set, expose methodology, support playground proof, trace panel, and README GIF. +- `chaos-vault/50-research/neuron-js-social-demand-gap.md`, lines 160-169 and 184-210: clean charts, visual proof, and the TypeScript-first explainability/validation wedge are the strongest trust signals. + +Hindsight recall was queried for this task and returned no relevant stored memories. The vault files above are the governing source. + +## Current data status + +No publishable benchmark data exists yet. + +The only available benchmark fixture is `benchmarks/sample-results.placeholder.json`. It is explicitly marked: + +- `result_kind: "placeholder_sample"` +- `is_placeholder: true` +- `claims_allowed: false` + +Therefore this pack contains non-numeric chart prompts and draft asset paths only. They must not be used as performance claims, README proof, social claims, npm copy, or comparison claims until replaced with measured harness output matching `docs/benchmarks/results.schema.json`. + +## Source data contract for final charts + +Final assets must use a benchmark artifact that satisfies all of these gates: + +- `result_kind` is `actual_benchmark`. +- `is_placeholder` is `false`. +- `claims_allowed` is `true`. +- Every visible number exists in the source artifact. +- The asset metadata includes source path, scenario, input size, Node version, package versions, commit SHA, benchmark date, warmup iterations, and measured iterations. +- Axis labels use the same metric meaning documented in `docs/benchmarks/methodology.md` and `docs/benchmarks/results.schema.json`. + +If any gate fails, render empty states or labels such as `pending measured data`; do not render numeric claims. + +## Competitor and scenario scope + +Competitor set: + +1. `@sebasoft/neuron-js` +2. `json-rules-engine` +3. `json-logic-js` / JsonLogic +4. `hand-coded-typescript` +5. `rule-engine-js` + +`rulepilot` remains an alternate only if `rule-engine-js` becomes infeasible and the benchmark contract is updated. + +Scenario set: + +- `pricing-discount` +- `eligibility-approval` +- `workflow-routing` + +Input-size set: + +- `smoke` +- `small` +- `medium` +- `large` only when runtime remains practical and the benchmark run explicitly includes it. + +## Asset set + +| Asset | Draft prompt path | Final generated asset path | Final source requirement | Primary surfaces | +| --- | --- | --- | --- | --- | +| Throughput comparison chart | `docs/benchmarks/assets/prompts/benchmark-chart-prompts.md#throughput-comparison-chart` | `docs/benchmarks/assets/generated/benchmark-chart-throughput.svg` | `throughput_decisions_per_second` from actual benchmark output | docs, README support, X/LinkedIn | +| Cold-start comparison chart | `docs/benchmarks/assets/prompts/benchmark-chart-prompts.md#cold-start-comparison-chart` | `docs/benchmarks/assets/generated/benchmark-chart-cold-start.svg` | `cold_start_ms` from actual benchmark output | docs, blog, social | +| Bundle-size comparison chart | `docs/benchmarks/assets/prompts/benchmark-chart-prompts.md#bundle-size-comparison-chart` | `docs/benchmarks/assets/generated/benchmark-chart-bundle-size.svg` | `bundle_size_minified_bytes` from actual benchmark output | docs, README support | +| Validation-overhead chart | `docs/benchmarks/assets/prompts/benchmark-chart-prompts.md#validation-overhead-chart` | `docs/benchmarks/assets/generated/benchmark-chart-validation-overhead.svg` | `validation_overhead_ms` from actual benchmark output | docs, social proof | +| Explanation-overhead chart | `docs/benchmarks/assets/prompts/benchmark-chart-prompts.md#explanation-overhead-chart` | `docs/benchmarks/assets/generated/benchmark-chart-explanation-overhead.svg` | `explanation_overhead_ms` from actual benchmark output | docs, social proof | +| Honest methodology infographic/card | `docs/benchmarks/assets/prompts/methodology-infographic.md` | `docs/benchmarks/assets/generated/methodology-card.svg` | No performance numbers; must cite methodology and no-fabrication policy | docs, social, PR review | + +## Aspect ratio and export matrix + +| Surface | Ratio | Recommended dimensions | Notes | +| --- | --- | --- | --- | +| README proof strip | `5:1` or `4:1` | `1600x320` or `1600x400` | Wide proof strip; use non-numeric placeholders until measured data exists. | +| Docs benchmark hero | `16:9` | `1600x900` | Best for methodology and chart overview. | +| Docs inline chart | `16:9` or `4:3` | `1200x675` or `1200x900` | Keep labels readable at docs column width. | +| X / LinkedIn link card | `16:9` | `1200x675` | Maximum headline eight words. Include methodology footnote when numbers appear. | +| LinkedIn / Instagram feed | `1:1` | `1200x1200` | Use one metric per card. | +| LinkedIn carousel | `4:5` | `1080x1350` | One claim per slide; source footnote on each measured slide. | +| Blog hero | `16:9` | `1600x900` | Prefer methodology-first framing over winner framing. | + +## Channel copy guardrails + +Use this claim structure only after measured data exists: + +- Method first: `Measured on at ; see methodology.` +- Metric second: name the exact metric and unit. +- Interpretation third: explain what the metric does and does not prove. + +Do not use: + +- `fastest`, `best`, `beats`, `dominates`, or winner language unless the source data proves it for the named scenario and input size. +- Synthetic placeholder values in copy, alt text, chart labels, posts, README, or npm copy. +- Cropped axes, trophy icons, rank badges, or chart decorations that imply unsupported superiority. + +## Final production workflow + +1. Receive actual benchmark artifact matching `docs/benchmarks/results.schema.json`. +2. Validate the data gates in this plan. +3. Generate five metric charts from the artifact, one metric per chart. +4. Attach methodology metadata to each SVG or adjacent `.md` metadata file. +5. Produce social exports using the aspect ratio matrix. +6. Run visual QA: axis labels, units, source path, contrast, legibility, alt text, and no unsupported claims. +7. Only then consider README or social publication. + +## Publication blocker + +Final publication is blocked until real measured benchmark output exists. The draft prompts and paths in this pack are structure-only assets for review, layout alignment, and downstream generation. diff --git a/docs/benchmarks/index.md b/docs/benchmarks/index.md index cba4689..e3a07a0 100644 --- a/docs/benchmarks/index.md +++ b/docs/benchmarks/index.md @@ -9,7 +9,10 @@ This section is a design and generation foundation. It does not publish benchmar - [Benchmark methodology and result contract](./methodology.md): competitor set, scenario matrix, input-size matrix, metric definitions, and placeholder-data policy. - [Result JSON schema](/benchmarks/results.schema.json): machine-readable contract for benchmark output and downstream chart data. - [Visual proof system](./visual-proof-system.md): palette, typography, composition, diagram style, chart rules, social-card constraints, and README proof strip guidance. +- [Benchmark visual pack plan](./benchmark-visual-pack.md): chart asset matrix, aspect ratios, channel guardrails, draft prompt paths, and publication blocker. +- [Explainability trace diagram metadata](./assets/generated/explainability-trace-diagram.md): source grounding, caption, alt text, integrity notes, and README-safe SVG path. - [Visual proof prompt kit](./prompt-kit.md): reusable prompts for benchmark infographics, explainability trace diagrams, playground README GIF storyboards, AI-rule safety carousels, and README proof strips. +- [Playground README demo capture specification](../playground/readme-demo-capture.md): route, selectors, storyboard, future capture command, and fixture-backed result contract. - [Asset folder recommendation](./assets/): recommended `docs/benchmarks/assets/` structure and data-integrity rules. ## Proof promise @@ -21,6 +24,18 @@ Neuron-JS visual proof assets should make four claims visible: 3. Execution is deterministic through a developer-owned registry and Synapse. 4. Explanation traces show why a rule matched or failed. +## README-safe proof visual + +The current README uses the explainability trace diagram because it is inspectable, asset-backed, and contains no benchmark numbers or superiority claims. + +![Neuron-JS diagram showing rule JSON and business input flowing through schema validation, developer registry, deterministic Synapse evaluation, result output, explanation trace, and audit-ready decision.](./assets/generated/explainability-trace-diagram.svg) + +Supporting files: + +- SVG asset: [`assets/generated/explainability-trace-diagram.svg`](./assets/generated/explainability-trace-diagram.svg) +- Metadata, caption, and integrity notes: [`assets/generated/explainability-trace-diagram.md`](./assets/generated/explainability-trace-diagram.md) +- HTML preview wrapper: [`assets/generated/explainability-trace-diagram.html`](./assets/generated/explainability-trace-diagram.html) + ## Data policy Do not fabricate benchmark results. When data is unavailable, use non-numeric structure, empty states, or labels such as `pending measured data`. diff --git a/docs/playground/readme-demo-capture.md b/docs/playground/readme-demo-capture.md new file mode 100644 index 0000000..953589f --- /dev/null +++ b/docs/playground/readme-demo-capture.md @@ -0,0 +1,203 @@ +# README playground demo capture specification + +Status: capture contract for the future Neuron-JS playground. The repository does not currently contain a `docs/playground/` implementation, so this page fixes the demo route, selectors, storyboard, output files, and data-integrity checks needed before capture automation is unblocked. + +Source of record: + +- `chaos-vault/50-research/neuron-js-growth-plan.md`, `NJS-GROWTH-07` lines 294-318: proof assets must include benchmarks, playground, visual explanation, and a README GIF when the playground is stable. +- `chaos-vault/50-research/neuron-js-marketing-assets-benchmark.md`, lines 160-174: live demos require browser playground, shareable URLs, explain trace panel, README GIF, and optional visual rule builder. +- `chaos-vault/50-research/neuron-js-social-demand-gap.md`, lines 160-169: social proof should show playground validation, explanation traces, before/after logic, and AI-generated rule validation. +- Runnable fixture source: `examples/pricing-rules/rules.json`, `examples/pricing-rules/input.json`, and `examples/pricing-rules/expected-output.json`. + +Do not record or publish this asset until the playground renders the fixture data below and the capture command can validate the result against `examples/pricing-rules/expected-output.json`. + +## Capture status + +Blocked on playground implementation, not on missing planning. + +Required future command: + +```sh +yarn capture:playground-readme --example pricing-rules --format mp4,gif +``` + +Required future outputs: + +```text +docs/benchmarks/assets/generated/playground-readme-demo.mp4 +docs/benchmarks/assets/generated/playground-readme-demo.gif +docs/benchmarks/assets/generated/playground-readme-demo.metadata.json +``` + +The metadata JSON must include: + +- capture command. +- git commit SHA. +- package version. +- Node version. +- browser name and version. +- source fixture paths. +- output dimensions and duration. +- validation result comparing the visible result panel to `examples/pricing-rules/expected-output.json`. + +## Playground route contract + +Canonical capture route: + +```text +/playground/?example=pricing-rules&capture=readme +``` + +Required route behavior: + +1. Load the pricing example without user interaction. +2. Render deterministic panels in the same order on every run. +3. Encode the loaded example in shareable URL state. +4. Disable random animation timing when `capture=readme` is present. +5. Use stable selectors from this page so browser automation can assert content before recording. + +Recommended shareable URL state: + +```text +/playground/?example=pricing-rules&state=&capture=readme +``` + +The decoded state must contain only serializable JSON data: + +```json +{ + "example": "pricing-rules", + "scriptPath": "examples/pricing-rules/rules.json", + "inputPath": "examples/pricing-rules/input.json", + "expectedOutputPath": "examples/pricing-rules/expected-output.json" +} +``` + +## Stable selectors + +Use `data-testid` for assertions and `data-capture-step` for timeline control. + +| Selector | Required content | Capture step | +| --- | --- | --- | +| `[data-testid="playground-root"]` | Whole playground shell | all | +| `[data-testid="example-select"]` | Active example label: `pricing-rules` | 1 | +| `[data-testid="rule-json-panel"]` | Pretty-printed `examples/pricing-rules/rules.json` | 2 | +| `[data-testid="input-json-panel"]` | Pretty-printed `examples/pricing-rules/input.json` | 3 | +| `[data-testid="validation-status"]` | `Validation: passed` for the canonical run | 4 | +| `[data-testid="validation-errors"]` | Empty state: `No validation errors` | 4 | +| `[data-testid="trace-panel"]` | Ordered explanation trace rows | 5 | +| `[data-testid="trace-row-vip-order-discount"]` | Highlighted matched rule `vip-order-discount` | 5 | +| `[data-testid="result-panel"]` | `ok: true`, `rulesExecuted: 1`, `finalTotal: 105`, `discountAmount: 20` | 6 | +| `[data-testid="share-url"]` | Shareable URL containing `example=pricing-rules` | 7 | + +Required trace classes: + +| Class | Meaning | +| --- | --- | +| `.trace-row--matched` | Rule or condition matched. | +| `.trace-row--failed` | Rule or condition failed. | +| `.trace-row--focused` | Current row during capture. | +| `.trace-row--muted` | Non-current rows during capture. | + +## Canonical visible data + +Rule fixture: + +```json +{ + "id": "pricing-rules-demo", + "rules": [ + { + "id": "vip-order-discount", + "type": "simple_rule", + "conditions": ["minimum-cart-subtotal >= 100"], + "actions": ["apply_discount 16%"] + } + ] +} +``` + +Input fixture: + +```json +{ + "messages": [], + "state": { + "cart": { "subtotal": 125, "currency": "USD" }, + "customer": { "segment": "vip" } + } +} +``` + +Expected result fixture: + +```json +{ + "ok": true, + "rulesExecuted": 1, + "finalTotal": 105, + "discountAmount": 20, + "messages": ["Applied 16% discount: -20"] +} +``` + +These visible values are backed by the runnable `examples/pricing-rules/run.ts` path. Capture automation must run the example or import the same fixture data before recording. + +## README-safe storyboard + +Target duration: 8-12 seconds. + +Target dimensions: + +- MP4: `1280x720`, H.264, under 8 MB when practical. +- GIF: `960x540`, 12 FPS, under 5 MB when practical. + +Timeline: + +| Time | Shot | Assertion before capture proceeds | +| --- | --- | --- | +| 0.0-1.0s | Playground opens with `pricing-rules` selected. | `[data-testid="example-select"]` contains `pricing-rules`. | +| 1.0-2.5s | Rule JSON panel focuses `vip-order-discount`. | `[data-testid="rule-json-panel"]` contains `vip-order-discount`. | +| 2.5-4.0s | Input JSON panel focuses `cart.subtotal: 125`. | `[data-testid="input-json-panel"]` contains `125`. | +| 4.0-5.5s | Validation status turns green. | `[data-testid="validation-status"]` contains `passed`. | +| 5.5-7.5s | Trace panel highlights the matched condition and action. | `[data-testid="trace-row-vip-order-discount"]` has `.trace-row--matched`. | +| 7.5-9.5s | Result panel displays deterministic output. | `[data-testid="result-panel"]` contains `finalTotal: 105`. | +| 9.5-11.0s | Shareable URL is shown/copied statefully. | `[data-testid="share-url"]` contains `example=pricing-rules`. | + +No benchmark chart, throughput claim, or performance language belongs in this demo. It proves inspectability, validation, deterministic execution, explanation, and shareability only. + +## Future automation requirements + +Recommended implementation stack: + +- Playwright for browser launch, assertions, and MP4/webm recording. +- `ffmpeg` for README-safe GIF conversion. +- A script at `scripts/capture-playground-readme.ts` wired to `yarn capture:playground-readme`. + +Automation sequence: + +1. Build the docs/playground site. +2. Start the preview server on a deterministic local port. +3. Open `/playground/?example=pricing-rules&capture=readme`. +4. Assert all selectors above are present. +5. Assert result-panel values match `examples/pricing-rules/expected-output.json`. +6. Record the MP4. +7. Convert a GIF from the same recording. +8. Write metadata JSON beside the output files. +9. Exit non-zero if any selector, fixture value, output file, or metadata field is missing. + +## README integration rule + +README may embed the generated asset only after all conditions are true: + +1. The playground route exists and is linked from the docs. +2. Capture automation passes locally and in CI or documented release workflow. +3. The visible rule/input/result pair is validated against the runnable example fixtures. +4. The README alt text describes the demo without claiming benchmark superiority. +5. The asset metadata file is committed beside the generated MP4/GIF. + +Suggested alt text: + +```text +Neuron-JS playground demo showing pricing rule JSON, input JSON, validation passed, matched explanation trace, deterministic discount result, and shareable state. +``` diff --git a/tests/contracts/playground-capture-contract.test.ts b/tests/contracts/playground-capture-contract.test.ts new file mode 100644 index 0000000..fefc320 --- /dev/null +++ b/tests/contracts/playground-capture-contract.test.ts @@ -0,0 +1,84 @@ +import { existsSync, readFileSync } from 'node:fs'; +import { describe, expect, it } from 'vitest'; + +const captureSpecPath = 'docs/playground/readme-demo-capture.md'; +const storyboardPath = 'docs/benchmarks/assets/storyboard/playground-readme-gif.md'; +const pricingRulePath = 'examples/pricing-rules/rules.json'; +const pricingInputPath = 'examples/pricing-rules/input.json'; +const pricingExpectedOutputPath = 'examples/pricing-rules/expected-output.json'; + +function readJson(path: string): T { + return JSON.parse(readFileSync(path, 'utf8')) as T; +} + +describe('playground README demo capture contract', () => { + it('publishes a concrete capture spec and storyboard while playground implementation is pending', () => { + expect(existsSync(captureSpecPath)).toBe(true); + expect(existsSync(storyboardPath)).toBe(true); + + const spec = readFileSync(captureSpecPath, 'utf8'); + const storyboard = readFileSync(storyboardPath, 'utf8'); + + expect(spec).toContain('Blocked on playground implementation, not on missing planning.'); + expect(spec).toContain('/playground/?example=pricing-rules&capture=readme'); + expect(spec).toContain('yarn capture:playground-readme --example pricing-rules --format mp4,gif'); + expect(storyboard).toContain('docs/benchmarks/assets/generated/playground-readme-demo.mp4'); + expect(storyboard).toContain('docs/benchmarks/assets/generated/playground-readme-demo.gif'); + }); + + it('locks browser automation to stable selectors for every required demo flow panel', () => { + const spec = readFileSync(captureSpecPath, 'utf8'); + const requiredSelectors = [ + '[data-testid="example-select"]', + '[data-testid="rule-json-panel"]', + '[data-testid="input-json-panel"]', + '[data-testid="validation-status"]', + '[data-testid="validation-errors"]', + '[data-testid="trace-panel"]', + '[data-testid="trace-row-vip-order-discount"]', + '[data-testid="result-panel"]', + '[data-testid="share-url"]', + ]; + + for (const selector of requiredSelectors) { + expect(spec).toContain(selector); + } + }); + + it('backs the visible demo result with the runnable pricing example fixtures', () => { + const spec = readFileSync(captureSpecPath, 'utf8'); + const script = readJson<{ id: string; rules: Array<{ id: string }> }>(pricingRulePath); + const input = readJson<{ state: { cart: { subtotal: number; currency: string } } }>(pricingInputPath); + const expected = readJson<{ + ok: boolean; + rulesExecuted: number; + finalTotal: number; + discountAmount: number; + messages: string[]; + }>(pricingExpectedOutputPath); + + expect(script.id).toBe('pricing-rules-demo'); + expect(script.rules[0]?.id).toBe('vip-order-discount'); + expect(input.state.cart.subtotal).toBe(125); + expect(input.state.cart.currency).toBe('USD'); + expect(expected).toEqual({ + ok: true, + rulesExecuted: 1, + finalTotal: 105, + discountAmount: 20, + messages: ['Applied 16% discount: -20'], + }); + + for (const value of ['pricing-rules-demo', 'vip-order-discount', '125', 'finalTotal: 105', 'discountAmount: 20']) { + expect(spec).toContain(value); + } + }); + + it('links the capture spec from proof-asset docs and VitePress navigation', () => { + const proofIndex = readFileSync('docs/benchmarks/index.md', 'utf8'); + const config = readFileSync('docs/.vitepress/config.ts', 'utf8'); + + expect(proofIndex).toContain('../playground/readme-demo-capture.md'); + expect(config).toContain("{ text: 'Playground Capture Spec', link: '/playground/readme-demo-capture' }"); + }); +});