diff --git a/docs/design/creative-ideas-thread.md b/docs/design/creative-ideas-thread.md index 3777350b..cca5bb3d 100644 --- a/docs/design/creative-ideas-thread.md +++ b/docs/design/creative-ideas-thread.md @@ -9,9 +9,9 @@ description: > and existing agent/skill invocation. Covers the CreativeIdea prospective-memory data model, the status state-machine, the generator thread inputs, the four-reviewer pipeline, the routing (idea→goal, idea→issue, idea-PR human-review gate), the safety - model, and a phased roadmap. Design + typed foundation + tests only — gated OFF by - default; no autonomous behavior; the operator redeploys after merge. -last_updated: 2026-07-05 + model, and a phased roadmap. Implemented and wired into the running daemon — + default-ON, opt-out via SIMARD_CREATIVE_IDEAS_ENABLED; the operator redeploys after merge. +last_updated: 2026-07-06 review_schedule: as-needed owner: simard doc_type: design @@ -61,42 +61,41 @@ The design must satisfy three constraints that shaped every decision below: 1. **One brain.** This is not a new service or a "Bridge." It is a **cognitive thread** plus a set of **reviewers** inside the single Simard brain, reusing the Mind scheduler and existing memory/goal/gh subsystems. -2. **Safe by construction.** The subsystem is OFF by default; high-risk or - irreversible ideas *always* route to human review; PRs born from creative ideas are - **blocked from merge** (draft + label + owner review-required) until a human - approves — enforced **without** `--admin` or `--no-verify`. +2. **Safe by construction.** The subsystem is **default-ON, opt-out** (consistent + with the Overseer and Journal threads), but safe regardless of the switch: + high-risk or irreversible ideas *always* route to human review; PRs born from + creative ideas are **blocked from merge** (draft + label + owner review-required) + until a human approves — enforced **without** `--admin` or `--no-verify`. 3. **Measured, not vibes.** Every accepted idea carries a concrete **success metric** (tied where possible to existing self-metrics) and only reaches `ImplementationCompleted` when that metric is met. ## Goals and non-goals -**Goals (this spike):** +**Delivered:** - A durable design (this document): data model, status state-machine, thread inputs, reviewer pipeline, routing, safety, and future milestones. -- Typed, tested Rust scaffolding for: the `CreativeIdea` prospective-memory type + - status state-machine + store/retrieve; the generator-thread skeleton (typed inputs, - pluggable idea-source, gated OFF); the reviewer-pipeline trait + four adapters + - synthesis (interfaces + fakes); the routing interfaces (idea→goal, idea→issue+owner, - idea-PR draft+label+owner-review gate) as typed functions with fakes. +- Typed, tested Rust for: the `CreativeIdea` prospective-memory type + status + state-machine + store/retrieve; the generator thread (typed inputs, pluggable + idea-source, **registered with the Mind scheduler**, default-ON opt-out); the + reviewer-pipeline trait + four adapters + synthesis; the routing (idea→goal, + idea→issue+owner, idea-PR draft+label+owner-review gate) as typed functions. +- **Daemon wiring:** the thread is registered from the OODA daemon's + cognitive-thread setup and ticks on its cadence, with a dedicated startup log + line and a per-tick summary (see [Daemon wiring](#daemon-wiring)). +- A production agent-backed idea source, real skill/agent reviewer adapters, and + a real `gh` routing seam — all driven by the wired thread. - Tests with **no network** (fakes for reviewers / goal-store / issue-filer / gh / - clock). + clock), including that an enabled tick generates + reviews + routes, that the + daemon registers the thread when enabled, and that it is not registered when + opted out. **Non-goals (explicitly future work):** -- Wiring the thread into the Mind scheduler / daemon `main` (no runtime behavior - change). -- A production LLM idea-source (the spike ships a pluggable trait + a deterministic - fake). -- Real reviewer execution against the `crusty-old-engineer` skill / `philosophy-guardian` - agent (the spike ships adapters that shape the invocation but run through fakes in - tests). -- Extending `GhClient` with real label/assignee/draft/review-request subprocess calls - (the spike defines the seam and tests it with a fake; the real `gh` calls are a - marked `// FUTURE:` stub). -- A native "links" edge type in prospective memory (see Decision 1 — links are carried - in the payload for now). +- Adaptive cadence / richer portfolio + novelty scoring (M6 tuning). +- A native "links" edge type in prospective memory (see Decision 1 — links are + carried in the payload for now; a `payload_version` 2 migration is future work). ## Key decisions (resolved ambiguities) @@ -105,7 +104,7 @@ The design must satisfy three constraints that shaped every decision below: | 1 | Model `CreativeIdea` as a typed struct that **round-trips to/from** a `CognitiveProspective` node. Store `status`, `context`, and typed `links` as a JSON payload in `action_on_trigger`; `description` = the idea text; `status` mirrored into the prospective `status: String`. A thin `CreativeIdeaStore` seam wraps `store_prospective` / `list_all_prospective`. | Zero schema change to prospective memory; fully round-trippable; marks a native links field as future work. | | 2 | Reuse **`ThreadKind::BackgroundThought`** for the generator thread; do not add an enum variant. | The variant is already reserved for "idle associative background thought"; avoids touching the enum and its exhaustive matches. | | 3 | New top-level **`src/creative_ideas/`** owns the reviewer-pipeline trait, the four adapters, synthesis, routing, dedup/portfolio, and the config flag. The prospective type lives in `src/cognitive_memory/creative_idea.rs`; the generator thread in `src/cognitive_threads/threads/creative_ideas.rs`. | Respects "prospective type in `cognitive_memory`, thread in `cognitive_threads`" while keeping pipeline/routing cohesive and independently testable. | -| 4 | Do **not** mutate the live `gh` tooling. Define an `IdeaGhClient` **extension seam** (labeled+assigned issue creation; PR draft/label/owner-review-request) with a fake for tests; the real subprocess wiring is a marked `// FUTURE:` stub. Reuse the existing `GhClient` (`src/stewardship/gh_client.rs`) pattern; never `--admin` / `--no-verify`. | The existing `GhClient` has only `search_issues` / `create_issue`; the gate needs labels/assignees/draft/review. Extending it live would touch the daemon; the seam keeps the spike additive and testable. | +| 4 | Do **not** mutate the live `gh` tooling. Define an `IdeaGhClient` **extension seam** (labeled+assigned issue creation; PR draft/label/owner-review-request), faked in tests and backed in production by `RealIdeaGhClient` (a `gh` subprocess impl using pure argv builders). Reuse the existing `GhClient` (`src/stewardship/gh_client.rs`) pattern; never `--admin` / `--no-verify`. | The existing `GhClient` has only `search_issues` / `create_issue`; the gate needs labels/assignees/draft/review. A separate seam adds these without touching the daemon's `gh` tooling and keeps every side effect independently testable with a fake. | ## Reuse map (what this design builds on — no duplication) @@ -298,11 +297,14 @@ pub trait IdeaSource { } ``` -- **Spike:** a deterministic `FakeIdeaSource` (used by tests) + a marked `// FUTURE:` - `LlmIdeaSource` stub. The thread targets **ten** ideas per run (the design's fixed - batch), then applies dedup/portfolio filtering (below). +- **Production:** `AgenticIdeaSource` renders the generation prompt asset and runs + one agentic turn through the shared `AgentInvoker` seam (idle-liveness, no + wall-clock cap), fail-closed on a missing JSON envelope. **Tests:** a + deterministic `FakeIdeaSource`. The thread targets **ten** ideas per run (the + design's fixed batch), then applies dedup/portfolio filtering (below). - Each surviving `RawIdea` becomes a `CreativeIdea { status: New, links, context }`, - is persisted via `CreativeIdeaStore`, and (in the wired future) enqueued for review. + is persisted via `CreativeIdeaStore`, and is driven through the review-and-route + pipeline by the same tick. ## Reviewer pipeline @@ -455,10 +457,11 @@ pub trait IdeaGhClient { } ``` -- **Spike:** `FakeIdeaGhClient` records calls for assertions; the real subprocess impl - (`gh issue create --label ... --assignee ...`, `gh pr ready --undo`, `gh pr edit - --add-label`, `gh pr edit --add-reviewer`) is a marked `// FUTURE:` stub that reuses - the `RealGhClient` subprocess pattern from `src/stewardship/gh_client.rs`. +- **Production:** `RealIdeaGhClient` shells out via pure argv builders + (`gh issue create --label … --assignee …`, `gh pr ready --undo`, `gh pr edit + --add-label`, `gh pr edit --add-reviewer`), reusing the `RealGhClient` + subprocess pattern from `src/stewardship/gh_client.rs`. **Tests:** + `FakeIdeaGhClient` records calls for assertions. - **Never** emits `--admin` or `--no-verify`; a unit test asserts the constructed argument vectors contain neither flag. @@ -561,8 +564,8 @@ remain stable are three; each has an explicit version discipline: | **Node-type sentinel** (`trigger_condition = "creative-idea"`) | Never renamed — it is the retrieval key for already-stored rows | Literal `const CREATIVE_IDEA_TRIGGER`; a rename is a breaking migration, not an edit | | **Operator / automation surface** (`SIMARD_CREATIVE_IDEAS_*` env vars; labels `creative-idea`, `creative-idea-needs-human-review`; assignee `rysweet`) | Names are the operator + GitHub-automation contract; treated as stable identifiers | Centralized as `const`s in `CreativeIdeasConfig`; changes are documented, breaking changes | -- **Trait/type API** — during the spike the Rust surface is `#![allow(dead_code)]` - and carries **no** semver promise yet. The `Reviewer`, `FeedbackSynthesizer`, +- **Trait/type API** — the Rust surface carries `#![allow(dead_code)]` + and **no** semver promise yet. The `Reviewer`, `FeedbackSynthesizer`, `IdeaSource`, and `IdeaGhClient` traits are the intended long-term extension seams, so future capability is added via **new adapter impls** or **new trait methods with defaults**, never by changing an existing method signature. @@ -591,6 +594,56 @@ remain stable are three; each has an explicit version discipline: | **Dry-run honored** | The global `ThreadContext.dry_run` switch suppresses every destructive side-effect (goal/issue/PR writes); a dry-run tick still generates, reviews, and logs but writes nothing external. | `tick` checks `ctx.dry_run` before routing; routing seams are skipped. | | **Cooperative shutdown** | The thread observes `ThreadContext.shutdown` (the scheduler's cancellation flag) and returns promptly between pipeline stages instead of blocking a daemon stop. | `tick` polls `ctx.shutdown` between generate → review → route. | +## Daemon wiring + +The generator is registered from the OODA daemon's cognitive-thread setup +(`src/operator_commands_ooda/daemon/mod.rs`), alongside the Overseer and Journal +threads, and ticks on its configured cadence — it is not a separate process or a +"Bridge." + +**Runtime gate.** The generic cognitive-thread scheduler master switch +(`SIMARD_COGNITIVE_THREADS_ENABLED`) is default-OFF and owns only the +maintenance / engineer-log threads. Creative Ideas is **not** behind it: the +daemon builds the `Mind` runtime when *either* that switch is truthy **or** +`CreativeIdeasConfig::from_env().enabled()` is true (default). This keeps the +existing threads' gating and timing byte-for-byte unchanged while letting the +default-ON Creative Ideas thread run on a stock deployment. + +**Registration seam.** The daemon registers via +`register_creative_ideas_if_enabled(&mut mind, &cfg) -> bool`, which registers +only when `cfg.enabled()` and returns whether it did — so "opted out ⇒ not +registered" is a direct unit assertion (via `mind.health()` / `mind.len()`). + +**Startup log line** (mirrors the Journal thread): + +```text +[simard] OODA daemon: creative-ideas thread ENABLED (default) (interval = 86400s; SIMARD_CREATIVE_IDEAS_ENABLED opt-out) +``` + +or, when opted out: + +```text +[simard] OODA daemon: creative-ideas thread DISABLED (SIMARD_CREATIVE_IDEAS_ENABLED opt-out) +``` + +**Per-tick log line** (through the shared scheduler prefix, whenever the thread +actually runs — due on the first cycle after startup, then every interval): + +```text +[simard] cognitive-thread: creative_ideas: generated 10 idea(s), 8 survived dedup, 8 persisted, 8 reviewed (2 → goal, 1 → issue), 0 review error(s) +``` + +**Isolation.** Because generation + review touch the network with no wall-clock +cap, the tick runs on a background thread with an overlap guard (a slow tick is +dropped rather than stacked) and panic isolation, so it can never stall or crash +the authoritative OODA loop. Per-idea failures are logged and never abort the +batch; the tick is total (`ThreadOutcome::failed`, never `Err`/panic). + +**Deployment.** The `simard-ooda` systemd unit (`scripts/simard-ooda.service`) +runs the subsystem on by default; the operator opts out with a drop-in setting +`SIMARD_CREATIVE_IDEAS_ENABLED=0`. See +[Configure and operate the Creative Ideas thread](../howto/configure-creative-ideas-thread.md#daemon-wiring-startup-how-the-operator-sees-it-run). + ## Configuration & gating | Env var | Default | Effect | @@ -611,14 +664,19 @@ convention). Spans/metrics keyed on the stable thread id `creative_ideas` and re ids (`crusty_old_engineer`, `philosophy_guardian`, `measurability`, `idea_feedback_synthesis`): ideas generated, deduped, per-verdict counts, routed→goal / routed→issue / pr-gated, and status-transition events. These reuse the existing -cognitive-thread telemetry surface (`src/cognitive_threads/telemetry.rs`). +cognitive-thread telemetry surface (`src/cognitive_threads/telemetry.rs`). The daemon +also emits a dedicated startup line (`… creative-ideas thread ENABLED (default) …`) and +a per-tick summary (`[simard] cognitive-thread: creative_ideas: generated N idea(s) …`), +which surface in `journalctl`, the Overseer activity feed, and the dashboard/journal — +see [Daemon wiring](#daemon-wiring). ## Module layout & scaffold-vs-future ``` src/cognitive_memory/creative_idea.rs # CreativeIdea, IdeaStatus (+ try_transition), # MemoryLink, CreativeIdeaStore + Prospective adapter -src/cognitive_threads/threads/creative_ideas.rs # CreativeIdeasThread (CognitiveThread, gated OFF), +src/cognitive_threads/threads/creative_ideas.rs # CreativeIdeasThread (CognitiveThread, default-ON), + # register + register_creative_ideas_if_enabled, # GenerationInputs, IdeaSource trait + FakeIdeaSource src/creative_ideas/ mod.rs # CreativeIdeasConfig::from_env (gating) @@ -673,14 +731,26 @@ portfolio/novelty scoring). `creative-idea-needs-human-review`, and requests review from `rysweet`; a test asserts the constructed gh args contain **no** `--admin` / `--no-verify`. 7. **Dedup** — a near-duplicate of a prior idea is rejected. -8. **Off-by-default** — a default `CreativeIdeasConfig` is disabled and the thread's - `enabled()` returns `false`. +8. **Default-ON, opt-out** — a default `CreativeIdeasConfig` is **enabled** + (`enabled()` returns `true`), and only an explicit falsey value + (`0`/`false`/`no`/`off`) via the env resolver opts out. 9. **Error contracts** — an illegal `try_transition` returns `InvalidIdeaTransition { from, to }`; a payload that fails to parse (bad JSON, or a `payload_version` newer than the reader) returns `InvalidCreativeIdeaRecord`, and an unknown `IdeaStatus`/`ReviewVerdict` string is rejected (fail-closed), never defaulted. 10. **Tick is total** — a `tick` whose idea source/reviewer returns `Err` yields `ThreadOutcome::failed` (not a panic, not `Err`), leaving the daemon unaffected. +11. **Wired tick — end to end** — with the thread enabled and injected fakes + (`FakeIdeaSource`, stub reviewers, `DefaultSynthesizer`, in-memory `GoalStore`, + `FakeIdeaGhClient`), one `tick` persists ideas into prospective memory, runs the + four-reviewer pipeline, and produces routing outcomes — an accepted idea lands a + goal in the store, a human-review idea lands an issue in the fake `gh` client. +12. **Daemon registration — enabled** — `register_creative_ideas_if_enabled` registers + the `creative_ideas` thread with the `Mind` when the config is enabled (asserted via + `mind.health()` / `mind.len()`) and returns `true`. +13. **Daemon registration — disabled** — with `SIMARD_CREATIVE_IDEAS_ENABLED=0` (via the + env resolver), `register_creative_ideas_if_enabled` does **not** register the thread + and returns `false`. ## Phased roadmap diff --git a/docs/howto/configure-creative-ideas-thread.md b/docs/howto/configure-creative-ideas-thread.md index 9e612586..11800dcc 100644 --- a/docs/howto/configure-creative-ideas-thread.md +++ b/docs/howto/configure-creative-ideas-thread.md @@ -65,6 +65,78 @@ If instead you want to add an unrelated scheduled process, see *engineer* concurrency, that is the AIMD action-slot scaler (`SIMARD_MAX_CONCURRENT_ACTIONS`), not this thread. +## Daemon wiring & startup (how the operator sees it run) + +The thread is **wired into the running OODA daemon** — it is not something you +start separately. At startup the daemon builds the cognitive-thread runtime, +registers the Creative Ideas thread through +`register_creative_ideas_if_enabled(&mut mind, &cfg)`, and then ticks it on its +configured cadence alongside the Overseer and Journal threads. + +### It runs even when the rest of the cognitive-thread scheduler is off + +The generic cognitive-thread scheduler master switch +(`SIMARD_COGNITIVE_THREADS_ENABLED`, which turns on the maintenance and +engineer-log threads) is **default-OFF**. Creative Ideas is **not** gated behind +it. The daemon builds the `Mind` runtime whenever *either* the generic scheduler +**or** Creative Ideas is enabled: + +```text +runtime built ⇔ SIMARD_COGNITIVE_THREADS_ENABLED is truthy + OR CreativeIdeasConfig::from_env().enabled() (default true) +``` + +So on a stock deployment — with `SIMARD_COGNITIVE_THREADS_ENABLED` unset — the +maintenance/engineer-log threads stay off, but the Creative Ideas thread is +still registered and runs. Set `SIMARD_CREATIVE_IDEAS_ENABLED=0` to opt the +Creative Ideas thread (and, if nothing else needs the runtime, the runtime +itself) out. + +### Startup log line + +The daemon emits one dedicated line at startup, mirroring the Overseer and +Journal threads, so you can confirm the wiring at a glance: + +```text +[simard] OODA daemon: creative-ideas thread ENABLED (default) (interval = 86400s; SIMARD_CREATIVE_IDEAS_ENABLED opt-out) +``` + +When opted out you instead see: + +```text +[simard] OODA daemon: creative-ideas thread DISABLED (SIMARD_CREATIVE_IDEAS_ENABLED opt-out) +``` + +Confirm it under systemd with: + +```bash +journalctl -u simard-ooda --no-pager | grep 'creative-ideas thread' +``` + +### Per-tick log line + +Every time the thread actually runs (it is *due* on the first cycle after +startup, then every `SIMARD_CREATIVE_IDEAS_INTERVAL_SECS`), the scheduler +surfaces its summary through the shared cognitive-thread log prefix: + +```text +[simard] cognitive-thread: creative_ideas: generated 10 idea(s), 8 survived dedup, 8 persisted, 8 reviewed (2 → goal, 1 → issue), 0 review error(s) +``` + +A dry-run tick reads `generated (dry-run) …` and writes nothing external. Follow +the activity live with: + +```bash +journalctl -u simard-ooda -f | grep -E 'creative[-_]ideas' +``` + +The same per-tick summary and the thread's heartbeat (`last_run` / `next_run` / +consecutive-error count) also flow into the Overseer activity feed and the +dashboard/journal, so a healthy Creative Ideas thread is visible without reading +raw logs. Because generation + review touch the network, the tick runs on a +background thread with an overlap guard and panic isolation — a slow or failing +tick can never stall or crash the authoritative OODA loop. + ## Turn it off (opt out) The subsystem is **default-ON**. `CreativeIdeasConfig::from_env()` is the single @@ -96,6 +168,26 @@ export SIMARD_DAILY_BUDGET_USD=5.00 # reused budget ceiling (exis The truthiness check mirrors `overseer_acting_enabled()` — `1`, `true`, `yes`, etc. count as on; unset or `0` is off. +### Opting out under systemd + +The deployed daemon runs from the `simard-ooda` systemd unit +(`scripts/simard-ooda.service`), where the subsystem is on by default. To opt a +deployment out, add a drop-in (or edit the unit) so the master switch is falsey, +then reload and restart: + +```bash +sudo systemctl edit simard-ooda # creates a drop-in override +# In the editor add: +# [Service] +# Environment=SIMARD_CREATIVE_IDEAS_ENABLED=0 +sudo systemctl daemon-reload +sudo systemctl restart simard-ooda +journalctl -u simard-ooda --no-pager | grep 'creative-ideas thread' # now DISABLED +``` + +The unit itself documents this opt-out (and the least-privilege `gh`/`GITHUB_TOKEN` +scope the routing step needs) in a comment block near the other feature switches. + ## What one generation tick does When enabled and due, `CreativeIdeasThread::tick` runs this pipeline. It is @@ -271,8 +363,10 @@ impl Reviewer for SecuritySmellReviewer { ``` Production reviewer adapters invoke an amplihack skill/agent through the -`invoke_agent(prompt) -> String` seam; the prompt bodies are marked `// FUTURE:` -until the real reviewers are wired (M3). +`AgentInvoker::invoke(prompt) -> String` seam (the same blessed session path the +OODA brain uses). The three vetting reviewers ship with real prompt assets +(`prompt_assets/simard/creative_ideas_review_*.md`); tests inject a deterministic +fake invoker so no network is touched. ## Test it (no network, all fakes) diff --git a/docs/reference/creative-ideas-api.md b/docs/reference/creative-ideas-api.md index b61e11a7..0e450281 100644 --- a/docs/reference/creative-ideas-api.md +++ b/docs/reference/creative-ideas-api.md @@ -9,12 +9,13 @@ description: > SynthesisOutcome, SuccessMetric), the routing functions (route_idea_to_goal, route_idea_to_issue, mark_idea_pr) with the IdeaGhClient seam, the two new SimardError variants, configuration (SIMARD_CREATIVE_IDEAS_*), telemetry, and - the test fakes. Subsystem is gated OFF by default. -last_updated: 2026-07-05 + the test fakes. Subsystem is wired into the running daemon — default-ON, opt-out + via SIMARD_CREATIVE_IDEAS_ENABLED. +last_updated: 2026-07-06 review_schedule: as-needed owner: simard doc_type: reference -status: spike — typed foundation + tests (OFF by default) +status: implemented — wired into the daemon (default-ON, opt-out) related: - ../design/creative-ideas-thread.md - ../howto/configure-creative-ideas-thread.md @@ -86,26 +87,35 @@ asserted in tests. ## Configuration `CreativeIdeasConfig::from_env()` is the single source of truth for gating and -cadence. A default-constructed config is **disabled**. +cadence. A default-constructed config is **enabled** (default-ON, opt-out), +consistent with the Overseer and Journal threads. | Env var | Default | Effect | |---------|---------|--------| -| `SIMARD_CREATIVE_IDEAS_ENABLED` | `false` | Master switch. When false the thread never ticks and nothing is generated or routed. | +| `SIMARD_CREATIVE_IDEAS_ENABLED` | `true` | Master switch (default-ON). Only an explicit falsey value (`0`/`false`/`no`/`off`) opts out ⇒ the thread never ticks and nothing is generated or routed. | | `SIMARD_CREATIVE_IDEAS_INTERVAL_SECS` | `86400` | Generator cadence — a large (≥ 24 h) observation window. | | `SIMARD_CREATIVE_IDEAS_BATCH` | `10` | Ideas targeted per run (the design's fixed batch of ten). | | `SIMARD_DAILY_BUDGET_USD` | *(existing)* | Reused for budget-awareness before an expensive tick. | ```rust pub struct CreativeIdeasConfig { - pub enabled: bool, // SIMARD_CREATIVE_IDEAS_ENABLED (default false) + pub enabled: bool, // SIMARD_CREATIVE_IDEAS_ENABLED (default true; opt-out) pub interval_secs: u64, // SIMARD_CREATIVE_IDEAS_INTERVAL_SECS (default 86_400) pub batch: usize, // SIMARD_CREATIVE_IDEAS_BATCH (default 10) } +impl Default for CreativeIdeasConfig { + /// A default config is ENABLED (default-ON, opt-out). + fn default() -> Self; +} + impl CreativeIdeasConfig { - /// Parse from the environment. Truthy check mirrors overseer_acting_enabled(). + /// Parse from the environment. Gate mirrors overseer_acting_enabled(): + /// unset/empty ⇒ enabled; only an explicit falsey value opts out. pub fn from_env() -> Self; - /// True only when the master switch is truthy. + /// Parse from an arbitrary env resolver (test seam). + pub fn from_lookup(lookup: impl Fn(&str) -> Option) -> Self; + /// True unless the master switch was explicitly set to a falsey value. pub fn enabled(&self) -> bool; } ``` @@ -288,15 +298,42 @@ payload fidelity (`links`, `context`, `success_metric`). | `kind()` | `ThreadKind::BackgroundThought` | Reuses the reserved variant; no enum change. | | `policy()` | `SchedulePolicy::Interval(Duration::from_secs(interval_secs))` | Default 24 h; reserved to become `Adaptive` later. | | `priority()` | `Priority::Low` | Never competes with OODA. | -| `enabled()` | `config.enabled()` → **`false` by default** | A disabled thread never ticks (scheduler contract), so the subsystem is inert unless explicitly turned on. | +| `enabled()` | `config.enabled()` → **`true` by default** | Default-ON, opt-out. A disabled thread never ticks (scheduler contract), so an opted-out subsystem is inert. | | `tick(&mut ThreadContext)` | `ThreadOutcome` | **Never returns `Err`** — internal errors are folded into `ThreadOutcome::failed(reason, elapsed)`. Uses `ctx.now_epoch` (injected clock) and may `block_on` `ctx.runtime`. Honors `ctx.shutdown` (returns promptly between stages) and `ctx.dry_run` (performs no goal/issue/PR side-effect). | ```rust -/// Register the thread with the Mind scheduler. NOT called from the live -/// daemon during the spike; the call site is a marked `// FUTURE:` seam. +/// Register the Creative Ideas thread with the Mind scheduler, using the +/// production idea source + review/route pipeline. Called from the daemon's +/// cognitive-thread setup behind SIMARD_CREATIVE_IDEAS_ENABLED (default-ON). pub fn register(mind: &mut Mind, config: CreativeIdeasConfig); + +/// Register only when `cfg.enabled()`; returns whether it registered. The daemon +/// calls this so "disabled ⇒ not registered" is directly unit-testable. +pub fn register_creative_ideas_if_enabled(mind: &mut Mind, cfg: &CreativeIdeasConfig) -> bool; ``` +### Daemon registration + +The thread is registered from the OODA daemon's cognitive-thread setup +(`src/operator_commands_ooda/daemon/mod.rs`), next to the Overseer and Journal +threads. Two facts make it observable and testable: + +- **Runtime gate.** The daemon builds the `Mind` runtime when *either* the + generic scheduler master switch (`SIMARD_COGNITIVE_THREADS_ENABLED`, + default-OFF, which owns the maintenance/engineer-log threads) **or** Creative + Ideas is enabled — so the default-ON Creative Ideas thread runs even on a stock + deployment where the generic scheduler is off. Registration goes through + `register_creative_ideas_if_enabled`, so an opted-out config leaves the thread + unregistered (asserted by a test via `mind.health()` / `mind.len()`). +- **Startup + per-tick logs.** At startup the daemon emits one dedicated line + mirroring the Journal thread: + `[simard] OODA daemon: creative-ideas thread ENABLED (default) (interval = {n}s; SIMARD_CREATIVE_IDEAS_ENABLED opt-out)` + (or `… DISABLED (SIMARD_CREATIVE_IDEAS_ENABLED opt-out)`). Each run surfaces its + summary through the shared scheduler prefix: + `[simard] cognitive-thread: creative_ideas: generated N idea(s), …`. + The tick executes on a background thread with an overlap guard and panic + isolation, so it cannot stall or crash the authoritative OODA loop. + ### Thread inputs (the observation window) `tick` assembles a typed `GenerationInputs` from **read-only** sources: @@ -324,13 +361,17 @@ pub trait IdeaSource { } ``` -- **Available now:** `FakeIdeaSource` (deterministic; used by tests). -- **`// FUTURE:`** `LlmIdeaSource` (the production generator). +- **Production:** `AgenticIdeaSource` (`src/creative_ideas/source.rs`) — renders + the generation prompt asset, runs one agentic turn through the shared + `AgentInvoker` seam (idle-liveness, no wall-clock cap), and parses the JSON + envelope into `RawIdea`s. Fail-closed: a response with no JSON envelope is a + hard error, never a silent empty batch. Built by `register`/`from_env`. +- **Tests:** `FakeIdeaSource` (deterministic; `with_ideas` / `failing`). The thread targets **ten** ideas per run, applies dedup + portfolio filtering -(below), then persists each survivor as `CreativeIdea { status: New, links, -context }` via `CreativeIdeaStore` and (in the wired future) enqueues it for -review. +(below), persists each survivor as `CreativeIdea { status: New, links, +context }` via `CreativeIdeaStore`, then drives it through the review-and-route +pipeline. ## Reviewer pipeline @@ -375,9 +416,11 @@ The four reviewers and their stable ids: | 3 | `measurability` | agent adapter (NEW) | Emits a concrete `SuccessMetric`, tied where relevant to existing self-metrics (`recall_precision_at_k`, distill fact-yield, reasoner-reliability). This metric is the only thing that can later move the idea to `ImplementationCompleted`. | | 4 | `idea_feedback_synthesis` | synthesis step | Reads **all** reviews + context, summarizes next steps, and **sets the status** per the state machine. | -Production adapters shape an amplihack skill/agent invocation via the -`invoke_agent(prompt) -> String` seam (the prompt bodies are marked `// FUTURE:` -stubs); all tests inject deterministic fakes. +Production adapters shape an amplihack skill/agent invocation via the shared +`AgentInvoker::invoke(prompt) -> String` seam (the same blessed session path the +OODA brain uses); the three vetting reviewers ship with real prompt assets +(`prompt_assets/simard/creative_ideas_review_*.md`). All tests inject +deterministic fake invokers. ### Synthesis @@ -575,7 +618,7 @@ With no wire protocol, three externally-observable contracts must remain stable: | Node-type sentinel (`trigger_condition = "creative-idea"`) | Never renamed — it is the retrieval key for stored rows | Literal `const CREATIVE_IDEA_TRIGGER`; a rename is a breaking migration | | Operator / automation surface (`SIMARD_CREATIVE_IDEAS_*`, labels, assignee) | Stable identifiers | Centralized `const`s in `CreativeIdeasConfig` | -- **Trait/type API** — during the spike the Rust surface is +- **Trait/type API** — the Rust surface is `#![allow(dead_code)]` and carries no semver promise. `Reviewer`, `FeedbackSynthesizer`, `IdeaSource`, and `IdeaGhClient` are the long-term extension seams: extend via new impls or new trait methods with defaults, diff --git a/scripts/simard-ooda.service b/scripts/simard-ooda.service index 0c32e517..04decca4 100644 --- a/scripts/simard-ooda.service +++ b/scripts/simard-ooda.service @@ -26,6 +26,18 @@ Environment=RUST_LOG=info Environment=SIMARD_GIT_GUARDRAILS=enabled Environment=SIMARD_GIT_PROTECTED_REPOS=/home/azureuser/src +# Creative Ideas background thread (issue #2647) — an idea-generation cognitive +# thread wired into this daemon. It is DEFAULT-ON (opt-out); leave it unset to +# run it. To disable it on this deployment, uncomment the line below: +# Environment=SIMARD_CREATIVE_IDEAS_ENABLED=0 +# Its routing step files a goal, or (for human-review ideas) a labeled + owner- +# tagged GitHub issue via `gh`, and gates idea-derived PRs (draft + blocking +# label + owner review — never --admin/--no-verify). That needs a least-privilege +# token: issues:write + pull_requests:write on the single target repo only. +# Optional tuning (defaults shown): +# Environment=SIMARD_CREATIVE_IDEAS_INTERVAL_SECS=86400 +# Environment=SIMARD_CREATIVE_IDEAS_BATCH=10 + # Resource limits LimitNOFILE=65536 MemoryMax=4G diff --git a/src/cognitive_memory/creative_idea.rs b/src/cognitive_memory/creative_idea.rs index 4574404f..547f5c71 100644 --- a/src/cognitive_memory/creative_idea.rs +++ b/src/cognitive_memory/creative_idea.rs @@ -12,8 +12,9 @@ //! | `action_on_trigger` | JSON payload (versioned) with status/context/links/reviews/metric | //! | `priority` | derived from portfolio/risk | //! -//! The subsystem is a **spike**: this file is real, tested typed foundation but -//! nothing here is wired into the daemon and the generator is gated OFF (see +//! The subsystem is **live**: this file is the real, tested typed foundation +//! for ideas the daemon's wired Creative Ideas thread generates, reviews, and +//! routes (default-ON, opt-out via `SIMARD_CREATIVE_IDEAS_ENABLED`; see //! [`crate::creative_ideas`]). `status` changes **only** through //! [`CreativeIdea::try_transition`], which validates every edge. #![allow(dead_code)] diff --git a/src/cognitive_threads/threads/creative_ideas.rs b/src/cognitive_threads/threads/creative_ideas.rs index f272eee4..d0f037a5 100644 --- a/src/cognitive_threads/threads/creative_ideas.rs +++ b/src/cognitive_threads/threads/creative_ideas.rs @@ -1,11 +1,13 @@ //! The Creative Ideas generator thread (design spike #2419). //! //! `CreativeIdeasThread` implements [`CognitiveThread`] and reuses -//! [`ThreadKind::BackgroundThought`] (Decision 2). It is **gated OFF by -//! default** (`enabled()` reads [`CreativeIdeasConfig::enabled`], default -//! `false`), so the scheduler never ticks it unless explicitly turned on. It is -//! **not registered** with the `Mind` during the spike — [`register`] is a -//! marked `// FUTURE:` seam. +//! [`ThreadKind::BackgroundThought`] (Decision 2). It is **default-ON, opt-out** +//! (`enabled()` reads [`CreativeIdeasConfig::enabled`], default `true`), so the +//! scheduler ticks it on a stock deployment unless `SIMARD_CREATIVE_IDEAS_ENABLED` +//! is set to a falsey value. The OODA daemon registers it via +//! [`register_creative_ideas_if_enabled`], independent of the generic +//! `SIMARD_COGNITIVE_THREADS_ENABLED` master switch — consistent with the +//! default-ON Overseer/Journal threads. //! //! `tick` is **total by contract**: every internal `Err` is caught, //! `tracing::warn!`-logged with the stable `creative_ideas` id, and returned as @@ -412,7 +414,7 @@ impl CognitiveThread for CreativeIdeasThread { fn tick(&mut self, ctx: &mut ThreadContext<'_>) -> ThreadOutcome { let start = Instant::now(); - // Gated OFF: never do work unless explicitly enabled. + // Opt-out gate (default-ON): do no work when explicitly disabled. if !self.cfg.enabled() { return ThreadOutcome::skipped(); } @@ -495,9 +497,10 @@ fn raw_to_creative_idea(raw: &RawIdea, inputs: &GenerationInputs, now_epoch: u64 /// Register the Creative Ideas thread with the `Mind` scheduler. /// -/// Called from the daemon startup path behind `SIMARD_CREATIVE_IDEAS_ENABLED` -/// (default-ON, opt-out). Builds the production idea source + review/route -/// pipeline; the thread's `enabled()` gate still makes an opted-out thread inert. +/// The daemon reaches this through [`register_creative_ideas_if_enabled`], which +/// owns the default-ON/opt-out gate. Builds the production idea source + +/// review/route pipeline; the thread's `enabled()` gate is defence-in-depth that +/// still makes an opted-out thread inert even if it were registered directly. pub fn register(mind: &mut Mind, config: CreativeIdeasConfig) { let thread = CreativeIdeasThread::with_pipeline( config, @@ -506,3 +509,75 @@ pub fn register(mind: &mut Mind, config: CreativeIdeasConfig) { ); mind.register(Box::new(thread)); } + +/// Register the Creative Ideas thread **only when its config gate is ON**, +/// returning whether it was registered. +/// +/// This is the daemon's startup seam (issue #2647). It mirrors the +/// Overseer/Journal default-ON opt-out pattern: an opted-out subsystem registers +/// nothing, so "opted out ⇒ not registered" is a direct `mind.len()` / +/// `mind.health()` assertion rather than relying on a registered-but-inert +/// thread. The gate is independent of the generic `SIMARD_COGNITIVE_THREADS_ENABLED` +/// master switch. +pub fn register_creative_ideas_if_enabled(mind: &mut Mind, cfg: &CreativeIdeasConfig) -> bool { + if cfg.enabled() { + register(mind, cfg.clone()); + true + } else { + false + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn register_if_enabled_registers_when_default_on() { + let mut mind = Mind::new(); + let registered = + register_creative_ideas_if_enabled(&mut mind, &CreativeIdeasConfig::default()); + assert!(registered, "default-ON config must register the thread"); + assert_eq!(mind.len(), 1); + } + + #[test] + fn register_if_enabled_skips_when_opted_out() { + let mut mind = Mind::new(); + let cfg = CreativeIdeasConfig { + enabled: false, + ..CreativeIdeasConfig::default() + }; + let registered = register_creative_ideas_if_enabled(&mut mind, &cfg); + assert!(!registered, "opted-out config must register nothing"); + assert!(mind.is_empty()); + } + + #[test] + fn register_if_enabled_registers_when_env_unset() { + // Env seam (hermetic — no real process env): an unset + // `SIMARD_CREATIVE_IDEAS_ENABLED` is default-ON, so the daemon's startup + // seam registers the thread. Drives the full env → gate → register path. + let cfg = CreativeIdeasConfig::from_lookup(|_| None); + let mut mind = Mind::new(); + let registered = register_creative_ideas_if_enabled(&mut mind, &cfg); + assert!(registered, "unset env is default-ON ⇒ thread registered"); + assert_eq!(mind.len(), 1); + } + + #[test] + fn register_if_enabled_skips_when_disabled_via_env() { + // Env seam (hermetic): `SIMARD_CREATIVE_IDEAS_ENABLED=0` opts out, so the + // startup seam registers nothing — "disabled via env ⇒ not registered". + let cfg = CreativeIdeasConfig::from_lookup(|k| { + (k == crate::creative_ideas::ENABLED_ENV).then(|| "0".to_string()) + }); + let mut mind = Mind::new(); + let registered = register_creative_ideas_if_enabled(&mut mind, &cfg); + assert!( + !registered, + "SIMARD_CREATIVE_IDEAS_ENABLED=0 ⇒ nothing registered" + ); + assert!(mind.is_empty()); + } +} diff --git a/src/cognitive_threads/threads/mod.rs b/src/cognitive_threads/threads/mod.rs index 5fe54abb..362492ef 100644 --- a/src/cognitive_threads/threads/mod.rs +++ b/src/cognitive_threads/threads/mod.rs @@ -8,9 +8,9 @@ pub mod engineer_log_analysis; pub mod maintenance; pub mod ooda; -// Issue #2419 (design spike): the Creative Ideas generator thread — reuses -// `ThreadKind::BackgroundThought`, gated OFF by default, not registered with -// the `Mind` during the spike. +// Issue #2419 (design spike) / #2647 (wiring): the Creative Ideas generator +// thread — reuses `ThreadKind::BackgroundThought`, default-ON opt-out, and +// registered with the `Mind` by the OODA daemon at startup. pub mod creative_ideas; pub use creative_ideas::CreativeIdeasThread; diff --git a/src/creative_ideas/mod.rs b/src/creative_ideas/mod.rs index 9597bcf1..c6ba2f56 100644 --- a/src/creative_ideas/mod.rs +++ b/src/creative_ideas/mod.rs @@ -1,16 +1,16 @@ -//! Creative Ideas subsystem (design spike #2419) — an idea-generation subsystem +//! Creative Ideas subsystem (issue #2647) — an idea-generation subsystem //! that primes a pool of candidate self-improvement ideas inside the single //! Simard brain. //! -//! **Status: design + typed foundation + tests, gated OFF.** This module owns -//! the reviewer-pipeline trait and adapters ([`reviewers`]), the synthesis step -//! ([`synthesis`]), the routing functions ([`routing`]), the dedup/portfolio/ -//! budget helpers ([`dedup`]), and the config flag ([`CreativeIdeasConfig`]). -//! The `CreativeIdea` prospective type lives in -//! [`crate::cognitive_memory::creative_idea`] and the generator thread in -//! [`crate::cognitive_threads::threads::creative_ideas`]. Nothing here is wired -//! into the daemon; the generator is OFF by default behind -//! `SIMARD_CREATIVE_IDEAS_ENABLED`. The operator redeploys after merge. +//! **Status: live, default-ON opt-out.** This module owns the reviewer-pipeline +//! trait and adapters ([`reviewers`]), the synthesis step ([`synthesis`]), the +//! routing functions ([`routing`]), the dedup/portfolio/budget helpers +//! ([`dedup`]), and the config flag ([`CreativeIdeasConfig`]). The `CreativeIdea` +//! prospective type lives in [`crate::cognitive_memory::creative_idea`] and the +//! generator thread in [`crate::cognitive_threads::threads::creative_ideas`], +//! which the OODA daemon registers on startup — default-ON, opt-out via +//! `SIMARD_CREATIVE_IDEAS_ENABLED`, consistent with the Overseer/Journal threads +//! and independent of the generic `SIMARD_COGNITIVE_THREADS_ENABLED` switch. //! //! No type or module is a transport shim — this is one brain with //! cognitive threads and reviewers, not a separate service. diff --git a/src/lib.rs b/src/lib.rs index 64d0be99..74579e7a 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -45,10 +45,12 @@ mod copilot_task_submit; // `MeetingBackend`; `SignalConversation` (feature-gated below) is a third. pub mod conversation_channel; pub mod cost_tracking; -// Issue #2419 (design spike): the Creative Ideas subsystem — an idea-generation -// cognitive thread + four-reviewer pipeline that primes a pool of candidate -// self-improvement ideas. Typed foundation + tests only; gated OFF by default -// (`SIMARD_CREATIVE_IDEAS_ENABLED`) and not wired into the daemon. See +// Issue #2419 (design spike) / #2647 (wiring): the Creative Ideas subsystem — an +// idea-generation cognitive thread + four-reviewer pipeline that primes a pool +// of candidate self-improvement ideas. Wired into the OODA daemon and +// **default-ON, opt-out** via `SIMARD_CREATIVE_IDEAS_ENABLED` (consistent with +// the Overseer/Journal threads, independent of the generic +// `SIMARD_COGNITIVE_THREADS_ENABLED` switch). See // `docs/design/creative-ideas-thread.md`. pub mod creative_ideas; pub mod disk_health; diff --git a/src/operator_commands_ooda/daemon/mod.rs b/src/operator_commands_ooda/daemon/mod.rs index eac6d26d..6eeaf290 100644 --- a/src/operator_commands_ooda/daemon/mod.rs +++ b/src/operator_commands_ooda/daemon/mod.rs @@ -575,20 +575,29 @@ pub fn run_ooda_daemon( ); // ------------------------------------------------------------------- - // ── Cognitive-thread scheduler (issue #2419) ─────────────────────── - // ADDITIVE and OFF by default. When SIMARD_COGNITIVE_THREADS_ENABLED is - // truthy, a `Mind` hosts NEW background cognitive threads (safe - // maintenance/cleanup + engineer-log analysis) on their own cadence, - // subsuming the ad-hoc periodic-task pattern for these threads. OODA - // itself stays driven by this loop's authoritative inline cycle below so - // its external cadence and side-effects are byte-for-byte preserved; the - // scheduler is invoked only AFTER the inline cycle and never gates it. + // ── Cognitive-thread scheduler (issue #2419 + #2647) ─────────────── + // ADDITIVE. A `Mind` hosts background cognitive threads on their own + // cadence, subsuming the ad-hoc periodic-task pattern. OODA itself stays + // driven by this loop's authoritative inline cycle below so its external + // cadence and side-effects are byte-for-byte preserved; the scheduler is + // invoked only AFTER the inline cycle and never gates it. + // + // Two INDEPENDENT gates share the one runtime: + // * SIMARD_COGNITIVE_THREADS_ENABLED (default-OFF) owns the maintenance + + // engineer-log threads — their gating and timing stay unchanged. + // * SIMARD_CREATIVE_IDEAS_ENABLED (default-ON, opt-out) owns the Creative + // Ideas generator (issue #2647), consistent with the default-ON + // Overseer/Journal threads — so it runs on a stock deployment WITHOUT + // the generic master switch. let cognitive_threads_enabled = std::env::var("SIMARD_COGNITIVE_THREADS_ENABLED") .ok() .map(|s| matches!(s.trim(), "1" | "true" | "TRUE" | "yes" | "on")) .unwrap_or(false); + let creative_ideas_cfg = crate::creative_ideas::CreativeIdeasConfig::from_env(); + let creative_ideas_enabled = creative_ideas_cfg.enabled(); let cognitive_repo_root = bridges.repo_root.clone(); - let cognitive_runtime = if cognitive_threads_enabled { + // Build the shared runtime when EITHER gate wants a thread to run. + let cognitive_runtime = if cognitive_threads_enabled || creative_ideas_enabled { match tokio::runtime::Builder::new_multi_thread() .worker_threads(1) .enable_all() @@ -608,19 +617,23 @@ pub fn run_ooda_daemon( }; let mut mind = crate::cognitive_threads::Mind::new(); if cognitive_runtime.is_some() { - mind.register(Box::new( - crate::cognitive_threads::MaintenanceThread::from_env(), - )); - mind.register(Box::new( - crate::cognitive_threads::EngineerLogAnalysisThread::from_env(), - )); - // Creative Ideas generator thread (issue #2606-adjacent): a divergent + // Maintenance + engineer-log stay behind the generic master switch so + // their default-OFF gating and timing are unchanged. + if cognitive_threads_enabled { + mind.register(Box::new( + crate::cognitive_threads::MaintenanceThread::from_env(), + )); + mind.register(Box::new( + crate::cognitive_threads::EngineerLogAnalysisThread::from_env(), + )); + } + // Creative Ideas generator thread (issue #2647): a divergent // idea-generation background thread, default-ON opt-out via - // `SIMARD_CREATIVE_IDEAS_ENABLED` (its `enabled()` gate keeps an - // opted-out thread inert). Reuses `ThreadKind::BackgroundThought`. - crate::cognitive_threads::threads::creative_ideas::register( + // `SIMARD_CREATIVE_IDEAS_ENABLED`, INDEPENDENT of the generic master + // switch above. The gate seam registers nothing when opted out. + crate::cognitive_threads::threads::creative_ideas::register_creative_ideas_if_enabled( &mut mind, - crate::creative_ideas::CreativeIdeasConfig::from_env(), + &creative_ideas_cfg, ); // NOTE: the Overseer M1 read-only observer sensor that previously // registered here (default-OFF, `SIMARD_OVERSEER_ENABLED` truthy) is @@ -637,6 +650,20 @@ pub fn run_ooda_daemon( ), ); } + // Creative-ideas startup line (mirrors the Journal thread) — logged + // unconditionally so the operator can confirm the gate from journalctl. + let creative_ideas_startup_line = if creative_ideas_enabled { + format!( + "[simard] OODA daemon: creative-ideas thread ENABLED (default) \ + (interval = {}s; SIMARD_CREATIVE_IDEAS_ENABLED opt-out)", + creative_ideas_cfg.interval_secs + ) + } else { + "[simard] OODA daemon: creative-ideas thread DISABLED \ + (SIMARD_CREATIVE_IDEAS_ENABLED opt-out)" + .to_string() + }; + daemon_log(&state_root, &creative_ideas_startup_line); // ── Acting Overseer co-process (issue #2539 wiring) ───────────────── // ADDITIVE and DEFAULT-ON: the daemon drives the Overseer's meta-OODA loop