Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
152 changes: 111 additions & 41 deletions docs/design/creative-ideas-thread.md

Large diffs are not rendered by default.

98 changes: 96 additions & 2 deletions docs/howto/configure-creative-ideas-thread.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,78 @@ If instead you want to add an unrelated scheduled process, see
*engineer* concurrency, that is the AIMD action-slot scaler
(`SIMARD_MAX_CONCURRENT_ACTIONS`), not this thread.

## Daemon wiring & startup (how the operator sees it run)

The thread is **wired into the running OODA daemon** — it is not something you
start separately. At startup the daemon builds the cognitive-thread runtime,
registers the Creative Ideas thread through
`register_creative_ideas_if_enabled(&mut mind, &cfg)`, and then ticks it on its
configured cadence alongside the Overseer and Journal threads.

### It runs even when the rest of the cognitive-thread scheduler is off

The generic cognitive-thread scheduler master switch
(`SIMARD_COGNITIVE_THREADS_ENABLED`, which turns on the maintenance and
engineer-log threads) is **default-OFF**. Creative Ideas is **not** gated behind
it. The daemon builds the `Mind` runtime whenever *either* the generic scheduler
**or** Creative Ideas is enabled:

```text
runtime built ⇔ SIMARD_COGNITIVE_THREADS_ENABLED is truthy
OR CreativeIdeasConfig::from_env().enabled() (default true)
```

So on a stock deployment — with `SIMARD_COGNITIVE_THREADS_ENABLED` unset — the
maintenance/engineer-log threads stay off, but the Creative Ideas thread is
still registered and runs. Set `SIMARD_CREATIVE_IDEAS_ENABLED=0` to opt the
Creative Ideas thread (and, if nothing else needs the runtime, the runtime
itself) out.

### Startup log line

The daemon emits one dedicated line at startup, mirroring the Overseer and
Journal threads, so you can confirm the wiring at a glance:

```text
[simard] OODA daemon: creative-ideas thread ENABLED (default) (interval = 86400s; SIMARD_CREATIVE_IDEAS_ENABLED opt-out)
```

When opted out you instead see:

```text
[simard] OODA daemon: creative-ideas thread DISABLED (SIMARD_CREATIVE_IDEAS_ENABLED opt-out)
```

Confirm it under systemd with:

```bash
journalctl -u simard-ooda --no-pager | grep 'creative-ideas thread'
```

### Per-tick log line

Every time the thread actually runs (it is *due* on the first cycle after
startup, then every `SIMARD_CREATIVE_IDEAS_INTERVAL_SECS`), the scheduler
surfaces its summary through the shared cognitive-thread log prefix:

```text
[simard] cognitive-thread: creative_ideas: generated 10 idea(s), 8 survived dedup, 8 persisted, 8 reviewed (2 → goal, 1 → issue), 0 review error(s)
```

A dry-run tick reads `generated (dry-run) …` and writes nothing external. Follow
the activity live with:

```bash
journalctl -u simard-ooda -f | grep -E 'creative[-_]ideas'
```

The same per-tick summary and the thread's heartbeat (`last_run` / `next_run` /
consecutive-error count) also flow into the Overseer activity feed and the
dashboard/journal, so a healthy Creative Ideas thread is visible without reading
raw logs. Because generation + review touch the network, the tick runs on a
background thread with an overlap guard and panic isolation — a slow or failing
tick can never stall or crash the authoritative OODA loop.

## Turn it off (opt out)

The subsystem is **default-ON**. `CreativeIdeasConfig::from_env()` is the single
Expand Down Expand Up @@ -96,6 +168,26 @@ export SIMARD_DAILY_BUDGET_USD=5.00 # reused budget ceiling (exis
The truthiness check mirrors `overseer_acting_enabled()` — `1`, `true`, `yes`,
etc. count as on; unset or `0` is off.

### Opting out under systemd

The deployed daemon runs from the `simard-ooda` systemd unit
(`scripts/simard-ooda.service`), where the subsystem is on by default. To opt a
deployment out, add a drop-in (or edit the unit) so the master switch is falsey,
then reload and restart:

```bash
sudo systemctl edit simard-ooda # creates a drop-in override
# In the editor add:
# [Service]
# Environment=SIMARD_CREATIVE_IDEAS_ENABLED=0
sudo systemctl daemon-reload
sudo systemctl restart simard-ooda
journalctl -u simard-ooda --no-pager | grep 'creative-ideas thread' # now DISABLED
```

The unit itself documents this opt-out (and the least-privilege `gh`/`GITHUB_TOKEN`
scope the routing step needs) in a comment block near the other feature switches.

## What one generation tick does

When enabled and due, `CreativeIdeasThread::tick` runs this pipeline. It is
Expand Down Expand Up @@ -271,8 +363,10 @@ impl Reviewer for SecuritySmellReviewer {
```

Production reviewer adapters invoke an amplihack skill/agent through the
`invoke_agent(prompt) -> String` seam; the prompt bodies are marked `// FUTURE:`
until the real reviewers are wired (M3).
`AgentInvoker::invoke(prompt) -> String` seam (the same blessed session path the
OODA brain uses). The three vetting reviewers ship with real prompt assets
(`prompt_assets/simard/creative_ideas_review_*.md`); tests inject a deterministic
fake invoker so no network is touched.

## Test it (no network, all fakes)

Expand Down
83 changes: 63 additions & 20 deletions docs/reference/creative-ideas-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,13 @@ description: >
SynthesisOutcome, SuccessMetric), the routing functions (route_idea_to_goal,
route_idea_to_issue, mark_idea_pr) with the IdeaGhClient seam, the two new
SimardError variants, configuration (SIMARD_CREATIVE_IDEAS_*), telemetry, and
the test fakes. Subsystem is gated OFF by default.
last_updated: 2026-07-05
the test fakes. Subsystem is wired into the running daemon — default-ON, opt-out
via SIMARD_CREATIVE_IDEAS_ENABLED.
last_updated: 2026-07-06
review_schedule: as-needed
owner: simard
doc_type: reference
status: spiketyped foundation + tests (OFF by default)
status: implementedwired into the daemon (default-ON, opt-out)
related:
- ../design/creative-ideas-thread.md
- ../howto/configure-creative-ideas-thread.md
Expand Down Expand Up @@ -86,26 +87,35 @@ asserted in tests.
## Configuration

`CreativeIdeasConfig::from_env()` is the single source of truth for gating and
cadence. A default-constructed config is **disabled**.
cadence. A default-constructed config is **enabled** (default-ON, opt-out),
consistent with the Overseer and Journal threads.

| Env var | Default | Effect |
|---------|---------|--------|
| `SIMARD_CREATIVE_IDEAS_ENABLED` | `false` | Master switch. When false the thread never ticks and nothing is generated or routed. |
| `SIMARD_CREATIVE_IDEAS_ENABLED` | `true` | Master switch (default-ON). Only an explicit falsey value (`0`/`false`/`no`/`off`) opts out ⇒ the thread never ticks and nothing is generated or routed. |
| `SIMARD_CREATIVE_IDEAS_INTERVAL_SECS` | `86400` | Generator cadence — a large (≥ 24 h) observation window. |
| `SIMARD_CREATIVE_IDEAS_BATCH` | `10` | Ideas targeted per run (the design's fixed batch of ten). |
| `SIMARD_DAILY_BUDGET_USD` | *(existing)* | Reused for budget-awareness before an expensive tick. |

```rust
pub struct CreativeIdeasConfig {
pub enabled: bool, // SIMARD_CREATIVE_IDEAS_ENABLED (default false)
pub enabled: bool, // SIMARD_CREATIVE_IDEAS_ENABLED (default true; opt-out)
pub interval_secs: u64, // SIMARD_CREATIVE_IDEAS_INTERVAL_SECS (default 86_400)
pub batch: usize, // SIMARD_CREATIVE_IDEAS_BATCH (default 10)
}

impl Default for CreativeIdeasConfig {
/// A default config is ENABLED (default-ON, opt-out).
fn default() -> Self;
}

impl CreativeIdeasConfig {
/// Parse from the environment. Truthy check mirrors overseer_acting_enabled().
/// Parse from the environment. Gate mirrors overseer_acting_enabled():
/// unset/empty ⇒ enabled; only an explicit falsey value opts out.
pub fn from_env() -> Self;
/// True only when the master switch is truthy.
/// Parse from an arbitrary env resolver (test seam).
pub fn from_lookup(lookup: impl Fn(&str) -> Option<String>) -> Self;
/// True unless the master switch was explicitly set to a falsey value.
pub fn enabled(&self) -> bool;
}
```
Expand Down Expand Up @@ -288,15 +298,42 @@ payload fidelity (`links`, `context`, `success_metric`).
| `kind()` | `ThreadKind::BackgroundThought` | Reuses the reserved variant; no enum change. |
| `policy()` | `SchedulePolicy::Interval(Duration::from_secs(interval_secs))` | Default 24 h; reserved to become `Adaptive` later. |
| `priority()` | `Priority::Low` | Never competes with OODA. |
| `enabled()` | `config.enabled()` → **`false` by default** | A disabled thread never ticks (scheduler contract), so the subsystem is inert unless explicitly turned on. |
| `enabled()` | `config.enabled()` → **`true` by default** | Default-ON, opt-out. A disabled thread never ticks (scheduler contract), so an opted-out subsystem is inert. |
| `tick(&mut ThreadContext)` | `ThreadOutcome` | **Never returns `Err`** — internal errors are folded into `ThreadOutcome::failed(reason, elapsed)`. Uses `ctx.now_epoch` (injected clock) and may `block_on` `ctx.runtime`. Honors `ctx.shutdown` (returns promptly between stages) and `ctx.dry_run` (performs no goal/issue/PR side-effect). |

```rust
/// Register the thread with the Mind scheduler. NOT called from the live
/// daemon during the spike; the call site is a marked `// FUTURE:` seam.
/// Register the Creative Ideas thread with the Mind scheduler, using the
/// production idea source + review/route pipeline. Called from the daemon's
/// cognitive-thread setup behind SIMARD_CREATIVE_IDEAS_ENABLED (default-ON).
pub fn register(mind: &mut Mind, config: CreativeIdeasConfig);

/// Register only when `cfg.enabled()`; returns whether it registered. The daemon
/// calls this so "disabled ⇒ not registered" is directly unit-testable.
pub fn register_creative_ideas_if_enabled(mind: &mut Mind, cfg: &CreativeIdeasConfig) -> bool;
```

### Daemon registration

The thread is registered from the OODA daemon's cognitive-thread setup
(`src/operator_commands_ooda/daemon/mod.rs`), next to the Overseer and Journal
threads. Two facts make it observable and testable:

- **Runtime gate.** The daemon builds the `Mind` runtime when *either* the
generic scheduler master switch (`SIMARD_COGNITIVE_THREADS_ENABLED`,
default-OFF, which owns the maintenance/engineer-log threads) **or** Creative
Ideas is enabled — so the default-ON Creative Ideas thread runs even on a stock
deployment where the generic scheduler is off. Registration goes through
`register_creative_ideas_if_enabled`, so an opted-out config leaves the thread
unregistered (asserted by a test via `mind.health()` / `mind.len()`).
- **Startup + per-tick logs.** At startup the daemon emits one dedicated line
mirroring the Journal thread:
`[simard] OODA daemon: creative-ideas thread ENABLED (default) (interval = {n}s; SIMARD_CREATIVE_IDEAS_ENABLED opt-out)`
(or `… DISABLED (SIMARD_CREATIVE_IDEAS_ENABLED opt-out)`). Each run surfaces its
summary through the shared scheduler prefix:
`[simard] cognitive-thread: creative_ideas: generated N idea(s), …`.
The tick executes on a background thread with an overlap guard and panic
isolation, so it cannot stall or crash the authoritative OODA loop.

### Thread inputs (the observation window)

`tick` assembles a typed `GenerationInputs` from **read-only** sources:
Expand Down Expand Up @@ -324,13 +361,17 @@ pub trait IdeaSource {
}
```

- **Available now:** `FakeIdeaSource` (deterministic; used by tests).
- **`// FUTURE:`** `LlmIdeaSource` (the production generator).
- **Production:** `AgenticIdeaSource` (`src/creative_ideas/source.rs`) — renders
the generation prompt asset, runs one agentic turn through the shared
`AgentInvoker` seam (idle-liveness, no wall-clock cap), and parses the JSON
envelope into `RawIdea`s. Fail-closed: a response with no JSON envelope is a
hard error, never a silent empty batch. Built by `register`/`from_env`.
- **Tests:** `FakeIdeaSource` (deterministic; `with_ideas` / `failing`).

The thread targets **ten** ideas per run, applies dedup + portfolio filtering
(below), then persists each survivor as `CreativeIdea { status: New, links,
context }` via `CreativeIdeaStore` and (in the wired future) enqueues it for
review.
(below), persists each survivor as `CreativeIdea { status: New, links,
context }` via `CreativeIdeaStore`, then drives it through the review-and-route
pipeline.

## Reviewer pipeline

Expand Down Expand Up @@ -375,9 +416,11 @@ The four reviewers and their stable ids:
| 3 | `measurability` | agent adapter (NEW) | Emits a concrete `SuccessMetric`, tied where relevant to existing self-metrics (`recall_precision_at_k`, distill fact-yield, reasoner-reliability). This metric is the only thing that can later move the idea to `ImplementationCompleted`. |
| 4 | `idea_feedback_synthesis` | synthesis step | Reads **all** reviews + context, summarizes next steps, and **sets the status** per the state machine. |

Production adapters shape an amplihack skill/agent invocation via the
`invoke_agent(prompt) -> String` seam (the prompt bodies are marked `// FUTURE:`
stubs); all tests inject deterministic fakes.
Production adapters shape an amplihack skill/agent invocation via the shared
`AgentInvoker::invoke(prompt) -> String` seam (the same blessed session path the
OODA brain uses); the three vetting reviewers ship with real prompt assets
(`prompt_assets/simard/creative_ideas_review_*.md`). All tests inject
deterministic fake invokers.

### Synthesis

Expand Down Expand Up @@ -575,7 +618,7 @@ With no wire protocol, three externally-observable contracts must remain stable:
| Node-type sentinel (`trigger_condition = "creative-idea"`) | Never renamed — it is the retrieval key for stored rows | Literal `const CREATIVE_IDEA_TRIGGER`; a rename is a breaking migration |
| Operator / automation surface (`SIMARD_CREATIVE_IDEAS_*`, labels, assignee) | Stable identifiers | Centralized `const`s in `CreativeIdeasConfig` |

- **Trait/type API** — during the spike the Rust surface is
- **Trait/type API** — the Rust surface is
`#![allow(dead_code)]` and carries no semver promise. `Reviewer`,
`FeedbackSynthesizer`, `IdeaSource`, and `IdeaGhClient` are the long-term
extension seams: extend via new impls or new trait methods with defaults,
Expand Down
12 changes: 12 additions & 0 deletions scripts/simard-ooda.service
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,18 @@ Environment=RUST_LOG=info
Environment=SIMARD_GIT_GUARDRAILS=enabled
Environment=SIMARD_GIT_PROTECTED_REPOS=/home/azureuser/src

# Creative Ideas background thread (issue #2647) — an idea-generation cognitive
# thread wired into this daemon. It is DEFAULT-ON (opt-out); leave it unset to
# run it. To disable it on this deployment, uncomment the line below:
# Environment=SIMARD_CREATIVE_IDEAS_ENABLED=0
# Its routing step files a goal, or (for human-review ideas) a labeled + owner-
# tagged GitHub issue via `gh`, and gates idea-derived PRs (draft + blocking
# label + owner review — never --admin/--no-verify). That needs a least-privilege
# token: issues:write + pull_requests:write on the single target repo only.
# Optional tuning (defaults shown):
# Environment=SIMARD_CREATIVE_IDEAS_INTERVAL_SECS=86400
# Environment=SIMARD_CREATIVE_IDEAS_BATCH=10

# Resource limits
LimitNOFILE=65536
MemoryMax=4G
Expand Down
5 changes: 3 additions & 2 deletions src/cognitive_memory/creative_idea.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@
//! | `action_on_trigger` | JSON payload (versioned) with status/context/links/reviews/metric |
//! | `priority` | derived from portfolio/risk |
//!
//! The subsystem is a **spike**: this file is real, tested typed foundation but
//! nothing here is wired into the daemon and the generator is gated OFF (see
//! The subsystem is **live**: this file is the real, tested typed foundation
//! for ideas the daemon's wired Creative Ideas thread generates, reviews, and
//! routes (default-ON, opt-out via `SIMARD_CREATIVE_IDEAS_ENABLED`; see
//! [`crate::creative_ideas`]). `status` changes **only** through
//! [`CreativeIdea::try_transition`], which validates every edge.
#![allow(dead_code)]
Expand Down
Loading
Loading