Skip to content

joshyorko/codex-memoryd

Repository files navigation

codex-memoryd

codex-memoryd is a local-first, agent-agnostic memory substrate for coding agents. The durable store is the product; the daemon, CLI, Docker Compose heartbeat, adapter exports, and MCP stdio are delivery modes. Memory is recall, not authority: retrieved context can inform a turn, but it never overrides the current user, repo, or policy state.

Current Surface

This is the landed MVP surface today:

Surface Status Notes
Local loopback server landed Native daemon binds 127.0.0.1:8787 with persistent SQLite storage.
Docker Compose dogfood landed Uses .dogfood/memory.db as the real daemon DB and keeps host publish loopback-only.
Compose heartbeat landed scripts/dogfood-compose-heartbeat.sh rebuilds, restarts, and smoke-checks the stack.
Fixture substrate demo landed scripts/demo-substrate.sh runs a one-command end-to-end demo against a temporary fixture DB.
Dogfood write sandbox landed scripts/dogfood-write-sandbox.sh refreshes a sandbox from the real DB, runs write canaries only there, and emits content-free diffs.
v0.1 release gate landed scripts/v0.1-release-gate.sh runs the local release-quality gate and records artifacts under target/release-gate/.
Codex memory import landed sync-local --preview / --apply import local Codex memories.
Native memory migration plan documented Phases native Codex memory from import/fallback toward optional memoryd-canonical mode.
Subject / episode substrate landed Stable subjects, append-only episodes, and the evidence ledger are the core memory shape.
Recall policy metadata landed recall_not_authority, ranking, admission, and provenance metadata travel with recall.
Temporal recall landed Added --as-of and --include-history recall paths for as-of and historical inspection.
Current-state cards landed MVP workspace_summary, subject_summary, active_preferences, open_questions, recent_scars, procedures_index.
Context packs landed MVP default, debugging, onboarding, planning, active_task, review, and personal_context pack modes are supported.
Adapter exports landed agents-md, claude-code, copilot, github-instructions, mcp-json, mcp-pack, markdown, and markdown-wiki.
Adapter conformance landed conformance adapters emits a deterministic report for adapter authority, provenance, and budget behavior.
Adapter packages landed Installable templates for Codex MCP, Claude-style local MCP, Copilot instructions, and generic MCP/markdown clients.
Git import landed Imports commit trailers plus refs fixtures for JSON and JSONL exports.
MCP dogfood landed read-only Exposes memory_status, memory_recall, and memory_search only.
Dreamer patch lifecycle landed MVP Preview/apply exists for reviewable consolidation; broader procedural automation is still roadmap work.

OpenAI Build Week Provenance

codex-memoryd existed before OpenAI Build Week as a local-first daemon, CLI, MCP surface, SQLite memory substrate, recall policy system, temporal record model, local Codex-memory importer, and deterministic Dreamer prototype. That foundation should not be described as newly built during Build Week.

The Build Week GPT-5.6/Codex work was split across a small set of auditable threads and commits:

  • Canonical governed-continuity implementation thread: 019f60a8-8ddd-7b00-8a8e-daf055ed45b5.
  • Baseline/audit/product-shaping thread: 019f5922-57c3-7493-9281-29a4634b3d22.
  • Supporting submission-finalization/bootstrap UX thread: 019ed048-8f2b-73d1-ae82-8db3f4e13403.

The core new Build Week implementation is the Codex-side governed recall bridge on the sibling Codex fork branch build-week/codex-memoryd-governed-runtime:

  • acb7d19c2d296116e2c8f793a48b53d59e764d55: maintenance-only port of the portable memory runtime to the current upstream Codex APIs.
  • 2189ae1835df12dc6f017934290d642519e8a2e0: preserves structured governed MemoryD recall through Codex, sends real repository/branch/thread/file scope, requests active_task packs with a 900-token budget, keeps legacy MemoryD/Honcho compatibility, and separates model-visible admitted context from sanitized inspection metadata.
  • 631c5125dff17b0bef948bdd3df7458a7c336b2e: records the governed recall bridge milestone in Build Week evidence docs.

This repository's submission-finalization thread produced the CLI-first bootstrap/runtime UX slice:

  • 7cf1e30: makes codex-memoryd init --port 8989, up, status, sync-local --apply ~/.codex/memories, and recall usable without Docker Compose or hidden image-tag knowledge; adds explicit resolved runtime config, optional managed-container mode, a local image build path, safer container errors, Dreamer env pass-through for managed containers, and a stale/superseded default-recall regression test.

The official submission Session ID comes from /feedback executed in the representative core implementation thread, not the finalization thread. The generated Session ID is 019f60a8-8ddd-7b00-8a8e-daf055ed45b5.

Safety Model

  • Loopback-only is the default dogfood posture.
  • recall_not_authority is mandatory. Recall can inform, not command.
  • Preview happens before apply.
  • Apply is idempotent and policy-gated.
  • No automatic prompt injection.
  • Secrets, tokens, .env dumps, hidden reasoning, private keys, auth files, and raw confidential logs are rejected before durable write.
  • Profile and workspace boundaries are enforced.
  • Recall admits only the requested profile/workspace by default. Metadata-marked quarantined, high/unsafe risk, unsafe/rejected/blocked, and superseded records are withheld unless a future explicit review path changes the state.
  • Stale records may still be returned, but they are marked stale and deprioritized. Archived/superseded records remain explainable through withheld counts rather than raw recalled content.
  • Provider failure fails open instead of blocking Codex turns.
  • The read-only MCP path exposes no write tools.

First Run

First-run path (CLI-first product mode)

An installed codex-memoryd binary can initialize and manage a local native daemon without Docker, Podman, Compose, or a repo checkout. Josh's active dogfood path uses 127.0.0.1:8989 because Headroom owns 127.0.0.1:8787:

codex-memoryd init --port 8989
codex-memoryd up
codex-memoryd status
codex-memoryd sync-local --preview ~/.codex/memories
codex-memoryd sync-local --apply ~/.codex/memories
codex-memoryd recall --query "safe dogfood mode"
codex-memoryd dream status

init seeds config/runtime state only and prints codex-memoryd up as the next command. up, status, recall, and sync-local use the resolved CODEX_MEMORYD_URL, host, and port from runtime config/env. Product fallback remains ~/.codex-memoryd on loopback http://127.0.0.1:8787 when no port is configured. Direct SQLite/admin mode remains explicit:

codex-memoryd --local --db ~/.codex-memoryd/memory.db doctor
codex-memoryd --local --db ~/.codex-memoryd/memory.db backup create \
  --dest ~/.codex-memoryd/backups/pre-change.db

Runtime choices are explicit:

codex-memoryd init --runtime native --port 8989
codex-memoryd init --runtime container --port 8989
codex-memoryd image build --tag codex-memoryd:local
codex-memoryd init --dogfood

Native is the default and requires no container runtime. Managed container runtime is the optional no-Compose path. It pulls/runs CODEX_MEMORYD_IMAGE directly with Docker/Podman; for a local image, build with codex-memoryd image build --tag codex-memoryd:local, then run CODEX_MEMORYD_IMAGE=codex-memoryd:local codex-memoryd --runtime container up. Compose remains for development and debugging.

Inspect the resolved client/runtime/daemon registry with:

codex-memoryd config show --resolved
codex-memoryd config env
codex-memoryd config doctor

The resolved registry reports each setting's owner, source, restart behavior, container pass-through, commit safety, and secret posture. In managed container mode, sync-local --preview ~/.codex/memories and sync-local --apply ~/.codex/memories still use the host path; the CLI packages host files for the daemon so users do not need to know container mount paths.

For local dogfood with a second app beside the daemon, add an explicit config lane instead of teaching up/down to manage that app:

[runtime.adjacent]
enabled = true
name = "dogfood-router"
url = "http://127.0.0.1:4318"

codex-memoryd status, codex-memoryd paths --format json, and codex-memoryd config show --resolved then report the adjacent lane separately, including endpoint ownership and conflicts with codex-memoryd's own front door.

First-run path (source build)

For source checkouts, the same product commands work against the built binary. Manual serve is still useful for debugging:

cargo build --release
mkdir -p .dogfood/logs .dogfood/exports

CODEX_MEMORYD_DB="$PWD/.dogfood/memory.db" \
CODEX_MEMORYD_BIND="127.0.0.1:8989" \
CODEX_MEMORYD_PROFILE="personal" \
CODEX_MEMORYD_WORKSPACE="josh-personal" \
target/release/codex-memoryd serve

In another shell:

curl -fsS http://127.0.0.1:8989/healthz
curl -fsS http://127.0.0.1:8989/v1/status | jq
target/release/codex-memoryd doctor
target/release/codex-memoryd sync-local --preview ~/.codex/memories \
  --profile personal --workspace josh-personal
target/release/codex-memoryd sync-local --apply ~/.codex/memories \
  --profile personal --workspace josh-personal
target/release/codex-memoryd conclude --profile personal --workspace josh-personal \
  --content "Decision: keep codex-memoryd local-first."
target/release/codex-memoryd recall --profile personal --workspace josh-personal \
  --pack-mode onboarding --query "What is the safe dogfood mode?"
target/release/codex-memoryd card show --profile personal --workspace josh-personal \
  --type workspace_summary

Fail-open note: if codex-memoryd is unavailable, Codex provider or hybrid mode should skip provider recall and continue the turn instead of blocking work.

Canonical local runtime helper

For day-to-day native dogfood, use the checked-in runtime helper instead of copying process-management snippets by hand:

scripts/codex-memoryd-local-runtime.sh start
scripts/codex-memoryd-local-runtime.sh status
scripts/codex-memoryd-local-runtime.sh smoke
scripts/codex-memoryd-local-runtime.sh restart-survival
scripts/codex-memoryd-local-runtime.sh stop

The helper defaults to CODEX_MEMORYD_BIND=127.0.0.1:8787, CODEX_MEMORYD_DB=$PWD/.dogfood/memory.db, CODEX_MEMORYD_PROFILE=personal, and CODEX_MEMORYD_WORKSPACE=josh-personal. It refuses non-loopback binds unless CODEX_MEMORYD_ALLOW_NON_LOOPBACK=1 is set for an explicitly self-hosted deployment behind HTTPS authentication.

Compose smoke and heartbeat

Compose is useful for reproducible smoke checks, but it is not the primary dogfood surface.

docker compose up -d --build
curl -fsS http://127.0.0.1:8787/healthz
curl -fsS http://127.0.0.1:8787/v1/status | jq
scripts/dogfood-compose-heartbeat.sh
docker compose down

To run the Compose daemon with the background Dreamer scheduler enabled, use a single environment flag. No config override file is required:

CODEX_MEMORYD_DREAM_SCHEDULER_ENABLED=1 docker compose up -d --build
curl -fsS http://127.0.0.1:8787/v1/status | jq '.data.features.dream_scheduler'

For day-to-day CLI work against the same .dogfood/memory.db, use the local front door instead of docker compose exec:

scripts/memd status | jq
scripts/memd doctor --format json | jq
scripts/memd dream --preview
scripts/memd dream --apply
scripts/memd recall --query "safe dogfood mode"

Fixture substrate demo

For a reviewer-safe end-to-end walkthrough that uses only synthetic fixture data and a temporary SQLite database:

scripts/demo-substrate.sh

See docs/demo-substrate.md for the exact demo steps and safety guarantees.

Dogfood write sandbox

For write-capable dogfood checks, keep the real DB read-only and run the sandbox lane:

scripts/dogfood-write-sandbox.sh run

See docs/dogfood-write-sandbox.md for the content-free diff report and manual promotion workflow.

v0.1 release gate

Before tagging, run:

scripts/v0.1-release-gate.sh

For the live dogfood lane, rerun from the dogfood checkout with:

scripts/v0.1-release-gate.sh --include-dogfood

See docs/release/v0.1-hardening.md for the checklist, caveats, and issue coverage map.

Operator Flows

Local Codex memory import

sync-local is the supported path for importing existing Codex memories. Use preview first, then apply.

scripts/memd sync-local --preview ~/.codex/memories
scripts/memd sync-local --apply ~/.codex/memories

For the native Codex memory migration phases, parity canaries, duplicate-loop risks, and canonical-mode checklist, see docs/native-codex-memory-migration.md.

Recall and search

Recall is contextual evidence, not authority. Pack modes currently accept default, debugging, onboarding, planning, active_task, review, and personal_context. Hyphenated CLI input such as active-task is normalized to the wire value active_task.

Mode Bias
default balanced recall
debugging gotchas, failures, rollback/recovery, commands
onboarding conventions, architecture, setup, current state
planning checkpoints, decisions, blockers, open questions, next steps
active_task current handoff, blockers, next steps, commands
review PR/review risk, verification, regressions, rollback notes
personal_context user preferences and operating workflow defaults
target/release/codex-memoryd recall --profile personal --workspace josh-personal \
  --pack-mode default --query "How do we use codex-memoryd?"
target/release/codex-memoryd search --profile personal --workspace josh-personal \
  --query "safe dogfood"
target/release/codex-memoryd recall --profile personal --workspace josh-personal \
  --query "What was my workspace preference on 2026-03-01?" \
  --as-of "2026-03-01T00:00:00Z"
target/release/codex-memoryd recall --profile personal --workspace josh-personal \
  --query "How has this preference evolved?" \
  --include-history

Cards and exports

Current-state cards are deterministic views from the same store. Adapter exports compile the same substrate into downstream file formats.

target/release/codex-memoryd card show --profile personal --workspace josh-personal \
  --type subject_summary
target/release/codex-memoryd eval retrieval --format summary
target/release/codex-memoryd eval retrieval --format json
target/release/codex-memoryd eval benchmark synthetic --subset temporal --format summary
target/release/codex-memoryd eval benchmark synthetic --input ./datasets/local-benchmark.json --full --format json
target/release/codex-memoryd adapter export --target agents-md \
  --profile personal --workspace josh-personal > AGENTS.memory.md
target/release/codex-memoryd adapter export --target mcp-pack \
  --profile personal --workspace josh-personal > mcp-pack.json
target/release/codex-memoryd adapter export --target mcp-json \
  --profile personal --workspace josh-personal > mcp-json.json
target/release/codex-memoryd conformance adapters --format json
target/release/codex-memoryd eval substrate --format summary
target/release/codex-memoryd eval substrate --format json

Cards are generated on demand, so there is no persisted card cache to invalidate. Each record carries explicit freshness metadata (freshness.stale and freshness.age_days), and the card-level freshness value is contains_stale_records whenever any included record is past the stale display window. CLI markdown and adapter views render the same fresh/stale label. The card smoke suite includes a fixture-backed markdown snapshot for this contract.

eval substrate is the deterministic, model-free review gate for substrate correctness, safety, procedure memory, patch rollback, and adapter/context-pack economics. eval retrieval is the deterministic, fixture-backed retrieval quality loop for issue #153, including long-history baselines and ablations. eval benchmark synthetic is the first public-benchmark runner foundation: it loads a neutral JSON corpus shape from the checked-in synthetic fixture by default and can point at an operator-supplied local file for tiny subset runs outside CI. See docs/eval-substrate.md.

Adapter packages

Installable adapter package templates live under adapters/. They are thin host-specific wrappers around codex-memoryd and default to read-only operation:

  • adapters/codex-mcp: Codex MCP config snippet plus wizard reference.
  • adapters/claude-local: Claude-style local MCP server JSON snippet.
  • adapters/copilot-instructions: Copilot instructions export wrapper.
  • adapters/generic-mcp-markdown: generic MCP plus AGENTS.md and GEMINI.md markdown export wrappers.

Each package includes .env.example, install steps, verify commands, and uninstall notes. The packages do not duplicate memory logic; they call codex-memoryd mcp stdio --read-only or codex-memoryd adapter export. For Codex specifically, the preferred setup path is now:

codex-memoryd mcp codex preview
codex-memoryd mcp codex apply
codex-memoryd mcp codex status
codex-memoryd mcp codex remove

Git evidence import

git-import reads recent commit trailers such as Memory-Decision, Memory-Verify, and Memory-Gotcha, plus refs fixtures for JSON and JSONL exports.

target/release/codex-memoryd git-import --preview /path/to/repo
target/release/codex-memoryd git-import --apply /path/to/repo
target/release/codex-memoryd git-import --preview --refs-fixture /path/to/refs.jsonl /path/to/repo
target/release/codex-memoryd git-import --apply --refs-fixture /path/to/refs.json /path/to/repo

Preview/apply/idempotency are the same path in both modes. Apply writes safe subject episodes and evidence ledger rows only; it does not promote Git evidence to active memory records on its own.

MCP read-only dogfood

The Codex-facing MCP runbook is intentionally read-only. mcp stdio defaults to the read-only tool tier; --read-only is still accepted for explicit adapter configs.

target/release/codex-memoryd --db .dogfood/mcp-sandbox-memory.db mcp stdio --read-only

The server exposes only memory_status, memory_recall, and memory_search. See docs/dogfood-mcp.md for the wizard flow, generated ~/.codex/config.toml snippet, and smoke checks.

The write-capable tier is intentionally opt-in:

target/release/codex-memoryd --db ~/.codex-memoryd/memory.db mcp stdio --write-tools

That tier adds memory_create, memory_conclude, memory_checkpoint, memory_import_preview, and memory_import_apply. These tools still pass through the same write policy, secret detection, import preview/apply, and provenance paths as the HTTP and CLI surfaces. Do not expose --write-tools to a Codex sandbox or remote client unless an adapter capability review has explicitly allowed that client to write.

Memory Model

  • Subject: stable identity for what the memory is about.
  • Episode: immutable event attached to a subject.
  • Evidence ledger: append-only provenance for writes, imports, and synthesis.
  • Card: deterministic current-state view for cheap recall.
  • Pack: budgeted recall bundle for a specific adapter or mode.
  • Adapter export: downstream rendering such as AGENTS.md, CLAUDE.md, mcp-json, or markdown-wiki.
  • Recall policy metadata: admission, ranking, provenance, and recall_not_authority.
  • Operational valence marker: audited Dreamer metadata for battle_scar, comfort_path, surprise, recovery_pattern, and confidence_delta. Markers carry trigger/outcome/recovery JSON, intensity, half-life decay, counter-evidence refs, and optional retirement state. They are ranking and explanation signals only; they never become authority or automatic commands.

This substrate is intentionally not a graph engine, CRM, scheduler, or agent harness. Subjects and episodes are internal anchors, not a standalone exported surface.

Release Snapshot

The #50-#86 substrate tranche is now landed or closed by this README's eval suite PR. Future work should add sharper evals, production hardening, and operator polish rather than reopen the old stale snapshot.

Issue Lane Status
#50 current-state cards landed deterministic cards with stale metadata and fixture-backed rendering coverage
#53 eval suite closed by eval substrate: deterministic JSON/summary report for recall, safety, procedure memory, patch rollback, and adapter/context-pack economics
#55 recall policy landed admission gates that withhold quarantined/high-risk/unsafe/superseded records by default; further hardening belongs to newer eval/review lanes
#56 adapter context packs landed deterministic recall packs, adapter-specific pack names, budget/truncation reporting, and regression coverage
#57 git import landed for local trailers and refs fixtures covering commits, PRs, issues, and review comments
#66 procedural memory landed schema v6 procedure records, candidate preview/apply flow, and recall/export surfaces
#67 operational valence landed marker v2 fields, deterministic decay, counter-evidence retirement, and ranking-only recall signals
#68 trust quarantine landed trust/quarantine controls that keep unsafe evidence out of default recall
#69 multimodal evidence landed artifact metadata, redaction policy, and non-text evidence import shape
#70 adapter conformance conformance adapters report now certifies adapter authority, provenance, and budget behavior
#80 deployment runtime landed local runtime helper and restart-survival smoke path
#81 MCP v2 landed default read-only tier plus explicit write-tool tier
#82 hosted app adapter landed design doc for hosted app memory adapter boundaries
#83 plugin packages landed installable adapter package templates
#84 native Codex migration landed migration plan, parity canaries, and duplicate-loop guardrails
#85 dogfood heartbeat landed Compose heartbeat rebuild/import/MCP canary script
#86 README overhaul landed refreshed first-run and substrate story
#155 temporal records landed temporal fields + RFC3339 as-of/history recall; schema bumped to 9 with migration marker 0010_temporal_records.sql.

Related Docs

License

MIT.

About

No description, website, or topics provided.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors