Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 4 additions & 9 deletions BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,22 +7,17 @@ neither side drifts.

## Open

_Deferred from the 2026-07-01 adversarial code review (shipped fixes are in 3.7.6, [#132](https://github.com/CryptoJones/omind/issues/132)); these are the findings that need a design change or a larger change rather than a contained fix._

- [ ] **Web UI: sanitize rendered note HTML (stored XSS) + localhost auth/Host allowlist** ([#125](https://github.com/CryptoJones/omind/issues/125)) — _bug (security)_ — agent-written note markdown renders through marked v15 into `innerHTML` unsanitized (same-origin JS against the CRUD API on open); no auth/CSRF/Host validation. Sanitize + add a Host allowlist.
- [ ] **CI: add macOS to the test matrix and build + smoke-test the wheel** ([#126](https://github.com/CryptoJones/omind/issues/126)) — _chore_ — no macOS leg despite a macOS fleet; the wheel's file-selection (hook scripts, fonts) is never exercised because CI installs editable.
- [ ] **Mesh: expire / GC tombstones so a re-created note is not silently deleted** ([#127](https://github.com/CryptoJones/omind/issues/127)) — _bug_ — `.omi-tombstones` never expires, so re-creating a previously-purged filename gets deleted fleet-wide with no signal.
- [ ] **loopguard: scope the autonomous-loop guard per session** ([#128](https://github.com/CryptoJones/omind/issues/128)) — _bug_ — the machine-global guard traps concurrent unrelated sessions and its block counter is reset cross-session.
- [ ] **Web UI: fix O(n^2) synchronous graph settle and leaked render loops** ([#129](https://github.com/CryptoJones/omind/issues/129)) — _bug (perf)_ — the graph force-settle freezes the tab at scale; every Graph open leaks a rAF loop + observers.
- [ ] **MCP server + web: run vault I/O off the event loop and lock the shared store** ([#130](https://github.com/CryptoJones/omind/issues/130)) — _bug_ — sync handlers block the MCP loop and the shared store cache is mutated from FastAPI's threadpool (dict-changed-size 500s).
- [ ] **Packaging: pin runtime dependency upper bounds and install the fleet by tag** ([#131](https://github.com/CryptoJones/omind/issues/131)) — _chore_ — open dep floors + untagged-`main` bootstrap let a breaking upstream release land fleet-wide overnight.
_No open items from the 2026-07-05 adversarial code review after this branch's fixes._

## Not planned

- [ ] **Long game: fine-tune a model on the accumulated violation corpus** ([#91](https://github.com/CryptoJones/omind/issues/91), closed not-planned) — _roadmap (Phase 4)_ — deferred: the blocker is data, not compute. The live `compliance.jsonl` corpus is ~91% relevance-noise, ~6% real denies, and 100% DENY (zero ALLOW), so training on it as-is yields an always-deny model. Revisit only after `export-corpus` is reworked to synthesize balanced ALLOW examples (from the deterministic `guard.decide()`) and split the relevance corpus from the action corpus. The mechanical guard remains the backstop.

## Done

- [x] **Adversarial review hardening: MCP/web transport deadlocks and API fallback** — _bug (availability/security)_ — `omind node` no longer depends on the SDK's AnyIO file-wrapper stdio path; it uses fd readiness and still feeds the normal MCP session streams, so stdin handshakes and EOF shutdown cannot wedge. The web API no longer relies on Starlette's thread-backed sync handlers or `StaticFiles` fallback; malformed encoded `/api/...` traversal paths return API 404/400 instead of falling into static serving, and packaged assets are served by a direct path-resolved responder that rejects escapes before reading bytes.
- [x] **Adversarial review hardening: transfer archives + subprocess error redaction** — _bug (security/perf)_ — tar.gz export now excludes VCS control directories (`.git`, `.hg`, `.svn`) so mesh vault exports do not leak git history or produce giant bundles; tar.gz import now rejects control-directory members so crafted bundles cannot plant git config/hooks. Shared subprocess failures now redact URL userinfo, GitHub tokens, and Authorization headers before surfacing command/error text.
- [x] **Deferred 2026-07-01 adversarial review items shipped by 3.8.5** ([#125](https://github.com/CryptoJones/omind/issues/125), [#126](https://github.com/CryptoJones/omind/issues/126), [#127](https://github.com/CryptoJones/omind/issues/127), [#128](https://github.com/CryptoJones/omind/issues/128), [#129](https://github.com/CryptoJones/omind/issues/129), [#130](https://github.com/CryptoJones/omind/issues/130), [#131](https://github.com/CryptoJones/omind/issues/131)) — _meta_ — current tree includes DOMPurify + Host allowlist, macOS CI + installed-wheel smoke, tombstone expiry/GC, per-session loopguard ownership, bounded graph settle + graph teardown, store cache locking / graph caching, and runtime dependency upper bounds.
- [x] **Hardening batch from adversarial code review** ([#132](https://github.com/CryptoJones/omind/issues/132), [PR #124](https://github.com/CryptoJones/omind/pull/124)) — _meta_ — v3.7.6: note data-integrity (frontmatter/lead + fence-aware parse, symmetric mesh-merge convergence), one-bad-byte read hardening, guard false-positive fixes (freshness `-C`/compound forms, command-anchored forge rules, bare `>` side-effect, project-vs-global `.claude/`, negation-aware auth), guard crash-hardening, enforcement fail-open holes (adapter fail-closed, contentless-consult gate-dodge, secret-output `2>/dev/null` leak), atomic config/hook/backup writes, checkpoint/mesh/update/lint availability, and the migrate-hook data-loss. 714 tests + ruff + mypy green. Deferred items → #125–#131. Shipped in 3.7.6.
- [x] **Codex CLI: `omind setup` only wired the guard, not the `omi` MCP server** ([GitHub #114](https://github.com/CryptoJones/omind/issues/114)) — _enhancement_ — `omind setup --agent codex` now also merges `[mcp_servers.omi]` into `~/.codex/config.toml` (via `tomlkit`, TOML round-trip preserved) so Codex can call the OMI memory tools directly, not just get blocked by the guard. `doctor --agent codex` reports `codex_mcp_registration` alongside `codex_guard`. Shipped in 3.7.0.
- [x] **Rotate `MCP_CONFORMANCE_TOKEN` before it expires** ([Codeberg #88](https://codeberg.org/CryptoJones/omind/issues/88), [GitHub #105](https://github.com/CryptoJones/omind/issues/105)) — _chore_ — `MCP_CONFORMANCE_TOKEN` is set on the omind repo's Actions secrets and verified live: a re-run of `test.yml` installed the private `mcp-conformance` package and ran the suite (`10 passed, 1 skipped`), no graceful-skip. The PAT is non-expiring (Contents:Read on `CryptoJones/mcp-conformance`), so there is no rotation-before-expiry deadline.
Expand Down
8 changes: 7 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,12 @@ dependencies = [
"fastapi>=0.110,<1.0",
"uvicorn[standard]>=0.29,<1.0",
"pyyaml>=6.0,<7.0",
"mcp>=1.10,<2.0",
"mcp>=1.28.1,<2.0",
"tomlkit>=0.12,<1.0",
# Security floors for runtime transitives pulled by fastapi/mcp.
"cryptography>=48.0.1,<50.0",
"pydantic-settings>=2.14.2,<3.0",
"starlette>=1.3.1,<2.0",
]

[project.optional-dependencies]
Expand All @@ -60,6 +64,8 @@ dev = [
"mypy>=1.9",
"httpx>=0.27",
"pip-audit>=2.10.0",
# Security floor for pip-audit/cachecontrol's msgpack edge.
"msgpack>=1.2.1,<2.0",
"types-PyYAML",
]

Expand Down
9 changes: 8 additions & 1 deletion src/omind/guard.py
Original file line number Diff line number Diff line change
Expand Up @@ -745,7 +745,14 @@ def _git_dash_c_path(command: str) -> Path | None:
parts = _split_simple_commands(command)
if not parts:
return None
tokens = shlex.split(parts[0])
lexer = shlex.shlex(parts[0], posix=True)
lexer.whitespace_split = True
lexer.commenters = ""
# POSIX shlex treats backslash as an escape and would turn a literal
# Windows path such as C:\\repo into C:repo. Quotes are still honoured;
# disabling only escapes preserves paths on every supported platform.
lexer.escape = ""
tokens = list(lexer)
except ValueError:
return None
if not tokens or tokens[0] != "git":
Expand Down
26 changes: 23 additions & 3 deletions src/omind/proc.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,32 @@
from __future__ import annotations

import os
import re
import shutil
import subprocess

DEFAULT_TIMEOUT = 600.0
"""Seconds before a spawned command is killed; generous for slow npm installs."""

_URL_USERINFO_RE = re.compile(r"([A-Za-z][A-Za-z0-9+.-]*://)([^/\s@]+@)")
_GITHUB_TOKEN_RE = re.compile(r"\b(?:ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9_]{20,}\b")
_GITHUB_PAT_RE = re.compile(r"\bgithub_pat_[A-Za-z0-9_]{20,}\b")
_AUTH_HEADER_RE = re.compile(
r"(?i)(authorization:\s*(?:bearer|token|basic)\s+)[A-Za-z0-9._~+/\-]+=*"
)


def _redact(text: str) -> str:
"""Scrub credentials from command/error text before surfacing it."""
text = _URL_USERINFO_RE.sub(r"\1[redacted]@", text)
text = _GITHUB_TOKEN_RE.sub("[redacted-token]", text)
text = _GITHUB_PAT_RE.sub("[redacted-token]", text)
return _AUTH_HEADER_RE.sub(r"\1[redacted-token]", text)


def _cmd_text(cmd: list[str]) -> str:
return _redact(" ".join(cmd))


def run_command(
cmd: list[str],
Expand Down Expand Up @@ -62,7 +82,7 @@ def run_command(
except FileNotFoundError as exc:
raise error(f"command not found: {cmd[0]}") from exc
except subprocess.TimeoutExpired as exc:
raise error(f"command timed out after {timeout:g}s: {' '.join(cmd)}") from exc
raise error(f"command timed out after {timeout:g}s: {_cmd_text(cmd)}") from exc
except subprocess.CalledProcessError as exc:
detail = (exc.stderr or exc.stdout or "").strip()
raise error(f"command failed: {' '.join(cmd)}\n{detail}") from exc
detail = _redact((exc.stderr or exc.stdout or "").strip())
raise error(f"command failed: {_cmd_text(cmd)}\n{detail}") from exc
108 changes: 107 additions & 1 deletion src/omind/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,17 @@

import contextlib
import logging
import os
import sys
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from pathlib import Path

import anyio
import mcp.types as mcp_types
from anyio.streams.memory import MemoryObjectReceiveStream, MemoryObjectSendStream
from mcp.server.fastmcp import FastMCP
from mcp.shared.message import SessionMessage

from omind import graph
from omind.store import ActionItem, NoteFields, OmiStore, parse_note
Expand All @@ -35,6 +42,95 @@
logger = logging.getLogger(__name__)


@asynccontextmanager
async def _fd_stdio_server() -> AsyncIterator[
tuple[
MemoryObjectReceiveStream[SessionMessage | Exception],
MemoryObjectSendStream[SessionMessage],
]
]:
"""MCP stdio transport using fd readiness instead of AnyIO file wrappers."""
stdin_fd = sys.stdin.buffer.fileno()
stdout_fd = sys.stdout.buffer.fileno()

read_stream_writer, read_stream = anyio.create_memory_object_stream[
SessionMessage | Exception
](0)
write_stream, write_stream_reader = anyio.create_memory_object_stream[SessionMessage](0)

async def send_line(line: str) -> None:
try:
message = mcp_types.JSONRPCMessage.model_validate_json(line)
except Exception as exc:
await read_stream_writer.send(exc)
return
await read_stream_writer.send(SessionMessage(message))

async def stdin_reader() -> None:
buffer = b""
try:
async with read_stream_writer:
while True:
if sys.platform == "win32":
# Windows pipe handles are not sockets and cannot be
# registered with AnyIO's readiness backend. Keep the
# event loop free while one worker blocks on a line.
chunk = await anyio.to_thread.run_sync(sys.stdin.buffer.readline)
else:
await anyio.wait_readable(stdin_fd)
try:
chunk = os.read(stdin_fd, 65536)
except BlockingIOError:
continue
if not chunk:
break
buffer += chunk
while b"\n" in buffer:
raw, buffer = buffer.split(b"\n", 1)
await send_line(raw.decode("utf-8", errors="replace"))
if buffer:
await send_line(buffer.decode("utf-8", errors="replace"))
except anyio.ClosedResourceError: # pragma: no cover
await anyio.lowlevel.checkpoint()

async def write_all(data: bytes) -> None:
if sys.platform == "win32":
def write_stdout() -> None:
sys.stdout.buffer.write(data)
sys.stdout.buffer.flush()

await anyio.to_thread.run_sync(write_stdout)
return

offset = 0
while offset < len(data):
try:
written = os.write(stdout_fd, data[offset:])
except BlockingIOError:
await anyio.wait_writable(stdout_fd)
continue
if written == 0:
await anyio.wait_writable(stdout_fd)
continue
offset += written

async def stdout_writer() -> None:
try:
async with write_stream_reader:
async for session_message in write_stream_reader:
payload = session_message.message.model_dump_json(
by_alias=True, exclude_none=True
)
await write_all((payload + "\n").encode("utf-8"))
except (anyio.ClosedResourceError, BrokenPipeError): # pragma: no cover
await anyio.lowlevel.checkpoint()

async with anyio.create_task_group() as tg:
tg.start_soon(stdin_reader)
tg.start_soon(stdout_writer)
yield read_stream, write_stream


def _parse_action_items(items: list[str]) -> list[ActionItem]:
"""``"[x] text"`` marks a completed item; anything else is open."""
parsed: list[ActionItem] = []
Expand Down Expand Up @@ -283,5 +379,15 @@ def run_node(omi_dir: Path, node_id: str | None = None) -> int:
"""CLI entry: serve the node over stdio until the client closes stdin."""
# stdout is the protocol channel; everything else goes to stderr.
logging.basicConfig(stream=sys.stderr, level=logging.WARNING)
build_server(omi_dir, node_id=node_id).run("stdio")

async def run_stdio() -> None:
mcp = build_server(omi_dir, node_id=node_id)
async with _fd_stdio_server() as (read_stream, write_stream):
await mcp._mcp_server.run( # noqa: SLF001 - FastMCP exposes no public lower-level runner.
read_stream,
write_stream,
mcp._mcp_server.create_initialization_options(), # noqa: SLF001
)

anyio.run(run_stdio)
return 0
17 changes: 16 additions & 1 deletion src/omind/transfer.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
import tempfile
from collections.abc import Callable
from dataclasses import dataclass, field
from pathlib import Path
from pathlib import Path, PurePosixPath
from typing import Any

from omind import __version__, paths
Expand All @@ -40,6 +40,12 @@
EXPORT_VERSION = 1
FORMATS = ("json", "targz")

# VCS control directories are executable trust/config state, not vault data. A
# crafted import that writes .git/config or hooks/ can change what later mesh
# git operations execute, and exporting .git/ leaks deleted-note history while
# making bundles enormous.
CONTROL_DIR_NAMES = frozenset({".git", ".hg", ".svn"})


class TransferError(Exception):
"""Raised on a bad format, unreadable bundle, or unsafe member path."""
Expand Down Expand Up @@ -90,6 +96,11 @@ def _runtime_artifact(name: str) -> bool:
return name == LOCK_FILENAME or name.startswith(".tmp-")


def _control_artifact(rel: str | PurePosixPath) -> bool:
"""True when an archive path crosses a VCS control directory."""
return any(part in CONTROL_DIR_NAMES for part in PurePosixPath(str(rel)).parts)


def _exportable_md(omi_dir: Path) -> list[Path]:
"""Top-level ``*.md`` notes, excluding the derived index and dotfiles."""
if not omi_dir.is_dir():
Expand Down Expand Up @@ -154,6 +165,8 @@ def _export_targz(omi: Path, out: Path) -> int:
if not path.is_file() or _runtime_artifact(path.name):
continue
arcname = path.relative_to(omi).as_posix()
if _control_artifact(arcname):
continue
tar.add(path, arcname=arcname)
if (
path.suffix == ".md"
Expand Down Expand Up @@ -310,6 +323,8 @@ def _import_targz(
# Traversal guard: every member must land inside the OMI dir.
if target != omi_resolved and omi_resolved not in target.parents:
raise TransferError(f"archive member escapes the OMI directory: {rel!r}")
if _control_artifact(rel):
raise TransferError(f"archive member targets a VCS control directory: {rel!r}")
if Path(rel).name == paths.INDEX_FILENAME and Path(rel).parent == Path("."):
continue # derived top-level index; regenerated after import
if _runtime_artifact(Path(rel).name):
Expand Down
Loading