feat(daemon): idle timeout with MCP heartbeat — stop holding the embedding model in RAM forever by junzh0u · Pull Request #215 · cocoindex-io/cocoindex-code

junzh0u · 2026-07-15T18:18:37Z

Summary

The daemon currently lives forever once auto-started, pinning the embedding model in memory indefinitely — after the last search of the day, that memory is held all night for nobody.

Measured on my machine (macOS, Apple Silicon): a freshly started daemon idles at ~51 MB RSS. After a single search with Snowflake/snowflake-arctic-embed-xs — one of the smallest usable models — it sits at 588 MB RSS / 1.4 GB macOS footprint, of which ~1.0 GB is GPU-shared (MPS IOAccelerator) memory that competes with everything else using the GPU. That 1.4 GB is held until reboot. With mid/large models (arctic-embed-m, nomic-embed, Qwen embedders) the number is multiple GB, and it hurts most on the 16 GB laptops that can least afford it.

This PR makes the daemon exit after a configurable idle period and lets clients revive it transparently on the next request (a few seconds of model reload):

daemon.idle_timeout_minutes in global_settings.yml — default 180, 0 = never (old behavior).
The reaper never fires while a connection is live, an indexing run is in flight, or under an external supervisor (COCOINDEX_CODE_DAEMON_SUPERVISED=1, e.g. the Docker entrypoint restart loop, where idle-exit would just cause respawn churn).
MCP heartbeat: a live ccc mcp session (either entry point) pings the daemon every timeout/3 (clamped 30 s–5 min), so an open editor session keeps the model warm across a workday's pauses. Heartbeats never start a daemon — ccc daemon stop wins. If the MCP process dies, even by SIGKILL, heartbeats stop and the daemon exits on the normal timeout — self-healing, no stale-lease problem.
ccc daemon status now shows idle time and the configured timeout.
Windows: shutdown now unblocks the named-pipe accept thread, so an idle-exit (or any exit) releases the pipe promptly instead of leaving it held until a final client connects.

The default and overall shape follow established practice for auto-started dev-tool daemons: Gradle stops its daemon after 3 h idle, Bazel's server defaults to --max_idle_secs=10800 (3 h), sccache exits after 10 min, and the Nx daemon shuts down after ~3 h of inactivity. The cautionary tale in the other direction is watchman-style forever-resident processes that users discover squatting on RAM weeks later.

⚠️ Open question for reviewers — the default value. Making 180 the default is a debatable call: it changes existing behavior on upgrade, and one can argue 0 (never exit) is the least surprising default in terms of backwards-compatibility. I went with 180 because it matches the Gradle/Bazel/Nx convention, the change is self-healing (the daemon transparently restarts on the next request, costing only a few seconds of model reload), and a memory-saving feature that's off by default would help exactly nobody who doesn't already know about it. Happy to flip the default to 0 if maintainers prefer strict compatibility.

Builds on #214 (now merged): the graceful-exit marker is what lets clients treat an idle-exit as routine (silent restart) while still surfacing real crashes loudly.

Tests

Settings parsing (absent section → default, explicit 0), 7 reaper-predicate unit tests, heartbeat-interval formula, and in-process daemon E2Es with seconds-scale injected timeouts: idle-exit with socket/PID cleanup and correct marker reason; live connection blocks exit; an in-flight index run survives past the timeout; heartbeats keep the daemon alive and it exits after they stop; a heartbeat against a stopped daemon returns False and spawns nothing. Full suite green (pytest, mypy, ruff). Also verified live against a real daemon.

Note: rebased on main after #214 merged, then extended with two review-response commits: the idle timeout is now typed as timedelta end-to-end, and last_heartbeat tracking was dropped from the daemon entirely (a heartbeat's only effect is the connection itself, which already counts as activity) — so heartbeat age no longer appears in the wire protocol or ccc daemon status.

🤖 Generated with Claude Code

junzh0u · 2026-07-17T18:00:51Z

+        if sys.platform == "win32":
+            try:
+                Client(sock_path, family=connection_family()).close()
+            except OSError:
+                pass


Proposal: ungate this and do it unconditionally?

POSIX doesn't need it, but it's harmless there too.
And if anything breaks in this path, ungating surfaces it locally on any platform instead of waiting for windows-latest.

Agree, this makes sense. Let's make it unconditional. Thanks!

junzh0u · 2026-07-17T18:03:29Z

This work surfaced a real bug on Windows. Fix attempt in 16800f6, with a question about an implementation choice (see inline comment).

…settings The daemon now tracks client activity (every accepted connection, plus each handler task's completion so long streaming index runs count to their end) and exits after daemon.idle_timeout_minutes (default 180, 0 = never) without it, instead of holding the embedding model in RAM forever. The idle exit goes through the existing graceful shutdown path, so it leaves a last_exit marker (reason "idle_timeout") and clients transparently and silently restart the daemon on the next request. - settings: new DaemonSettings dataclass with idle_timeout_minutes, parsed from an optional daemon: section in global_settings.yml (absent section and missing file both fall back to the default) - protocol: HeartbeatRequest/HeartbeatResponse (daemon-side dispatch only; the MCP client loop comes separately) and idle observability fields on DaemonStatusResponse (idle_seconds, idle_timeout_minutes, last_heartbeat_seconds) - daemon: IdleReaper with a pure should_exit predicate — exit requires a positive timeout, no external supervisor (COCOINDEX_CODE_DAEMON_SUPERVISED), no live handler task, no project indexing (new ProjectRegistry.any_indexing), and idle time past the timeout. A periodic asyncio task (60 s interval, injectable for tests along with a seconds-scale timeout on run_daemon) triggers the shutdown. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JaFKwEvJXo7WQhZvrW5Qoe

Both MCP entry points (ccc mcp and the legacy cocoindex-code serve) now run a background heartbeat loop alongside the initial background index. Each heartbeat is one short-lived daemon connection that counts as activity for the idle reaper, so the daemon never idle-exits while an MCP session is live; when the MCP process dies (even SIGKILL) the heartbeats stop and the daemon exits on the normal timeout. - client.send_heartbeat(): uses the raw non-auto-starting connect path and returns False when no compatible daemon answers — a heartbeat must never start or restart a daemon, or `ccc daemon stop` mid-session would fight the loop - server.run_heartbeat_loop(): interval = timeout/3 clamped to [30 s, 300 s], reading the timeout from global_settings.yml (default when missing); a 0 timeout disables the loop entirely Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JaFKwEvJXo7WQhZvrW5Qoe

…ment idle-exit - ccc daemon status now prints idle duration with the configured timeout (or "disabled" when 0) and the age of the last MCP heartbeat (or "never") - README: document daemon.idle_timeout_minutes in the global settings reference - CLAUDE.md: note the idle-exit + heartbeat + crash-marker behavior in the process model section Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JaFKwEvJXo7WQhZvrW5Qoe

… accept thread PipeListener.close() only closes the queued next-instance handle; it cannot cancel an in-flight ConnectNamedPipe wait, so a blocked accept() kept a pipe instance open after the daemon exited. The named pipe therefore still existed when the in-process idle tests asserted cleanup, and the test's own os.path.exists() probe completed the pending connection, making the accept thread call call_soon_threadsafe on a closed loop (the PytestUnhandledThreadExceptionWarning noise in CI). Shutdown now sets a shutting_down flag, wakes a blocked accept() on Windows with a dummy client connection, and joins the accept thread before returning, so the last pipe-instance handle is closed before run_daemon exits. POSIX behavior is unchanged (listener.close() already unblocks accept() there). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

georgeh0 · 2026-07-17T19:27:15Z

+    other connection.
+    """
+
+    def __init__(self, timeout_s: float, *, supervised: bool) -> None:


nit: may consider using a stronger type (e.g. timedelta) here, with clearer semantic

georgeh0 · 2026-07-17T19:38:46Z

+        self.timeout_s = timeout_s
+        self.supervised = supervised
+        self.last_activity = time.monotonic()
+        self.last_heartbeat: float | None = None


last_heartbeat is specific to MCP. It's not interesting for the daemon, and not related to idle reaping. Also the invariance of "heartbeat connections refresh last_activity like any other connection" is not guaranteed here as well.

So I feel for separation of concern, last_heartbeat may not belong here.

If we really want to track and report it, we may consider doing it on the MCP side.

WDYT?

Review feedback on #215: a bare float-with-suffix-naming carries the unit only by convention. IdleReaper and run_daemon now take timedelta, converting to seconds only at the comparison against monotonic timestamps and on the wire. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EQpbhCAsYRJuwL7X83fzDu

…activity Review feedback on #215: last_heartbeat is MCP-specific state that the daemon has no use for — it played no part in idle reaping and existed only so 'ccc daemon status' could echo it back. A heartbeat's real effect is the connection itself, which the accept path already records as activity. Remove the field from IdleReaper, the wire protocol, and the status output. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EQpbhCAsYRJuwL7X83fzDu

junzh0u · 2026-07-17T21:13:42Z

@georgeh0 addressed both inline comments. My ungating proposal is purely optional, can follow up later if needed.

POSIX doesn't need it (listener.close() makes accept() raise), but it's harmless there and keeps the path exercised on every platform instead of only on Windows CI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vgj6XtKW6dHST75F5VtavC

georgeh0 · 2026-07-18T02:39:27Z

Thanks for saving unnecessary cost for cocoindex-code!

badmonster0 · 2026-07-18T03:46:25Z

thanks @junzh0u !

junzh0u force-pushed the feat/daemon-idle-timeout branch from 27f477d to 6815206 Compare July 16, 2026 23:17

badmonster0 requested a review from georgeh0 July 17, 2026 03:11

junzh0u commented Jul 17, 2026

View reviewed changes

junzh0u and others added 4 commits July 17, 2026 12:35

georgeh0 reviewed Jul 17, 2026

View reviewed changes

junzh0u force-pushed the feat/daemon-idle-timeout branch from 16800f6 to 6bf123f Compare July 17, 2026 19:39

junzh0u and others added 2 commits July 17, 2026 14:09

georgeh0 merged commit df021fb into cocoindex-io:main Jul 18, 2026
4 checks passed

junzh0u deleted the feat/daemon-idle-timeout branch July 18, 2026 04:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(daemon): idle timeout with MCP heartbeat — stop holding the embedding model in RAM forever - #215

feat(daemon): idle timeout with MCP heartbeat — stop holding the embedding model in RAM forever#215
georgeh0 merged 7 commits into
cocoindex-io:mainfrom
junzh0u:feat/daemon-idle-timeout

junzh0u commented Jul 15, 2026 •

edited

Loading

Uh oh!

junzh0u Jul 17, 2026

Uh oh!

georgeh0 Jul 18, 2026

Uh oh!

junzh0u commented Jul 17, 2026

Uh oh!

georgeh0 Jul 17, 2026

Uh oh!

georgeh0 Jul 17, 2026

Uh oh!

junzh0u commented Jul 17, 2026

Uh oh!

georgeh0 commented Jul 18, 2026

Uh oh!

Uh oh!

badmonster0 commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

junzh0u commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Tests

Uh oh!

junzh0u Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

georgeh0 Jul 18, 2026

Choose a reason for hiding this comment

Uh oh!

junzh0u commented Jul 17, 2026

Uh oh!

georgeh0 Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

georgeh0 Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

junzh0u commented Jul 17, 2026

Uh oh!

georgeh0 commented Jul 18, 2026

Uh oh!

Uh oh!

badmonster0 commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

junzh0u commented Jul 15, 2026 •

edited

Loading