perf(shard): hot-path lock quick wins + event-driven SPSC wake (removes the ~1ms cross-shard floor)

.add/CONVENTIONS.md

@@ -0,0 +1,10 @@
+# CONVENTIONS  (living documentation — set once, kept for the whole project)
+<!-- evidence-grounded: CLAUDE.md "Coding Rules", Cargo.toml, .github/workflows -->
+
+Language/framework: Rust 1.94 (edition 2024, MSRV enforced in CI) · runtimes: monoio (default, io_uring) + tokio (portability/CI)
+Folders: production code in `src/<module>/` (directory-module convention: `mod.rs` + split read/write files); tests in `tests/` (integration, real server instances — no mocking) + `#[cfg(test)]` in `mod.rs`; benches in `benches/` (Criterion); fuzz targets in `fuzz/fuzz_targets/`. ADD task files: `.add/tasks/<slug>/TASK.md` (task tests/src point INTO the repo tree, declared per-task).
+Naming: snake_case files/functions · PascalCase types · SCREAMING_SNAKE consts · crate:: imports in submodules (never super::super)
+Lint/format: `cargo fmt --check` · `cargo clippy -- -D warnings` (default + tokio,jemalloc feature sets) · unsafe audit (`scripts/audit-unsafe.sh`) · unwrap ratchet (`scripts/audit-unwrap.sh`) — all enforced in CI
+Errors: command errors are `Frame::Error(Bytes)` with Redis-compatible `-ERR …` text on dispatch paths (no Result); library code uses `thiserror`; `anyhow` only in main.rs/tests; no unwrap/expect in library code
+Architecture: thread-per-core shared-nothing shards; per-shard locks only (parking_lot, never std::sync; never held across .await); no global locks on the write path; no allocation on hot paths (command dispatch, protocol parse, event loop, io drivers); cross-shard communication via flume/SPSC messages only; dual-runtime: all runtime-specific code compiles under both feature sets; Linux-only code behind `#[cfg(target_os = "linux")]` with non-Linux fallback; files ≤1500 lines (split into directory modules)
+Verification: red/green TDD per task; full local CI parity via OrbStack `moon-dev` before push; benchmark numbers from Linux VM only; new parsers get fuzz targets; new atomic state machines get loom models

.add/GLOSSARY.md

@@ -0,0 +1,40 @@
+# GLOSSARY  (one name per concept — used everywhere: specs, contracts, code)
+# Moon domain terms — evidence-grounded: src/ module tree, CLAUDE.md, README.md
+
+shard: a thread-per-core unit owning one keyspace slice, its event loop, DashTable, WAL writer, and blocking/pubsub registries; the unit of shared-nothing isolation (src/shard/).
+SPSC mesh: the grid of single-producer/single-consumer ring channels (256 entries) carrying ShardMessage between shards; the ONLY sanctioned cross-shard mutation path (src/shard/mesh.rs).
+shard event loop: the per-shard select! loop (monoio or tokio) draining I/O, SPSC messages, and the 1ms tick; nothing on it may block (src/shard/event_loop.rs).
+ShardSlice: the thread-local, lock-free per-shard state access path (`with_shard`) meant to replace the locked Arc<ShardDatabases>; migration in progress, gated on `is_initialized()` (src/shard/slice.rs).
+ShardDatabases: the legacy Arc-shared, RwLock-per-DB store every shard can reach — the shared-nothing leak the ShardSlice migration removes (src/shard/shared_databases.rs).
+Frame: the RESP protocol value enum (SimpleString, BulkString, Array, Error, …); command errors are Frame::Error, never Result (src/protocol/frame.rs).
+DashTable: the per-shard extendible-hashing table (segments × 60 slots) holding CompactKey→Entry (src/storage/dashtable/).
+CompactKey / CompactValue: SSO entry types — keys ≤23B and values ≤12B inline, heap beyond via tagged pointer (src/storage/compact_value.rs).
+WAL (wal_v3): the per-shard write-ahead log; 1ms buffered flush, 1s fdatasync under everysec (src/persistence/wal_v3/).
+AOF: append-only-file persistence layered on the WAL with rewrite/manifest machinery (src/persistence/aof/).
+segment (vector): the FT.* index lifecycle unit — mutable (brute-force) → compact → immutable (HNSW + quantized codes); merged in the background (src/vector/segment/).
+cross-shard latency floor: the ~1ms minimum cross-shard reply latency on monoio caused by the pending_wakers 1ms-tick relay (event_loop.rs:1049) — v1 milestone target.
+hot path: command dispatch, protocol parsing, shard event loop, io drivers — the zones where allocation and locks are forbidden (CLAUDE.md "Allocations on Hot Paths").
+2026-06 architecture review: the principal-level performance review (this session) cataloguing shared-nothing violations, dispatch-floor causes, and event-loop blocking work; source of v1+ milestone priorities.
+
+# ADD method vocabulary (domain-standard names; bridges to legacy terms)
+GOAL: the one durable outcome a project (and each milestone) runs toward — the loop's orientation anchor, declared as the lowercase `goal:` line in PROJECT.md / MILESTONE.md and surfaced by status/guide every session; distinct from a task's §1 Must (a single required behavior, not the whole-project outcome).
+deep verify: the deepened Verify evidence (v20) required beyond passing tests — for a task that produced code, that every new symbol is referenced (wiring) and no new dead/unused code exists; for prose/non-code, a recorded no-skim semantic read; which path applies is resolver-judged and the engine never classifies (a rubric, not add.py).
+onboarding: the install -> first-milestone path (formerly "on-ramp").
+primary flow: the solid forward path of the flow diagram — a phase starts only when its input exists (formerly "forward spine").
+cross-cutting concern: a concern running through every step rather than being one step — security, testing, observability, cost (formerly "spine / continuous concern").
+working state: everything an agent loads each session — skill router, active phase, PROJECT/MILESTONE/TASK, state.json (formerly "state surface").
+audit trail: the reference record read by people, never auto-loaded into agent context (formerly "story surface").
+method rationale: the why behind every rule — the AIDD book, loaded on demand, never duplicated (formerly "trust layer").
+failing-first suite: the test suite written before code, confirmed red for the right reason — a missing implementation (formerly "red safety net").
+non-functional review: the deliberate post-evidence check of what tests rarely catch — concurrency, security, architecture (formerly "blind-spot checks").
+change scope: the files a locked run may and may not touch (formerly "touch-boundary"; the <touch_boundary> XML prompt tag keeps its name).
+automated quality gate: the evidence-based Verify resolver under autonomy auto — may auto-PASS on complete evidence; security always escalates (formerly "evidence auto-gate").
+autonomy level: the per-task Verify resolver setting — auto (default) or conservative; declared in the TASK.md header, human-reviewed at the freeze (formerly "autonomy dial").
+living documentation: the durable project artifacts — conventions, glossary, frozen contracts — that outlive any particular code (formerly "survivor layer").
+scope level: the granularity a decision lives at — intake level (request -> versioned scope), milestone level, setup/foundation level, task level (formerly "altitude").
+baseline approval: the one human gate that freezes the AI-drafted foundation, first scope, and first contract together — runs as `add.py lock` (formerly "the lock-down").
+lesson learned: a single learning a loop produces, tagged by the competency it improves — the `- [DDD · open]` grammar and deltas.md/`add.py deltas` machine names stay (formerly "competency delta").
+lowest-confidence flag: the AI's ranked declaration of the 1–2 points most likely to be wrong in what a human is asked to approve — each with why + cost-if-wrong; the ⚠ glyph keeps its name as the machine marker (formerly "least-sure flag").
+decision point: a stop for human judgment — the contract-freeze approval, an escalated verify gate, intake confirmation, milestone close; the machine names seam (--json owner enum, decide key) and seam-audit (CI job) keep their names (formerly "seam").
+retrospective consolidation: gathering confirmed lessons learned at milestone close and writing them append-only into the versioned foundation — human-confirmed, never self-approved; the machine names fold.md, the folded status, and add.py deltas keep their names (formerly "the fold / fold ritual").
+specification bundle: a task's spec, scenarios, contract, and failing tests drafted as one piece and approved by a person once at the contract freeze (formerly "the one-approval front").

.add/MODEL_REGISTRY.md

@@ -0,0 +1,6 @@
+# MODEL_REGISTRY  (which AI wrote this project — for reproducibility & audit)
+
+Model: Claude (Anthropic) — Claude Code CLI
+Version: claude-fable-5 (adopted at ADD onboarding; earlier code AI-assisted with prior Claude versions per git history)
+Adopted: 2026-06-11
+Notes: pre-ADD codebase (v0.3.0) was substantially AI-assisted (Claude Code review runs on PRs per CI). Re-run the playbook golden-cases before changing this.

.add/PROJECT.md

@@ -0,0 +1,44 @@
+# PROJECT — living documentation (cross-milestone context)
+
+> The durable foundation that outlives every milestone and feeds context into each
+> TDD⇄ADD loop. Read this FIRST in any session. Keep it lean — one screen, not a
+> manual. Map to the AIDD diagram: Domain = DDD · Spec = SDD (living document) ·
+> UI/UX = UDD. When a loop reveals a gap here, come back and update this file —
+> that is the re-entrant arrow from the engine down to the foundation.
+
+slug: moon · stage: production · updated: 2026-06-11
+goal: a Redis-compatible server whose thread-per-core architecture measurably out-scales Redis on multi-core hardware — without sacrificing protocol compatibility or durability semantics
+
+---
+
+## Domain (DDD) — the language and the boundaries
+<!-- evidence-grounded: README.md, CLAUDE.md, src/ module tree -->
+- Core concepts: Shard (thread-per-core owner of a keyspace slice) · Frame (RESP protocol unit) · DashTable (per-shard hash storage) · CompactKey/CompactValue (SSO entry types) · SPSC mesh (cross-shard dispatch channels) · WAL/AOF (per-shard durability log) · Segment (vector index lifecycle unit: mutable → immutable)
+- Bounded contexts / modules: `protocol/` (RESP parse/serialize) · `shard/` (event loop, dispatch, coordinator) · `storage/` (dashtable, eviction, tiered) · `persistence/` (wal_v3, aof, page_cache) · `vector/` + `command/vector_search/` (FT.* engine) · `command/` (dispatch tables) · `io/` (uring driver) · `runtime/` (monoio/tokio abstraction) · cross-cutting: `acl/ pubsub/ replication/ cluster/ blocking/ transaction/ scripting/ tracking/`
+- Invariants that must always hold:
+  - Shared-nothing per shard: no global locks on the command write path (CLAUDE.md "Lock Handling") — **currently violated in ≥8 places (2026-06 architecture review)**
+  - Malformed client input never crashes the server (`parse_frame_zerocopy` returns `Frame::Null`)
+  - No new `unsafe` without approval + `// SAFETY:` comment (UNSAFE_POLICY.md)
+  - No alloc in command dispatch / protocol parse / shard event loop / io drivers
+  - All runtime-specific code compiles under both `runtime-monoio` and `runtime-tokio`
+
+## Spec / Living Document (SDD) — what we are building, now
+- Active milestone → `.add/milestones/v1-shared-nothing/MILESTONE.md` (see `add.py status`)
+- Frozen contracts (living docs): RESP2/RESP3 wire compatibility with Redis (external, immutable); CI matrix (fmt, clippy ×2, tests ×2, MSRV 1.94, unsafe/unwrap audits, fuzz)
+- Settled vs still open: settled — thread-per-core + SPSC mesh architecture, monoio default on Linux. Open — sub-linear multi-shard scaling (root causes mapped in 2026-06 review: leaky shared-nothing + 1ms monoio wake floor)
+
+## Users (UDD) — UI/UX: design before code
+- No UI — surface is the **Redis wire protocol** (RESP2/RESP3) plus CLI flags (`--port --shards --appendonly --dir …`) and INFO/Prometheus metrics.
+- Primary users & jobs: backend engineers replacing Redis for higher throughput/lower memory per node; benchmark parity tooling (`scripts/bench-*.sh`, `redis-benchmark`).
+- Core flows: connect → AUTH/HELLO → command pipeline → responses; ops flows: persistence reload, replication, FT.* vector search.
+- UI states: error surface = RESP error frames with Redis-compatible messages (`-ERR …`); never a crash, never a hang.
+- Design source of truth → README.md (architecture diagram + command reference).
+
+## Key Decisions (append-only)
+| date | decision | why | outcome |
+|------|----------|-----|---------|
+| pre-ADD | thread-per-core, monoio/io_uring default on Linux | syscall/wakeup cost dominates at high QPS | shipped v0.3.0 |
+| pre-ADD | per-shard WAL, no global append lock | Redis single-AOF serializes at depth | AOF advantage grows with pipeline depth |
+| pre-ADD | SSO CompactKey(23B)/CompactValue(12B) | per-key memory vs naive Arc<String> | lower RSS than Redis per key |
+| 2026-06-11 | adopt ADD; v1 milestone = restore shared-nothing integrity + remove cross-shard latency floor (review priorities 1·2·5) | 2026-06 architecture review: these two themes explain sub-linear multi-shard scaling | pending |
+| 2026-06-11 | FT.SEARCH off-event-loop + WAL group commit deferred to v2 | different themes (event-loop blocking; durability); keep v1 one outcome | recorded in v1 Out list |

.add/SETUP-REVIEW.md

@@ -0,0 +1,19 @@
+# SETUP REVIEW — moon
+
+production · brownfield · drafted by claude-fable-5 @ 2026-06-11
+
+| # | Decision | Lands in | Tag | Why / Evidence |
+|---|----------|----------|-----|----------------|
+| 1 | QW4 (per-shard command counters) assumes the Prometheus `counter!` path can split per-shard, and that scrape-read aggregation freshness is acceptable | first-contract | `guessed` | The `metrics` crate's recorder API may not register per-shard handles cleanly (src/admin/metrics_setup.rs uses global macros). Fallback shrinks the win to the bespoke `TOTAL_COMMANDS` atomic only. |
+| 2 | QW8 (client registry off the per-batch path) assumes CLIENT LIST tolerates on-change/atomic metadata updates instead of per-batch write-lock updates | first-contract | `guessed` | No test pins CLIENT LIST `cmd`/`age` freshness; redis-tooling expectations inferred. Fallback: per-connection atomics (still lock-free). |
+| 3 | v1 milestone scopes review priorities 1·2·5 together and defers FT.SEARCH offload + WAL group commit to v2 | scope | `guessed` | My sizing judgment: one outcome (restore shared-nothing + remove latency floor); priorities 3·4 are different themes. You may prefer FT.SEARCH offload sooner if vector workloads are imminent. |
+| 4 | Exit criterion "cross-shard SET/GET p99 < 1ms on monoio" is achievable once the 1ms waker relay is replaced | scope | `guessed` | Inferred from event_loop.rs:1049 comments; no prototype of the eventfd wake exists yet. If wrong: criterion relaxes to "p99 materially below current baseline". |
+| 5 | First task = the quick-wins batch (8 items in one task) rather than 8 micro-tasks or starting with the ShardSlice migration | scope | `guessed` | Items are mechanical and share one bench baseline; doing them first de-noises the bigger tasks' measurements. ShardSlice (your priority 1) lands as task 3 with the frozen contract named in MILESTONE.md. |
+| 6 | Project goal sentence ("out-scales Redis on multi-core hardware …") | PROJECT.md | `guessed` | Inferred from README positioning + bench tooling; the repo states no explicit mission sentence. |
+| 7 | The 8 quick-win code targets exist as described (Mutex'd WAL sender, RwLock'd offsets, global TOTAL_COMMANDS, per-byte backlog loop, Vec::remove(0), unpruned key maps, per-batch registry write, missing TCP_NODELAY) | first-contract | `evidence-grounded` | Each verified by direct read this session: shared_databases.rs:159 · spsc_handler.rs:3089 + state.rs:138 · metrics_setup.rs:401 · backlog.rs:35 · uring_handler.rs:346 · vector/store.rs:192 (zero `.remove` calls) · client_registry.rs:80 + handler_sharded/mod.rs:1868 · grep shows set_nodelay only in moon-bench.rs. |
+| 8 | Domain map, invariants, conventions, glossary terms | PROJECT.md / CONVENTIONS.md / GLOSSARY.md | `evidence-grounded` | CLAUDE.md (coding rules, gotchas), README.md, src/ module tree, Cargo.toml. |
+| 9 | Dependency allowlist | dependencies.allowlist | `evidence-grounded` | Cargo.toml `[dependencies]` + feature-gated crates, v0.3.0. |
+| 10 | Stage = production (full rigor, observe loop) | foundation | `evidence-grounded` | v0.3.0 shipped with signed release assets, CI matrix, fuzz + safety audits (git history, .github/workflows). |
+| 11 | Behavior parity is a frozen contract for all v1 tasks (RESP bytes, INFO fields, 132 consistency tests) | scope | `evidence-grounded` | scripts/test-consistency.sh + CI define the parity surface; v1 is perf-only by the user's request. |
+
+Sign: confirm in chat → the agent runs `add.py lock --by "Tin Dang"` (typing it yourself works too)

.add/dependencies.allowlist

@@ -0,0 +1,66 @@
+# dependencies.allowlist — one package per line; CI rejects anything not listed.
+# Defeats the AI "invented a plausible package name" supply-chain risk.
+# evidence-grounded: Cargo.toml [dependencies] (+ optional/feature-gated crates) as of v0.3.0.
+
+atoi
+bumpalo
+bytes
+memchr
+smallvec
+thiserror
+mimalloc
+crc32c
+crossbeam-utils
+flume
+atomic-waker
+tokio
+tokio-util
+clap
+tracing
+tracing-subscriber
+anyhow
+futures
+ordered-float
+phf
+rand
+crc16
+crc32fast
+arc-swap
+parking_lot
+itoa
+xxhash-rust
+ringbuf
+core_affinity
+once_cell
+mlua
+sha1_smol
+sha2
+hex
+uuid
+ctrlc
+metrics
+metrics-exporter-prometheus
+rustls
+rustls-pemfile
+aws-lc-rs
+tokio-rustls
+monoio-rustls
+roaring
+serde
+serde_json
+socket2
+memmap2
+lz4_flex
+dashmap
+# feature-gated / platform
+monoio
+io-uring
+libc
+jemallocator
+cudarc
+# dev/test/bench
+criterion
+loom
+proptest
+tempfile
+redis