Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ src/contextstore/kvservice_client/_pb/*.pyi
*.swp
*.swo
*~
.workbuddy/
.codegraph/

# Data & logs
/tmp/
Expand Down
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,8 @@ This keeps hot GETs to a single round trip while remaining correct under concurr

**Multi-endpoint + SGE reads.** A reader groups an object's stripes by owning node and issues one stripe-subset GET per node concurrently, so aggregate cold-read bandwidth scales with node count. The SGE variant (wire tag 15) additionally carries the client's destination segment list, letting the server RDMA-WRITE each byte range straight into the caller's scattered buffers — no client-side staging copy. Both RDMA control-plane sockets run with `TCP_NODELAY`. Measured on 2 nodes × 2 NVMe (fio ceiling ~28 GB/s): 6.4–8.8 GB/s per synchronous client stream, 23 GB/s aggregate at 8 concurrent streams, disk-byte-accounted cold reads.

**Multi-rail parallel read.** The same stripe-subset GET generalizes from *per-node* to *per-rail* concurrency: one client worker attaches N independent RDMA paths (NICs / RXE devices), the scheduler partitions the object's stripes across them (`kv-service/client-rs/src/multi_rail.rs`), and each rail transfers its subset concurrently into disjoint regions of the same destination buffer. The upper read interface and the on-disk stripe layout are unchanged; a single rail is a byte-exact pass-through of today's path. Because the rails run concurrently, the completion path needs memory-consistency guards against a late RDMA WRITE landing in a freed/reused buffer (pin-until-completion, an epoch buffer guard, and two-phase commit with per-stripe xxh3-64 verification). Failure semantics in v1 are strict: if any required rail fails, the whole request fails safely — no partial data, no transparent in-request retry. Soft-RoCE bring-up (`setup-wsl2-rxe.sh`) and the end-to-end / concurrency examples live under `kv-service/client-rs/examples/`; full design notes are in `docs/multi-rail-read-design.md`.

---

## Quick start
Expand Down
62 changes: 62 additions & 0 deletions build-and-test.cmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
@echo off
REM ============================================================================
REM build-and-test.cmd — 一键构建 + 测试 ContextStore 多轨代码
REM
REM 用途:在 Windows 上验证 multi-rail 代码可编译、测试全绿、基准可跑。
REM 不依赖 RDMA 硬件(默认 feature),不依赖 GPU。
REM
REM 前置:已安装 Rust(rustup)。本脚本会自动探测 toolchain 路径。
REM 用法:双击运行,或在终端执行 build-and-test.cmd
REM ============================================================================
setlocal enabledelayedexpansion

REM ---- 1. 定位 Rust toolchain(优先 rustup 默认位置)----
if exist "%USERPROFILE%\.cargo\bin\cargo.exe" (
set "CARGO_BIN=%USERPROFILE%\.cargo\bin"
goto :found
)
if exist "%LOCALAPPDATA%\Programs\Rust" (
set "CARGO_BIN=%LOCALAPPDATA%\Programs\Rust\bin"
goto :found
)
echo [错误] 未找到 cargo.exe。请先安装 Rust: https://rustup.rs
exit /b 1

:found
echo [信息] 使用 toolchain: %CARGO_BIN%
set "PATH=%CARGO_BIN%;%PATH%"

REM 重要:RUSTC 必须是 Windows 反斜杠绝对路径。
REM 若用正斜杠或依赖 PATH 中的 0 字节 shim,indexmap 等 crate 的
REM autocfg 构建探测会失败,导致 tower 报 E0107。
set "RUSTC=%CARGO_BIN%\rustc.exe"

REM ---- 2. 构建目录(避开可能的文件锁;可改回项目内 target)----
if "%CARGO_TARGET_DIR%"=="" set "CARGO_TARGET_DIR=%TEMP%\contextstore-target"
set "CARGO_INCREMENTAL=0"
echo [信息] CARGO_TARGET_DIR=%CARGO_TARGET_DIR%

cd /d "%~dp0kv-service\client-rs" || (echo [错误] 未找到 kv-service\client-rs & exit /b 1)

echo.
echo ============ 1/3 编译(default feature,无 RDMA 依赖)============
cargo build || goto :fail

echo.
echo ============ 2/3 多轨功能测试 ============
cargo test --test multi_rail_mock || goto :fail

echo.
echo ============ 3/3 多轨带宽容量模型(8 轨)============
set "MAX_RAILS=8"
cargo run --release --bin cs-mock-bench || goto :fail

echo.
echo [成功] 编译通过、测试全绿、基准可跑。
echo 下一步:开 Issue / 设计提案与社区对齐(见 docs/multi-rail-read-design.md)。
exit /b 0

:fail
echo.
echo [失败] 请把上面的报错贴回给助手。
exit /b 1
Loading