Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ dist/
# Trained adapter artifacts (never committed; see README)
artifacts/

# Downloaded checkpoints (README quick start puts them here)
models/

# macOS
.DS_Store

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ edge0-35b, 2.8 for edge0-8b** (MMLU-Pro is even above the base). Max 100:

## Benchmark

Measured with `examples/bench.py` (3.3k-token prompt prefill → 10 sampled
Measured with `examples/bench.py` (3.3k-token prompt prefill → 10 greedy
warmup steps → 200 timed sampled decode tokens, 2 runs per tier):

| Tier | Decode speed | Prefill throughput (cold / warm)* | Peak active memory** | Test machine |
Expand Down
2 changes: 1 addition & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ engine.close() # 释放 mmap / 专家缓存

## 性能实测

`examples/bench.py` 实测(3.3k token prompt prefill → 10 步采样 warmup →
`examples/bench.py` 实测(3.3k token prompt prefill → 10 步贪心 warmup →
200 token 计时段,每档 2 轮):

| 档位 | 解码速度 | Prefill 吞吐(冷/热)* | 峰值 active 内存** | 测试机器 |
Expand Down
487 changes: 487 additions & 0 deletions docs/nvidia.md

Large diffs are not rendered by default.

145 changes: 145 additions & 0 deletions docs/plans/jetson-orin-nano-review.md

Large diffs are not rendered by default.

427 changes: 427 additions & 0 deletions docs/plans/jetson-orin-nano.md

Large diffs are not rendered by default.

241 changes: 241 additions & 0 deletions docs/plans/rtx6000-task6-handoff.md

Large diffs are not rendered by default.

748 changes: 692 additions & 56 deletions examples/bench.py

Large diffs are not rendered by default.

Loading