Package the CPU-side core as winnow-core, and lock what the GPU half needs

.gitignore

@@ -6,7 +6,7 @@
 venv/
 .python-version
 
-# Secrets — never commit
+# Secrets - never commit
 .env
 .env.local
 .env.*.local
@@ -20,6 +20,11 @@ venv/
 experiments/eval_results.json
 experiments/eval_outputs*.json
 
+# Python build artifacts (winnow-core wheel/sdist)
+/dist/
+/build/
+*.egg-info/
+
 # Misc
 .DS_Store
 *.log

README.md

@@ -1,7 +1,7 @@
 # Winnow
 
 > Voice-first, real-time token compression for LLM pipelines.
-> Speak, and we transcribe, prune, and pipe — losing words, keeping meaning.
+> Speak, and we transcribe, prune, and pipe - losing words, keeping meaning.
 
 Winnow is a hackathon project that compresses live speech transcripts in real time before they reach an LLM, demonstrably without losing meaning. Spoken language is verbose; LLM input tokens cost money. Winnow shows you the savings, the diff, and a side-by-side fidelity audit, then lets you talk to the compressed transcript through a Claude-Projects-style workspace.
 
@@ -12,14 +12,14 @@ Winnow is a hackathon project that compresses live speech transcripts in real ti
 Two views, one pipeline.
 
 ### Compare
-Watch compression happen live. Raw transcript on the left as you speak, pruned transcript on the right within milliseconds of each pause. Pruned words are struck through. A stats bar tracks running token counts, % saved, $ saved per the chosen LLM, average pipeline latency, and a sparkline of compression ratio over time. A built-in A/B Q&A box asks the same question against both transcripts in parallel — when the answers match, you have proof compression preserved the buried details.
+Watch compression happen live. Raw transcript on the left as you speak, pruned transcript on the right within milliseconds of each pause. Pruned words are struck through. A stats bar tracks running token counts, % saved, $ saved per the chosen LLM, average pipeline latency, and a sparkline of compression ratio over time. A built-in A/B Q&A box asks the same question against both transcripts in parallel - when the answers match, you have proof compression preserved the buried details.
 
 ### Learn
-A voice-first Claude-Projects-style workspace built around the compressed transcript. Tap the mic, talk naturally, each pause auto-sends a chat message. Claude streams answers back. One-tap **Insights** generate a summary, decision log, action items, flashcards (flippable carousel), and glossary — all backed by Anthropic prompt caching so multi-turn sessions stay cheap.
+A voice-first Claude-Projects-style workspace built around the compressed transcript. Tap the mic, talk naturally, each pause auto-sends a chat message. Claude streams answers back. One-tap **Insights** generate a summary, decision log, action items, flashcards (flippable carousel), and glossary - all backed by Anthropic prompt caching so multi-turn sessions stay cheap.
 
 ### Stage-safety features
 - **Swappable input source.** Live mic and a pre-recorded JSON fixture both implement the same `TranscriptSource` interface. One click swaps them; downstream code can't tell which is active. If the live mic ever fails on stage, the fallback is a literal replay of you doing the demo, with original utterance timing.
-- **In-app fixture recorder.** Capture a live session, download as JSON, drop into `public/fixtures/demo-transcript.json` — and your fallback becomes you.
+- **In-app fixture recorder.** Capture a live session, download as JSON, drop into `public/fixtures/demo-transcript.json` - and your fallback becomes you.
 - **Director mode.** Keyboard shortcuts (`Space` start/stop, `R` swap source, `1/2/3` fire preset Q&A probes, `X` clear, `?` show overlay) let you drive the demo without touching a trackpad.
 
 ---
@@ -66,18 +66,18 @@ flowchart TD
 ```
 
 **Token-space (which words survive).** The text fans out to two compressor
-families at once — LLMLingua-2 (+ LongLLMLingua's query-aware reranker/perplexity
+families at once - LLMLingua-2 (+ LongLLMLingua's query-aware reranker/perplexity
 arm) and AttentionRAG (attention-guided sentence pruning). Both produce per-token
 keep decisions, which Winnow merges token-by-token over the *original* text under a
-user-picked boolean rule: **intersection** (keep only if both kept — aggressive) or
-**union** (keep if either kept — recall-safe). The merge aligns both methods to one
+user-picked boolean rule: **intersection** (keep only if both kept - aggressive) or
+**union** (keep if either kept - recall-safe). The merge aligns both methods to one
 canonical token sequence rather than doing string membership, and falls back to
 LLMLingua-only if AttentionRAG gates everything out.
 
 **Model-space (how it's answered).** The compressed prompt then goes either to a
 **black-box API** (Claude / GPT, provider-agnostic), or to our **self-hosted**
 workers: **Qwen + TurboQuant** (KV-cache quantized to ~4 bits, 3–4× smaller via a
-data-free random-rotation `DynamicCache`), or **LCLM + TurboQuant** — LCLM
+data-free random-rotation `DynamicCache`), or **LCLM + TurboQuant** - LCLM
 compresses the context into a few latent soft tokens (fewer KV *entries*) while
 TurboQuant compresses the *bits per entry* of the same decoder cache, so the savings
 multiply.
@@ -138,7 +138,7 @@ multiply.
 
 **Compression**
 - LLMLingua-2 (`microsoft/llmlingua-2-xlm-roberta-large-meetingbank`)
-- Hosted on Modal — T4 GPU, persistent HF cache volume, memory + GPU snapshots for ~1s cold start
+- Hosted on Modal - T4 GPU, persistent HF cache volume, memory + GPU snapshots for ~1s cold start
 - FastAPI / Pydantic / Uvicorn as a thin local proxy
 
 **LLM**
@@ -147,6 +147,32 @@ multiply.
 
 ---
 
+## Installing just the compression core (no GPU, no Modal, no keys)
+
+The CPU-side half of Winnow ships as a standalone distribution, `winnow-core`.
+It is the part an external consumer can actually use without a GPU: the
+AttentionRAG selection logic (`attentionrag`), the two-compressor token merge
+(`token_merge`), and the model-artifact guard (`model_guard`).
+
+Prerequisites: Python 3.11, 3.12 or 3.14. Nothing else - the core has zero
+runtime dependencies.
+
+```bash
+python3 -m venv .venv && . .venv/bin/activate
+pip install build && python -m build          # writes dist/winnow_core-0.1.0-*.whl
+pip install dist/winnow_core-0.1.0-py3-none-any.whl
+python -m attentionrag.test_core              # 10/10
+```
+
+The model-touching half (`attentionrag.hf_backend`, and reading real tensors
+through `model_guard.safe_load_state_dict`) needs torch and transformers:
+
+```bash
+pip install "dist/winnow_core-0.1.0-py3-none-any.whl[hf]"
+```
+
+Exact resolved versions are recorded in `requirements-hf.lock`.
+
 ## Local setup
 
 Prerequisites:
@@ -191,14 +217,14 @@ COMPRESS_BACKEND_URL=http://localhost:8000     # default, leave as-is
 
 ### 4. Run everything
 
-In one terminal — the GPU worker proxy:
+In one terminal - the GPU worker proxy:
 ```bash
 # from repo root
 ./run.sh                # deploys (if needed) + warms + serves FastAPI on :8000
 ```
 Honors `SKIP_DEPLOY=1` and `SKIP_WARMUP=1` for faster restarts during dev.
 
-In another terminal — the frontend:
+In another terminal - the frontend:
 ```bash
 cd web
 npm run dev             # http://localhost:3000
@@ -212,15 +238,15 @@ Open http://localhost:3000.
 
 1. Click **Start** in the top right. The browser asks for mic permission.
 2. Talk. Watch the raw column fill on the left, compressed on the right with strike-throughs on dropped words. Stats bar updates live.
-3. Hit one of the preset probes under the **Q&A** box (or type a question) — answers fire against raw and compressed in parallel. A green "answers match" badge is the proof.
+3. Hit one of the preset probes under the **Q&A** box (or type a question) - answers fire against raw and compressed in parallel. A green "answers match" badge is the proof.
 4. Switch to **Learn**. The compressed transcript is auto-pinned as a source.
-5. Tap the big neon mic. Talk to Claude about what was said. Each pause auto-sends. Tap any **Insight** card to generate flashcards, action items, glossary, etc. — results stay cached.
+5. Tap the big neon mic. Talk to Claude about what was said. Each pause auto-sends. Tap any **Insight** card to generate flashcards, action items, glossary, etc. - results stay cached.
 
 ### Stage-safety toggles
 
-- **Live mic ⇄ Recorded** at the top — instant swap. Recorded plays `web/public/fixtures/demo-transcript.json` with original timing.
-- **Rec fixture / Save** — capture a fresh recorded fallback from your current live session and download as JSON. Replace `public/fixtures/demo-transcript.json` with it so the fallback is literally you.
-- **Director** button or `?` key — pulls up the keyboard cheat sheet.
+- **Live mic ⇄ Recorded** at the top - instant swap. Recorded plays `web/public/fixtures/demo-transcript.json` with original timing.
+- **Rec fixture / Save** - capture a fresh recorded fallback from your current live session and download as JSON. Replace `public/fixtures/demo-transcript.json` with it so the fallback is literally you.
+- **Director** button or `?` key - pulls up the keyboard cheat sheet.
 
 ---
 
@@ -230,13 +256,13 @@ Open http://localhost:3000.
 Your Deepgram API key doesn't have project-admin scope, so `/v1/auth/grant` rejects it. Winnow's `/api/deepgram-token` falls back to shipping the raw key to the browser (fine for local). Make sure `DEEPGRAM_API_KEY` is set in `web/.env`. Restart `npm run dev` after editing.
 
 **Chat error: `invalid x-api-key` / `authentication_error`**
-`ANTHROPIC_API_KEY` is missing, malformed, or stale. Edit `web/.env`, **then restart `npm run dev`** — Next only reads env vars at boot.
+`ANTHROPIC_API_KEY` is missing, malformed, or stale. Edit `web/.env`, **then restart `npm run dev`** - Next only reads env vars at boot.
 
 **`backend unreachable` on compress**
 FastAPI isn't running. Start it: `./run.sh` from the repo root.
 
 **Modal call fails / cold start is slow**
-First call after a deploy or a long idle period builds the GPU snapshot — that's ~30s once, then restores in ~1s. `./run.sh` automatically warms the worker after deploy.
+First call after a deploy or a long idle period builds the GPU snapshot - that's ~30s once, then restores in ~1s. `./run.sh` automatically warms the worker after deploy.
 
 **Port 3000 already in use**
 A previous dev server is still up. `lsof -i :3000` to find the PID, then `kill <pid>`. Or use `npm run dev -- -p 3001`.

pyproject.toml

@@ -0,0 +1,52 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "winnow-core"
+version = "0.1.0"
+description = "Winnow's CPU-side compression core: AttentionRAG selection, the two-compressor token merge, and the model-artifact guard."
+readme = "README.md"
+requires-python = ">=3.11"
+license = { text = "MIT" }
+authors = [{ name = "Winnow" }]
+keywords = ["prompt-compression", "rag", "kv-cache", "llm"]
+classifiers = [
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.14",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+
+# The core is deliberately STDLIB ONLY. `attentionrag.core` is the paper's
+# selection logic behind a Backend protocol, `token_merge` is pure string/span
+# arithmetic, and `model_guard`'s validation path uses pickletools/hashlib/
+# zipfile. That is what makes this half installable and testable without a GPU.
+dependencies = []
+
+[project.optional-dependencies]
+# Only needed for `attentionrag.hf_backend` (the model-touching half) and for
+# `model_guard.safe_load_state_dict` reading real tensors. Pinned, not floated:
+# the guard's whole point is that "the same model name" must not mean different
+# bytes tomorrow, and the same argument applies to the loader.
+# numpy is pinned to 2.4.6, not the newest release: safetensors needs numpy to
+# convert torch tensors, and numpy 2.5.x publishes no cp311 wheel, which is the
+# interpreter the Modal images use. Verified by installing, not by reading.
+hf = [
+    "torch==2.14.0",
+    "transformers==5.16.1",
+    "safetensors==0.8.0",
+    "huggingface-hub==1.30.0",
+    "accelerate==1.14.0",
+    "numpy==2.4.6",
+]
+
+[project.urls]
+Homepage = "https://github.com/TC960/winnow"
+
+[tool.setuptools]
+py-modules = ["model_guard", "token_merge"]
+
+[tool.setuptools.packages.find]
+include = ["attentionrag*"]
+exclude = ["attentionrag.test_*"]

requirements-hf.lock

@@ -0,0 +1,48 @@
+# Lockfile for `pip install "winnow-core[hf]"` - the model-touching half.
+#
+# Produced by `pip freeze` in a CLEAN python 3.11 venv after installing the
+# built wheel with the [hf] extra, on 2026-09-05. Not hand-written: every line
+# below is a package pip actually resolved and installed.
+#
+# The CORE install (`pip install winnow-core`, no extra) needs NOTHING from this
+# file - it is stdlib-only, and its freeze is one line: winnow-core itself.
+#
+# Reproduce:
+#   python3.11 -m venv .venv && . .venv/bin/activate
+#   pip install "dist/winnow_core-0.1.0-py3-none-any.whl[hf]"
+#   pip freeze
+accelerate==1.14.0
+annotated-doc==0.0.5
+anyio==4.15.1
+certifi==2026.7.22
+click==8.5.0
+filelock==3.32.5
+fsspec==2026.7.0
+h11==0.16.0
+hf-xet==1.6.0
+httpcore==1.0.9
+httpx==0.28.1
+huggingface_hub==1.30.0
+idna==3.19
+Jinja2==3.1.6
+markdown-it-py==4.2.0
+MarkupSafe==3.0.3
+mdurl==0.1.2
+mpmath==1.3.0
+networkx==3.6.1
+numpy==2.4.6
+packaging==26.3
+psutil==7.2.2
+Pygments==2.21.0
+PyYAML==6.0.3
+regex==2026.9.3
+rich==15.0.0
+safetensors==0.8.0
+shellingham==1.5.4
+sympy==1.14.0
+tokenizers==0.23.2
+torch==2.14.0
+tqdm==4.70.0
+transformers==5.16.1
+typer==0.27.2
+typing_extensions==4.16.0

requirements.txt

@@ -0,0 +1,43 @@
+aiohappyeyeballs==2.6.2
+aiohttp==3.14.3
+aiosignal==1.4.0
+annotated-doc==0.0.4
+annotated-types==0.7.0
+anyio==4.14.0
+attrs==26.1.0
+cbor2==6.1.2
+certifi==2026.6.17
+click==8.4.1
+fastapi==0.138.0
+frozenlist==1.8.0
+grpclib==0.4.9
+h11==0.16.0
+h2==4.3.0
+hpack==4.1.0
+httptools==0.8.0
+hyperframe==6.1.0
+idna==3.18
+markdown-it-py==4.2.0
+mdurl==0.1.2
+modal==1.5.0
+multidict==6.7.1
+propcache==0.5.2
+protobuf==6.33.6
+pydantic==2.13.4
+pydantic_core==2.46.4
+Pygments==2.20.0
+python-dotenv==1.2.2
+PyYAML==6.0.3
+rich==15.0.0
+starlette==1.3.1
+synchronicity==0.12.5
+toml==0.10.2
+types-certifi==2021.10.8.3
+types-toml==0.10.8.20260518
+typing-inspection==0.4.2
+typing_extensions==4.15.0
+uvicorn==0.49.0
+uvloop==0.22.1
+watchfiles==1.2.0
+websockets==16.0
+yarl==1.24.2