Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,22 @@ EMBEDDING_DIM=1024
# EMBEDDING_MAX_RETRIES=12 # sized for a cold CPU sidecar, not for an API
# EMBEDDING_TIMEOUT_SECONDS=300 # ditto — their product is the worst-case stall,
# # and the worker consumes one job at a time
#
# Or reach the SAME model through Jina's hosted API instead of running it.
# EMBEDDING_PROVIDER selects HOW the model is reached, never WHICH model runs:
# keeping jina-embeddings-v2-base-code on both paths is what keeps the displayed
# evaluation figures a description of the running system (Deploy.md §2).
# Note the bare model id — the API does not take the Hugging Face `jinaai/` path.
# Switch as a SET; leaving the sidecar retry budget in place points an hour of
# patience at a service that fails in seconds.
# EMBEDDING_PROVIDER=jina_api
# EMBEDDING_API_KEY=jina_...
# EMBEDDING_ENDPOINT=https://api.jina.ai/v1
# EMBEDDING_MODEL=jina-embeddings-v2-base-code
# EMBEDDING_DIM=768
# EMBEDDING_BATCH_SIZE=32
# EMBEDDING_MAX_RETRIES=3
# EMBEDDING_TIMEOUT_SECONDS=30
# On MacBook: run `make embedding-host` in a separate terminal (recommended).
# Or Docker sidecar: docker compose --profile embedding up -d embedding
# If you change EMBEDDING_DIM after first migrate: make down-all && make migrate
Expand All @@ -70,6 +86,16 @@ RERANKER_MODEL=stub
# RERANKER_ENDPOINT=http://host.docker.internal:8003
# RERANKER_CANDIDATE_LIMIT=16 # kept small so BGE reranks within the interactive budget on CPU
# RERANKER_MAX_SEQ_LENGTH=512
#
# Or reach the SAME model through SiliconFlow's hosted API.
# Their two platforms are independent: api.siliconflow.cn serves
# BAAI/bge-reranker-v2-m3, api.siliconflow.com served only Qwen rerankers when
# this was written, and keys are NOT interchangeable — a .com key on .cn returns
# 401. The model this project needs is the one the evaluation used, so .cn it is.
# RERANKER_PROVIDER=siliconflow
# RERANKER_API_KEY=sk-...
# RERANKER_ENDPOINT=https://api.siliconflow.cn/v1
# RERANKER_MODEL=BAAI/bge-reranker-v2-m3
# On MacBook: run `make reranker-host` in a separate terminal (recommended).
# Or Docker sidecar: docker compose --profile reranker up -d reranker

Expand Down
26 changes: 26 additions & 0 deletions .env.production.example
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,22 @@ EMBEDDING_MODEL=stub
EMBEDDING_DIM=1024
EMBEDDING_ENDPOINT=

# EMBEDDING_PROVIDER selects HOW the model is reached, never WHICH model runs.
# 'sidecar' expects a self-hosted service; 'jina_api' calls Jina AI's hosted
# endpoint. Both serve jina-embeddings-v2-base-code, which is the point: the
# evaluation figures this project displays describe that model, so the deployed
# system has to be running it (Deploy.md §2). Switch as a SET — the model id is
# the bare name on the API, and the sidecar retry budget is an hour of patience
# pointed at a service that fails in seconds.
# EMBEDDING_PROVIDER=jina_api
# EMBEDDING_API_KEY=jina_...
# EMBEDDING_ENDPOINT=https://api.jina.ai/v1
# EMBEDDING_MODEL=jina-embeddings-v2-base-code
# EMBEDDING_DIM=768
# EMBEDDING_BATCH_SIZE=32
# EMBEDDING_MAX_RETRIES=3
# EMBEDDING_TIMEOUT_SECONDS=30

# Sized for a cold self-hosted sidecar. 12 retries against a 300s timeout can
# hold one batch of four chunks for about an hour, and the worker consumes one
# job at a time, so everything queued behind it waits. Lower both substantially
Expand All @@ -77,6 +93,16 @@ RERANKER_ENDPOINT=
# RERANKER_CANDIDATE_LIMIT=16
# RERANKER_MAX_RETRIES=3

# Or the same model through SiliconFlow's hosted API. Their two platforms are
# independent and keys are NOT interchangeable: api.siliconflow.cn serves
# BAAI/bge-reranker-v2-m3, while api.siliconflow.com served only Qwen rerankers
# when this was written. A .com key against .cn returns 401 with "Api key is
# invalid", which looks like a bad key rather than a wrong region.
# RERANKER_PROVIDER=siliconflow
# RERANKER_API_KEY=sk-...
# RERANKER_ENDPOINT=https://api.siliconflow.cn/v1
# RERANKER_MODEL=BAAI/bge-reranker-v2-m3

# Hybrid RRF fusion weights. Dense above sparse reduces keyword noise.
# RRF_DENSE_WEIGHT=2.0
# RRF_SPARSE_WEIGHT=1.0
Expand Down
119 changes: 95 additions & 24 deletions Deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,11 @@ Outstanding Work list in the same PR, not tracked in two places.
is written from an approved plan, and plans outrun implementations. Verify before
relying on a claim here, and correct it when it is wrong.

**Status: PR 1 of 5 landed; PR 2–5 not started.** The pre-deployment state is
frozen at tag `v1.0-submission` (`a4612b8`). Nothing is deployed anywhere yet,
and no live model API has been called from this codebase.
**Status: PR 1 and PR 2 landed. PR 3–5 not started.** The pre-deployment state
is frozen at tag `v1.0-submission` (`a4612b8`). Nothing is deployed anywhere
yet. Both hosted model APIs have been exercised end to end from a running local
stack — see § 2.1 and § 6 PR 2 for the observed values — but **no index has been
built through them**, so the indexing side of the hosted path is still unproven.

---

Expand Down Expand Up @@ -98,17 +100,43 @@ regenerated. It does mean that **if the H1 suite is ever re-run through the API
providers, that is a new run directory with its own `provenance.json`, not a
refresh of the recorded snapshot.**

### 2.1 Not yet verified
### 2.1 Verified against the live APIs, 2026-08-02

Two facts this plan depends on and that must be confirmed with a live call
**before** the database volume is migrated, because `chunks.embedding` fixes its
dimension at migration time:
Both facts this plan depended on were observed rather than assumed, before any
client code was written.

- **Jina's API returns 768-dimensional vectors for `jina-embeddings-v2-base-code`.**
The model card and the recorded run both say 768, but the API response was not
observed in this session. One request settles it.
- **SiliconFlow's rerank request/response field names.** The client in § 6.2 must
be written against the observed payload, not against an assumed one.
- **Jina returns 768-dimensional vectors for `jina-embeddings-v2-base-code`.**
Confirmed: `HTTP 200`, `dim=768`, `768/768` components non-zero. The dimension
matches the recorded run, so no volume re-migration is implied.
- **SiliconFlow's rerank payload.** `{"model", "query", "documents",
"return_documents"}` → `{"id", "meta", "results": [{"index", "document",
"relevance_score"}]}`.

**And one thing neither the plan nor the documentation predicted: the response
does not come back in input order.** A three-document probe returned indices
`[1, 0, 2]`. Jina's embedding response likewise identifies each vector by
`index` rather than promising positional order. Both clients therefore place
results by index, and that is not defensive coding — zipping arrival order
against the input would attach the wrong embedding to a chunk, or score every
passage with another passage's relevance. Either produces no error, no log line,
and worse retrieval. This is what D-5 anticipated: neither API is
OpenAI-compatible in the part that actually matters.

### 2.2 SiliconFlow runs two platforms and they are not interchangeable

Found the hard way, recorded so nobody else spends the time.

| | `api.siliconflow.cn` | `api.siliconflow.com` |
|---|---|---|
| Rerankers offered | `BAAI/bge-reranker-v2-m3`, `Pro/BAAI/bge-reranker-v2-m3`, four Qwen3 variants | Qwen3 only — **no BGE** |
| A `.com` key against it | `401 "Api key is invalid"` | works |
| A `.cn` key against it | works | `401 "Token is invalid."` |

The failure is misleading in both directions: a wrong-region key reads as a bad
key, and a right-region key with the wrong model reads as `400 Model does not
exist`. **This project needs `.cn`,** because the model the evaluation used is
only there and § 2 does not permit substituting it. Using the `.com` platform
would have meant deploying a Qwen reranker under BGE's measured numbers.

---

Expand Down Expand Up @@ -296,7 +324,7 @@ rather than defaulted, because a wrong value there is a re-index, not a slow que
of the above, and **each was confirmed to fail when its defect is put back** —
a config test that has never been seen failing is not evidence of anything.

### PR 2 — `feat/managed-model-providers`
### PR 2 — `feat/managed-model-providers` — **done, with one gap named below**

The capability behind D-1.

Expand All @@ -320,17 +348,60 @@ The capability behind D-1.
to model the difference anyway.
- Timeouts and retries appropriate to a hosted API, not to a cold local model.

**Acceptance.**

1. Unit tests with a mocked transport cover: correct reordering, dimension
validation, auth header present, non-retryable 4xx surfacing, retryable 5xx
retrying. No live calls (§ 5.3).
2. **A live local run, reported with real observed values:** index `psf/requests`
with `EMBEDDING_PROVIDER=jina_api`, then confirm from the database that the
vectors are 768-dimensional and non-zero — this is the § 2.1 check and it must
happen before any Railway volume is migrated. Confirm the reranker is
materially reordering results rather than passing through.
3. `make check` green, and all four modes in § 5.1 still start.
**Acceptance — 1 met, 2 partially met, 3 met.**

1. **Met.** 21 tests over a mocked transport (`httpx.MockTransport`) cover
index-based reordering, batching with order preserved across batch
boundaries, dimension mismatch, short response, duplicated index,
out-of-range index, auth header, non-retryable 401 surfacing on the first
attempt, retryable 429 backing off, provider selection, missing-key errors,
unknown-provider errors, and stub short-circuiting before the provider is
consulted. No live calls (§ 5.3), and the backoff sleep is captured rather
than waited out.

2. **Partially met.** The production client classes were exercised against both
live APIs, and the observed values are recorded here:

| Check | Observed |
|---|---|
| `JinaApiEmbeddingClient`, 5 inputs over 3 batches | 5 vectors, all `dim=768`, all non-zero, all distinct |
| Cross-batch ordering | re-embedding input 2 alone lands `0.000000` from its batched slot; nearest other slot is `0.586` |
| `SiliconFlowRerankerClient`, 3 passages | scores returned in input order; argmax is the `HTTPBasicAuth` passage for a credentials question |

The **running stack** was then switched to both hosted providers
(`docker compose up -d --build api agent`, since the images are not
live-mounted) and queried against the existing 768-dimension index — the one
the *sidecar* wrote. That combination is the parity claim under test: a query
vector from Jina's API searching vectors produced by the local model.

| Check | Result |
|---|---|
| `mode=dense`, the zero-keyword-overlap query from Operations.md | **identical ranking**, same five chunks in the same order; cosine differs in the third decimal (0.5155 → 0.5182) |
| `mode=hybrid` | same five chunks; the top two swap, and they were **tied at 0.0042** in the baseline — the hosted run broke the tie 0.0043/0.0042. A tie broken differently is not a behaviour change |
| Reranker identity | hosted BGE returns scores in the same magnitude band as the local BGE sidecar (~0.004). The Qwen reranker on SiliconFlow's other platform returned ~0.64 on a comparable query, so the band is circumstantial evidence that the model is the one asked for |
| Full agent path, `POST /api/v1/query` | 3 tool calls (`search_code` → `get_call_neighbors` → `read_file`), 2 citations, **both verified**, groundedness `1.0` |
| Against the documented run | reproduces Operations.md *Verified Current Path — 2026-07-30* citation for citation: `sessions.py:186` and `sessions.py:557` |

The third-decimal drift is exactly the caveat § 2 records: same weights,
different serving stack, not bit-identical. It is visible and it does not
move the ranking.

**What is still not done: no index has been built through the hosted
provider.** The verification above is query-side. 726 chunks at batch 32 is
roughly 23 requests, which is where rate limiting and the embed stage's Redis
cache interaction would appear, and neither has been exercised. That is
deliberate — re-indexing would replace the verified reference index — and it
will happen naturally at the first Railway index. Until then this PR proves
the clients are correct and cross-compatible with the sidecar, not that a
bulk index built through them is.

The sidecar path was not re-run live after the switch. Its code is untouched
and its factory selection is covered by tests, but that is a weaker claim
than having watched it work, and it is stated as the weaker one.

3. **Met.** `make check` green: ruff, eslint, the drift check, mypy strict over
83 files, 310 passed / 5 skipped, 73 frontend tests. All four modes in § 5.1
still resolve, and `sidecar` remains the default everywhere.

### PR 3 — `feat/auth-gate`

Expand Down
2 changes: 2 additions & 0 deletions apps/agent/src/dcode_agent/graph.py
Original file line number Diff line number Diff line change
Expand Up @@ -540,6 +540,8 @@ def _get_reranker_client() -> RerankerClient | None:
model=agent_settings.reranker_model,
endpoint=agent_settings.reranker_endpoint,
max_retries=agent_settings.reranker_max_retries,
provider=agent_settings.reranker_provider,
api_key=agent_settings.reranker_api_key,
)
_reranker_client_initialized = True
return _reranker_client
Expand Down
4 changes: 4 additions & 0 deletions apps/api/src/dcode_api/routes/internal.py
Original file line number Diff line number Diff line change
Expand Up @@ -371,6 +371,8 @@ def _get_query_embedding_client() -> EmbeddingClient:
batch_size=api_settings.embedding_batch_size,
max_retries=api_settings.embedding_max_retries,
timeout_seconds=api_settings.embedding_timeout_seconds,
provider=api_settings.embedding_provider,
api_key=api_settings.embedding_api_key,
)
return _query_embedding_client

Expand All @@ -396,6 +398,8 @@ def _get_query_reranker_client() -> RerankerClient | None:
model=api_settings.reranker_model,
endpoint=api_settings.reranker_endpoint,
max_retries=api_settings.reranker_max_retries,
provider=api_settings.reranker_provider,
api_key=api_settings.reranker_api_key,
)
_query_reranker_client_initialized = True
return _query_reranker_client
Expand Down
2 changes: 2 additions & 0 deletions apps/worker/src/dcode_worker/stages/embed.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ async def run(
batch_size=worker_settings.embedding_batch_size,
max_retries=worker_settings.embedding_max_retries,
timeout_seconds=worker_settings.embedding_timeout_seconds,
provider=worker_settings.embedding_provider,
api_key=worker_settings.embedding_api_key,
)

owns_redis = redis_client is None
Expand Down
8 changes: 8 additions & 0 deletions docker-compose.prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,12 +61,16 @@ services:
EMBEDDING_BATCH_SIZE: ${EMBEDDING_BATCH_SIZE:-4}
EMBEDDING_MAX_RETRIES: ${EMBEDDING_MAX_RETRIES:-12}
EMBEDDING_TIMEOUT_SECONDS: ${EMBEDDING_TIMEOUT_SECONDS:-300}
EMBEDDING_PROVIDER: ${EMBEDDING_PROVIDER:-sidecar}
EMBEDDING_API_KEY: ${EMBEDDING_API_KEY:-}
RRF_DENSE_WEIGHT: ${RRF_DENSE_WEIGHT:-2.0}
RRF_SPARSE_WEIGHT: ${RRF_SPARSE_WEIGHT:-1.0}
RERANKER_MODEL: ${RERANKER_MODEL:-stub}
RERANKER_ENDPOINT: ${RERANKER_ENDPOINT:-}
RERANKER_CANDIDATE_LIMIT: ${RERANKER_CANDIDATE_LIMIT:-16}
RERANKER_MAX_RETRIES: ${RERANKER_MAX_RETRIES:-3}
RERANKER_PROVIDER: ${RERANKER_PROVIDER:-sidecar}
RERANKER_API_KEY: ${RERANKER_API_KEY:-}
LOG_LEVEL: ${LOG_LEVEL:-info}
depends_on:
postgres:
Expand Down Expand Up @@ -96,6 +100,8 @@ services:
EMBEDDING_BATCH_SIZE: ${EMBEDDING_BATCH_SIZE:-4}
EMBEDDING_MAX_RETRIES: ${EMBEDDING_MAX_RETRIES:-12}
EMBEDDING_TIMEOUT_SECONDS: ${EMBEDDING_TIMEOUT_SECONDS:-300}
EMBEDDING_PROVIDER: ${EMBEDDING_PROVIDER:-sidecar}
EMBEDDING_API_KEY: ${EMBEDDING_API_KEY:-}
WORKDIR_BASE: /tmp/dcode-workdirs
INTERNAL_API_KEY: ${INTERNAL_API_KEY:?set INTERNAL_API_KEY in .env.production}
LOG_LEVEL: ${LOG_LEVEL:-info}
Expand Down Expand Up @@ -135,6 +141,8 @@ services:
RERANKER_MODEL: ${RERANKER_MODEL:-stub}
RERANKER_ENDPOINT: ${RERANKER_ENDPOINT:-}
RERANKER_MAX_RETRIES: ${RERANKER_MAX_RETRIES:-3}
RERANKER_PROVIDER: ${RERANKER_PROVIDER:-sidecar}
RERANKER_API_KEY: ${RERANKER_API_KEY:-}
# Not read directly by the agent, but dcode_shared.db.models binds the
# chunks.embedding column type to it at import time.
EMBEDDING_DIM: ${EMBEDDING_DIM:?set EMBEDDING_DIM in .env.production — it must equal the dimension the volume was migrated at}
Expand Down
14 changes: 14 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,12 +62,19 @@ services:
EMBEDDING_BATCH_SIZE: ${EMBEDDING_BATCH_SIZE:-4}
EMBEDDING_MAX_RETRIES: ${EMBEDDING_MAX_RETRIES:-12}
EMBEDDING_TIMEOUT_SECONDS: ${EMBEDDING_TIMEOUT_SECONDS:-300}
# 'sidecar' (default) or 'jina_api'. Selects HOW the model is reached,
# never WHICH model runs — both serve jina-embeddings-v2-base-code.
EMBEDDING_PROVIDER: ${EMBEDDING_PROVIDER:-sidecar}
EMBEDDING_API_KEY: ${EMBEDDING_API_KEY:-}
RRF_DENSE_WEIGHT: ${RRF_DENSE_WEIGHT:-2.0}
RRF_SPARSE_WEIGHT: ${RRF_SPARSE_WEIGHT:-1.0}
RERANKER_MODEL: ${RERANKER_MODEL:-stub}
RERANKER_ENDPOINT: ${RERANKER_ENDPOINT:-http://host.docker.internal:8003}
RERANKER_CANDIDATE_LIMIT: ${RERANKER_CANDIDATE_LIMIT:-16}
RERANKER_MAX_RETRIES: ${RERANKER_MAX_RETRIES:-3}
# 'sidecar' (default) or 'siliconflow' — both serve bge-reranker-v2-m3.
RERANKER_PROVIDER: ${RERANKER_PROVIDER:-sidecar}
RERANKER_API_KEY: ${RERANKER_API_KEY:-}
LOG_LEVEL: ${LOG_LEVEL:-info}
ports:
- "8000:8000"
Expand Down Expand Up @@ -99,6 +106,10 @@ services:
EMBEDDING_BATCH_SIZE: ${EMBEDDING_BATCH_SIZE:-4}
EMBEDDING_MAX_RETRIES: ${EMBEDDING_MAX_RETRIES:-12}
EMBEDDING_TIMEOUT_SECONDS: ${EMBEDDING_TIMEOUT_SECONDS:-300}
# 'sidecar' (default) or 'jina_api'. Selects HOW the model is reached,
# never WHICH model runs — both serve jina-embeddings-v2-base-code.
EMBEDDING_PROVIDER: ${EMBEDDING_PROVIDER:-sidecar}
EMBEDDING_API_KEY: ${EMBEDDING_API_KEY:-}
WORKDIR_BASE: /tmp/dcode-workdirs
INTERNAL_API_KEY: ${INTERNAL_API_KEY:?set INTERNAL_API_KEY in .env}
LOG_LEVEL: ${LOG_LEVEL:-info}
Expand Down Expand Up @@ -178,6 +189,9 @@ services:
EMBEDDING_DIM: ${EMBEDDING_DIM:-1024}
RERANKER_MODEL: ${RERANKER_MODEL:-stub}
RERANKER_ENDPOINT: ${RERANKER_ENDPOINT:-http://host.docker.internal:8003}
RERANKER_MAX_RETRIES: ${RERANKER_MAX_RETRIES:-3}
RERANKER_PROVIDER: ${RERANKER_PROVIDER:-sidecar}
RERANKER_API_KEY: ${RERANKER_API_KEY:-}
RETRIEVAL_BASE_URL: http://api:8000
WORKDIR_BASE: /tmp/dcode-workdirs
INTERNAL_API_KEY: ${INTERNAL_API_KEY:?set INTERNAL_API_KEY in .env}
Expand Down
Loading
Loading