Repository navigation
fix(deploy): give every service the retrieval settings it reads - #19
Merged
Merged
Conversation
Deploy.md is the authority for taking this online. It records the three decisions taken (real models via hosted APIs, a login gate between landing and workbench, Railway), the platform constraints that were verified rather than assumed, the invariants this workstream may not break, five PRs with binding acceptance criteria, and a findings register. Two things about the register are deliberate. Every row carries a file:line so it can be checked instead of believed, and each row is marked `direct` (confirmed in-session by reading the file or resolving the compose config) or `audit` (reported with a citation by a code-audit pass, not independently re-checked). Treating those two as the same kind of claim is how a plan acquires facts nobody verified. The register is not a second backlog. Final_Report.md keeps the single outstanding-work list for the product; Deploy.md covers only what blocks deployment, and where an item is both it gets struck from Outstanding Work in the PR that fixes it. The constraint that outranks the rest is model parity: every figure the UI and the docs display is generated from results/eval-h1-repeat3-2026-07-31/ and is a claim about that exact configuration. All three of its models are available as hosted APIs, so the deployment can match it without substituting anything — and if a future change substitutes one, it says so on the landing page and /methodology or it reverts. CLAUDE.md gets a pointer, not a copy: §2 carries only the seven invariants that stay true whichever PR is in flight, and says explicitly that the backlog lives in Deploy.md. Three copies of "what's left" is the drift pattern this project keeps having to fix. Sections renumbered 2-6 -> 3-7; the two internal §1 references still resolve.
D-5: explicit per-provider client classes rather than a generic OpenAI-compatible adapter. Neither Jina's embeddings response nor SiliconFlow's rerank response is OpenAI-shaped in the part that matters — both carry per-item indices that have to be mapped back to input order — so a shared abstraction would have to model the difference anyway. The cost is recorded: a fourth provider later means writing a class, not setting a variable. D-6: one shared account read from the environment. No user table, no migration. The consequences are written down rather than discovered later: the daily query cap becomes per session rather than per person, and a shared password cannot be revoked for one holder, only rotated. That is adequate for a gated demo and is not an authorization model. Open questions drop from five to two. Team ownership stays unresolved and is the reason this branch is being pushed; the durable fix for the Railway shared-volume constraint stays deferred and recorded.
The production compose file configured almost none of the retrieval stack, and the failure mode is silence rather than an error. The api service had no embedding or reranker configuration at all. Two consequences. `make prod-migrate` runs alembic inside that container, and 001_initial_schema reads shared_settings.embedding_dim to size the pgvector column — so production built a vector(1024) column whatever .env.production said, while the worker inserted at its own configured dimension. And the api is the service that embeds the search query and reranks candidates, so it silently ran the stub path: an all-zero query vector against real stored vectors, which is dense retrieval returning noise with nothing in the log. The other two services were short as well. The worker had a model and a dimension but no EMBEDDING_ENDPOINT, so it could not reach a service. The agent had RERANKER_ENDPOINT alone, which selects nothing — create_reranker_ client returns None for a stub model regardless of the endpoint — so the evidence-union rerank the agent performs was identity ordering in production. EMBEDDING_DIM is now required in production rather than defaulted. It is not only the migration that depends on it: dcode_shared.db.models binds the chunks.embedding column type at import time, so every service touching the ORM needs it right. A wrong value is a re-index, not a slow query, and that is not a thing to inherit from a default that happens to suit stub vectors. Also here: the dead RERANKER_ENDPOINT=http://localhost:9999 default is replaced by an empty one, which fails loudly for a real model and is ignored by a stub; the two documented compose invocations are reconciled on the standalone form, because the overlay form the env example documented republishes Postgres, Redis and the RabbitMQ UI to the host. EMBEDDING_TIMEOUT_SECONDS becomes env-tunable instead of hardcoded at 300s. The value is unchanged. Twelve retries against a 300s timeout can hold one batch for about an hour while prefetch_count=1 stalls the queue behind it, but those defaults are correct for a cold CPU sidecar; the path they are wrong for is the hosted API, which does not exist until PR 2. PR 2 sets them. Three tests pin all of this, and each was confirmed to fail when its defect is put back. They parse the compose file by hand rather than importing PyYAML, which is only a transitive dependency here and is imported by no other code in the repository. Deploy.md records a correction: F-02 was written as "the api service has no embedding configuration" and was wider than that. The register and the acceptance criteria are updated from what was observed, not from what was planned.
This was referenced Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #18. Until that merges, this PR also shows its two documentation commits; afterwards the diff reduces to the single fix commit here.
Closes F-02, F-11, F-12 and half of F-13 in
Deploy.md§7.The defect
The production compose file configured almost none of the retrieval stack, and every failure mode is silent.
apihad no embedding or reranker configuration at all. Two consequences:make prod-migrateruns alembic inside that container, and001_initial_schemareadsshared_settings.embedding_dimto size the pgvector column. Production therefore built avector(1024)column whatever.env.productionsaid, while the worker inserted at its own configured dimension.apiis the service that embeds the search query and reranks candidates. With no configuration it ran the stub path: an all-zero query vector against real stored vectors. That is dense retrieval returning noise, with nothing in the log.The other two were short as well — this is wider than the finding as originally written, and
Deploy.mdrecords the correction:workerEMBEDDING_ENDPOINT,EMBEDDING_BATCH_SIZE,EMBEDDING_MAX_RETRIESagentRERANKER_MODEL,RERANKER_MAX_RETRIES,EMBEDDING_DIM, synthesis knobsRERANKER_ENDPOINTalone selects nothing —create_reranker_clientreturnsNonefor a stub model whatever the endpoint says — so the agent's evidence-union rerank was identity ordering in productionEMBEDDING_DIM is now required in production
Not defaulted. It is not only the migration that depends on it:
dcode_shared/db/models.py:128binds thechunks.embeddingcolumn type at import time, so every service touching the ORM needs it correct. A wrong value is a re-index, not a slow query, and that is not something to inherit from a default that happens to suit stub vectors.Also here
RERANKER_ENDPOINT=http://localhost:9999default is replaced by an empty one, which fails loudly for a real model and is ignored by a stub.EMBEDDING_TIMEOUT_SECONDSbecomes env-tunable instead of hardcoded. The value is unchanged. Twelve retries against a 300s timeout can hold one batch for about an hour whileprefetch_count=1stalls the queue behind it — but those defaults are correct for a cold CPU sidecar, and the path they are wrong for is the hosted API, which does not exist until PR 2. PR 2 sets them.Tests
Three new tests in
packages/shared/tests/test_config_hardening.py, and each was confirmed to fail when its defect is put back. A config test that has never been seen failing is not evidence of anything.They parse the compose file by hand rather than importing PyYAML, which is only a transitive dependency here and is imported by no other code in the repository.
Verified / not verified
Verified:
docker compose configresolves 12 retrieval variables onapi, 6 onworker, 9 onagent, mutually consistent; omittingEMBEDDING_DIMaborts with the reason attached;make checkgreen — ruff, eslint, the eval-artifact drift check, mypy strict over 83 files, 289 passed / 5 skipped, 73 frontend tests.Not verified: no container was started, the production stack was not run, and no live model API was called. This round is configuration resolution and unit tests only.