Repository navigation
fix: the audit follow-ups, and the three limits that remain - #23
Merged
Merged
Conversation
… key
Four of the audit follow-ups, grouped because they are all "an input this
service trusted more than it should".
Request bodies had no size limits anywhere. QueryRequest.query had a
min_length and no max, so a caller could make the gateway parse an
arbitrarily large body into memory before anything rejected it; history was
trimmed only *after* parsing. Bounds are ceilings on abuse, not on use — no
Git URL approaches 2 KB and the gateway already trims turns to 4,000
characters.
git clone received the repository URL with no `--`, so a value beginning
with "-" is parsed as an option and `--upload-pack=<cmd>` is command
execution. The gateway validates the URL, but the worker consumes a queue
message and should not depend on the only check being upstream of it. Also
sets GIT_TERMINAL_PROMPT=0: git otherwise waits for a username on a private
or mistyped repository until the 180s clone timeout, when "authentication
required" after two seconds is the better answer.
INTERNAL_API_KEY had a working default published in this repository, and
nothing but compose's `${VAR:?}` stopped a service running on it — which
covers compose and not a container started straight from the Dockerfile,
i.e. what a platform deploy does. Both the API and the agent now refuse to
start on the placeholder. The additional length rule applies only when the
deployment authenticates its users, which is a proxy for production rather
than a fact about it; that is stated in the code rather than implied.
The SSRF check only ever looked at literal IPs, so a hostname resolving to
cloud metadata or an internal subnet passed. Every resolved address is now
checked, one private answer among several public ones is enough to refuse,
and a resolution failure does *not* reject — a DNS hiccup is not evidence
about the host, and reporting it as an invalid URL would send someone
hunting for a typo. Credentials in the URL are refused rather than
stripped: a submitted URL is persisted and echoed back by the status route,
so a token handed to this endpoint became readable by anyone who could read
that repository's status. And there is now an optional host allowlist,
empty by default, which is the only rule in the whole check that states
what is *allowed* rather than what is obviously wrong.
What resolution does not close is rebinding — git resolves the name again
when it clones. That needs egress filtering at the network, and it is
recorded as unclosed rather than implied to be handled.
Resolution goes through a module-level seam so the tests do not perform
real DNS lookups; an autouse fixture stubs it, and the tests that care
about resolution replace it. Without that the suite would need the network
and would occasionally fail for reasons unrelated to the code.
The tool cache key was (tool, repo_id, args) and carried nothing about which generation of the corpus produced the answer. Re-indexing a repository therefore left every cached result addressable by the same key for the next 24 hours, and the agent answered from the previous corpus. Nothing failed, nothing logged, and the only symptom was answers that disagreed with the index. The worker already increments repos.index_revision in the same transaction that replaces the chunks, so folding it into the key makes a re-index a cache miss by construction rather than by remembering to flush Redis — which is currently a documented manual step before evaluation runs. `index_revision` on tool_cache_key is keyword-only with no default. A default is what would let a new call site quietly reintroduce this. Read once per query and memoised on the state, not per tool call, so one walk sees one corpus even if a re-index lands mid-answer. When it cannot be read — no session, no such repository, a query error — the request bypasses the cache entirely instead of falling back to a revision-less key, because that fallback would recreate the collision in exactly the case nobody exercises. Three tests cover it: same generation hits, a bumped generation misses, and an unreadable revision produces no cache entry at all. An existing caching test needed a session stub, which is itself the change being made visible.
The remaining three audit follow-ups, each with a limit stated rather than implied. grep had no ceiling of any kind. Its argument is a pattern the planner derives from a user question, and `.` over a large repository produced hundreds of MB of JSON buffered whole in the agent process, then serialised into Redis and the SSE state. There is now a timeout (the clone path had one and this did not), per-file and total match caps, and a file-size cap — bounded at ripgrep rather than after the fact, since reading everything and discarding most of it is the behaviour being replaced. The Python fallback is bounded on everything countable and is honest about the rest: Python cannot time out a single `re` call, so a pattern with catastrophic backtracking against one long line can still block the loop. The agent image installs ripgrep, so that path runs only where it is missing. The docstring says so instead of implying the fallback is safe. /healthz answered `ok` with Postgres, Redis and RabbitMQ all down, and the first user request discovered it. /readyz checks all three — all of them, not short-circuited, because an operator wants to know whether one thing is broken or everything is. It stays a 503 rather than a 200 with a body: a load balancer reads the status line. It also checks the embedding dimension against the live column, which is the one misconfiguration in this stack that produces no error at all (Deploy.md §5.4): the worker's inserts fail, or dense search runs a stub all-zero query vector against real stored vectors and returns noise. A missing table is skipped rather than failed, because before the first migration there is nothing to disagree with and the Railway runbook starts the API before running it. /healthz is left shallow on purpose — a liveness probe that fails on a dependency gets the process restarted, which fixes nothing and loses the logs. Cloned workdirs were never removed, for any repository, ever, so disk grew monotonically with the number of distinct repositories ever submitted. Eviction is a trade, not a cleanup: the agent reads those trees at query time, so pruning a `ready` repository's checkout costs it read_file, grep and list_directory until re-index while search and the call graph carry on from Postgres. So it is a count cap, off by default, evicting oldest-first before a clone and never the repository being cloned. An undeletable directory is logged and skipped rather than failing the index. And because that failure is now reachable, the agent's missing-checkout error says what happened: a bare FileNotFoundError sends the reader looking for a missing file rather than a missing checkout.
Eight findings closed or narrowed, with the register updated from what was observed. Three of them are marked "mostly" rather than fixed, and the reasons are written out rather than left in a commit message: DNS rebinding survives resolution checking, because git resolves the name again when it clones. That needs egress filtering at the network, which on Railway is a platform capability this project does not configure. The workdir cap is a trade. The agent reads those trees at query time, so evicting a ready repository's checkout costs it read_file, grep and list_directory until re-index while search and the call graph carry on from Postgres. Off by default for that reason. The grep Python fallback can still block on one pathological re call, because Python offers no way to interrupt one. Everything countable is capped, and the agent image installs ripgrep so the path runs only where it is absent. A closed finding that quietly leaves the hole open is worse than an open one, so all three stay visible in the register rather than being struck through. The runbook also now says to point Railway's health check at /readyz rather than /healthz, and to decide WORKDIR_MAX_REPOS and REPO_URL_ALLOWED_HOSTS deliberately instead of inheriting defaults chosen to preserve existing behaviour.
Three config-as-code files and the variable sheet that cannot be one. TOML rather than JSON because several of these settings need a reason attached and JSON has nowhere to put one. numReplicas = 1 on agent-worker is the clearest case: a second replica would not share the volume, so its agent would read a different or empty checkout, and it would also be a second RabbitMQ consumer competing for jobs in a pipeline that is not written to run concurrently on one repository. A bare `1` invites someone to raise it. The migration becomes a preDeployCommand on api rather than a manual step. That is where it belongs, not where it is convenient: it reads EMBEDDING_DIM to size the pgvector column and that variable lives on the api service, so running it anywhere else is how the column ends up at a dimension nothing agrees with. It also fixes the ordering the runbook previously had to describe in prose — EMBEDDING_DIM must now simply be right before the first deploy of api. api's health check is /readyz, not /healthz. The shallow one answers ok with every dependency down. Also unifies the frontend build context to the repository root. It was the only image built from a subdirectory, which is awkward on a platform whose config-file path does not follow the service's Root Directory — the alternative was one service with a Root Directory and a `..` in its dockerfilePath. Removing the apps/frontend exclusion from .dockerignore costs almost nothing: node_modules and dist stay excluded, which is where the 227 MB was. Verified by building: 1.47 MB of context, and the image behaves identically — gzip, cache headers, SPA fallback, and both envsubst shapes with nginx's own variables intact. Still nothing deployed. The files and the runbook are written from the code and say so.
Odiethebest
added a commit
that referenced
this pull request
Aug 3, 2026
CI on main has been red since #21 and I did not look. Every report of "make check green" in this series was true and insufficient: it passes locally because a developer's .env supplies a real INTERNAL_API_KEY and a Postgres is running. CI has neither, and CI is the hermetic environment that exists to catch exactly this. Two failures, both mine. #21 added a test that asserted a protected route stops returning 401 by hitting /repos/{id}/status, which connects to Postgres. The claim being tested is "the cookie got past require_session", which is about the dependency and not about the database. It now stubs the session and asserts 404 for an unknown repo — the request reached the handler, which is the actual proof. #23 added a startup guard refusing the placeholder INTERNAL_API_KEY, so on a machine with no .env every test that boots a lifespan died with "still the placeholder". conftest.py now pins it alongside AUTH_ENABLED and DOCS_ENABLED. That file was written for precisely this asymmetry, and pinning two settings and not the third left the hole open — the guard just made it fatal instead of silent. Verified in both environments rather than one: 394 passed with .env present, and 394 passed with .env moved aside and DATABASE_URL pointed at a dead port, which is the closest local reproduction of CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #18–#22. The last PR in the deployment series. Eight findings, 40 new tests, three commits grouped by what they are about rather than by finding number.
queryhad amin_lengthand no max; history was trimmed only after parsingindex_revision/readyz, including the embedding-dimension check--before the clone URL,GIT_TERMINAL_PROMPT=0The ones worth reading
F-07 was invisible. The tool cache key was
(tool, repo_id, args)and carried nothing about which generation of the corpus produced the answer, so re-indexing left every cached result addressable for 24 hours and the agent answered from the previous corpus. Nothing failed, nothing logged.index_revisionis now keyword-only with no default — a default is what would let a new call site quietly reintroduce it. Read once per query so one walk sees one corpus; when it cannot be read the request bypasses the cache rather than falling back to a revision-less key, because that fallback recreates the collision in exactly the case nobody exercises.F-06's real gap was names. The old check only ever looked at literal IPs, so
evil.example.compointing at169.254.169.254passed. Credentials in the URL are refused rather than stripped — a submitted URL is persisted and echoed back by the status route, so a token handed to this endpoint became readable by anyone who could read that repository's status. Quietly rewriting what someone submitted is the worse behaviour.Resolution goes through a module seam so the suite performs no real DNS lookups; without it every test that submits a URL would need the network.
F-10 also catches the one silent misconfiguration.
/readyzcompares the configuredEMBEDDING_DIMagainst the live column — the failure §5.4 describes, where the worker's inserts fail or dense search runs a stub all-zero query vector against real vectors and returns noise, with no error either way. A missingchunkstable is skipped rather than failed, because the Railway runbook starts the API before the migration./healthzis left shallow on purpose: a liveness probe that fails on a dependency gets the process restarted, which fixes nothing and loses the logs.Three limits this PR does not remove
Stated because a closed finding that quietly leaves the hole open is worse than an open one. All three stay visible in the register rather than struck through.
REPO_URL_ALLOWED_HOSTSis the available mitigation.readyrepository's checkout costs itread_file/grep/list_directoryuntil re-index, while search and the call graph carry on from Postgres. Off by default for that reason, and the tool's error now says which half stopped working instead of raising a bareFileNotFoundError. The durable answer isDeploy.md§11 item 2.recall, so catastrophic backtracking against one long line is unbounded. Everything countable is capped, and the agent image installs ripgrep so the path runs only where it is absent.Unchanged on purpose
Docker health checks still probe
/healthz./readyzis for a load balancer; wiring it intodepends_on: condition: service_healthywould let an unrelated dependency outage block local startup chains. The runbook says to point Railway's health check at/readyz.make checkgreen: 394 passed / 5 skipped, 86 frontend tests, mypy strict over 88 files.