Repository navigation
Conversation
The console is a separate repository (Vite/React, static), so this repository documents the contract instead: token auth with per-person scoped tokens, the CORS setting for a separate hostname versus routing both behind one, and the warning that /admin must not be exposed without Access, a VPN or an allowlist.
SSO (Google, GitHub, enterprise OIDC), 2FA and passkeys belong to an identity provider, not to an LLM gateway. GATEWAY_PROXY_AUTH=true makes /admin/** trust the identity header an authenticating proxy injects (Authentik, Keycloak, oauth2-proxy, Cloudflare Access), so those features arrive with the IdP that already implements them properly. - fails closed: no header means 401 proxy_identity_missing, even if the admin token is sent - GATEWAY_PROXY_AUTH_SCOPES=read gives a viewer group that cannot write - audit entries are attributed to the proxy identity, not "shared-admin"; resolved per request from the principal - /admin/settings reports auth_mode so the console can skip the token form - a WARN at startup repeats that the gateway must not be reachable except through the proxy - docs/SSO.md: Authentik + Traefik forward-auth, Keycloak/oauth2-proxy, Cloudflare Access, and the spoofing caveat for each Tests: 170 (6 new) covering closed-by-default, token-not-enough, identity access, audit attribution and read-only scopes.
The insertion anchor did not match, so the WARN that tells an operator the gateway now
trusts a header was compiled out. Assert the anchor, and verify the line appears in a real
startup log rather than assuming the edit landed.
WARN DataSeeder - Admin auth is DELEGATED to a reverse proxy (X-Forwarded-User):
the gateway trusts that header for /admin/**. It must not be reachable except
through the proxy.
Verified with the gateway running in proxy mode: no header -> 401 proxy_identity_missing,
token alone -> 401, identity header -> 200 and auth_mode=proxy, and a provider created
through the proxy is audited to anindra@example.com.
provider.create and provider.update pass the whole input record to audit.record(), which serialised it verbatim — so the plaintext provider key landed in admin_audit.details, a table readable through GET /admin/audit. That defeats the AES-GCM encryption at rest the key is stored with. Sanitised centrally in AuditService: any field whose name matches api_key, apiKey, password, secret, token, authorization, new_key or private becomes "***", at any depth. Doing it in the service rather than at the call sites means a future admin endpoint cannot forget. Verified against a running gateway: a provider created with sk-live-SHOULD-NEVER-APPEAR audits as "api_key":"***" while the credential itself still works (hint ****PEAR). Anyone who created providers before this commit has keys in their audit table: rotate those provider keys. Tests: 172 (2 new).
Alerts were webhook-only, which is fine for a Slack channel and useless for the operator who is not looking at it. - GATEWAY_ALERT_EMAIL_TO (+ SPRING_MAIL_*) adds an email channel next to the webhook. They are independent: either, both or neither. Fire-and-forget on a virtual thread; a failing mailbox logs a warning and never touches the request. - one message per alert, subject "[Path] <event>", body as "key: value" lines - budget alerts keep their deduplication, so a busy project still sends one email a period - POST /admin/alerts/test sends a test alert and reports which channels are configured, so the config is proven before you depend on it - ObjectProvider<JavaMailSender>: the gateway still boots with email alerting off Verified against a fake SMTP server: subject "[Path] alert.test", To ops@example.com, body carrying the payload. Tests: 177 (5 new).
Alert mail was environment-only, which means a restart to change a recipient. - GET/PUT /admin/mail-settings: host, port, username, password, from, recipients, enabled. Stored in a single row, winning over SPRING_MAIL_*, which keeps working as the config-file route; the view reports which source is in force. - the SMTP password is AES-GCM encrypted with the master key, write-only in the API (has_password, never the value); blank keeps it, "-" clears it - enabling requires a host and at least one recipient, checked before saving - /admin/alerts/test sends through whatever is configured, console or environment - audit records the change without the password Found while clicking through it: PUT was missing from the CORS methods, so the console's save failed in the browser with "Failed to fetch" while curl worked. The error message the console produced named CORS as a possible cause, which is how it was diagnosed. Tests: 183 (11 new). Verified end to end against a fake SMTP server: save in the UI, test email delivered, subject "[Path] alert.test".
- Dockerfile gains curl in the runtime stage and a HEALTHCHECK on /actuator/health, so Dokploy and Docker can act on readiness instead of only "container is running" - deploy/dokploy/compose.yml: gateway + Postgres (+ optional Valkey), health-gated depends_on, stop_grace_period above the drain timeout, volumes for the database and the master key - deploy/dokploy/README.md: both apps end to end, the environment variables that matter, same-hostname routing that removes CORS, and the pre-public checklist (Access, API docs, master key, drain) The console keeps its own Dockerfile and /healthz (path-admin) and is deployed as a second Dokploy application.
'required:' on env_file needs compose >= 2.24, and this should not fail on an older Dokploy compose. Provider keys go in as environment variables or through the console.
…pose file Dokploy created no router for the compose service: the gateway container came up with zero traefik.* labels, so path.hacksubset.com returned 404 while the app was healthy. Declaring the labels here makes routing part of the deployment instead of a UI step that has to be repeated after every domain change. TLS uses the server default certificate, which is the Cloudflare Origin certificate when Cloudflare proxies the hostname.
SPRING_MAIL_HOST is passed through as an empty string by the Dokploy compose, which still counts as 'set' and activates Spring's mail health indicator. It then probes an empty host, gets Connection refused, and /actuator/health returns DOWN — so Docker marks the gateway unhealthy and the stack restarts it for a channel that is optional. Mail is configured from the console (settings/alert-email) via JavaMailSender built at runtime, so the stock indicator never reflects it anyway.
DataSeeder ran ensureProviders() unconditionally on every boot, so a fresh deployment showed openai, anthropic, groq and gemini in the console even though nobody had configured them, plus a ten-row route table. That reads as demo data in what is supposed to be an empty, real console. gateway.seed-defaults (GATEWAY_SEED_DEFAULTS, default true so tests and the local quickstart are unchanged) turns it off. Deleting the rows is not enough on its own: the seeder recreates them whenever the provider table is empty.
1. Admin auth limiter bypass (CWE-307). The per-address counter trusted the first X-Forwarded-For hop, which a client supplies, so rotating it (or arriving from a rotating Cloudflare edge) reset the counter on every request and thousands of token guesses were evaluated against a nominal cap of 10. X-Forwarded-For is now ignored unless gateway.client-ip-header names a trusted edge header (CF-Connecting-IP), and a process-wide cap (gateway.admin-auth.global-max-failures, default 200 per window) refuses all admin authentication once hit, whatever address it claims. 2. Unauthenticated Actuator. /actuator, /metrics, /prometheus and /health/providers answered anonymously, exposing the framework version, datasource internals, repository names and provider topology. Only GET /actuator/health is public now, and it reports status alone; everything else under /actuator needs the admin token. 3. Unauthenticated /v1/models. The catalogue names every model and alias, so gateway.public-models now defaults to false. Opting back in still works (PublicModelsTest). Tests: 188 pass, including the rotating-address case that reproduces the bypass.
…in bypass Follow-up to the pentest fixes: - The global failure cap was fail-closed and therefore a denial-of-service against the operator: one scanner spending 200 guesses locked the console for everyone in the window (observed live). It now defaults to 0 (off); the per-address cap is the control that matters once the client address can be trusted. - That trust required more than config: the origin answers on its own public IP, so Cloudflare -- and its bot challenge -- can be skipped entirely, and CF-Connecting-IP forged. The gateway router now carries the cf-only middleware (Cloudflare's published ranges), declared here so a redeploy cannot silently drop it. Per-address limiting now works: one source gets 10 attempts per window, and the operator is not collateral damage.
Direct-to-origin access is what made the pentest findings exploitable and makes client-IP headers forgeable. The compose label is in the repo; the middleware and the console's router are not, so this documents both the setup and the fact that a Dokploy domain change silently drops cf-only from the console router.
…er cache API responses Pentest findings #132 and #124. #132 -- the data plane answered 'missing_api_key', 'invalid_api_key' and 'key_disabled' for the three kinds of rejection, so a client could test whether a key existed, and each guess cost one database lookup with no budget: 193k guesses were measured at ~474/s with zero 429s, heavy enough to induce a proxy outage. All rejections now return one identical 'error' object, and ClientAuthLimiter caps rejected keys per address per minute before the lookup happens. The global cap is off by default for the same reason the admin one is: a fail-closed global cap is a denial-of-service against legitimate clients. #124 -- a CDN caches by URL, so a static extension on an authenticated URL is stored and served to anonymous clients for hours. NoStoreFilter now marks /v1, /admin and /actuator responses no-store before the chain runs, so error paths are covered too and the edge has nothing to keep. FailureWindow is shared by both auth gates; AdminAuthLimiter keeps its public shape. Tests: 192 pass (oracle, limiter, and no-store now covered).
A missing Authorization header skipped the hashed lookup while an unknown key performed one, so the bodies were identical but the cost was not: 'no header' was consistently faster. The lookup now runs for every rejection, and a test asserts the two responses are byte-for-byte equal. Found while reading the auth path against the pentest's timing-side-channel track, not from a report.
The wrapper is distributionType=only-script, so there is no maven-wrapper.jar for the IDE's Maven server to use. It resolves its own home, bootstraps Maven without the plexus artifact bindings, and dies with 'Guice/MissingImplementation: ArtifactHandlerManager'. Recording the symptom and the fix (point Maven home at the extracted 3.10.1) because it costs an hour to rediscover.
A single fire-and-forget POST lost an alert on a 500, a 429 or a two-second
network blip, and the operator believed they were monitored.
- three attempts with jittered exponential backoff (250ms, 1s, 4s)
- retry only 429, 5xx and connect/timeout; never a 4xx
- honour Retry-After (seconds or HTTP date), capped at 30s
- two workers and a 64-entry queue; a full queue drops and counts instead
of blocking the caller or growing without bound
- path_alerts_sent_total{outcome=delivered|retried|dropped}
Closes #87
A read-only token could read everything but could not write, and the console had no way to know: it showed New key, Delete and Revoke, and the operator found out through a 403 after the click. GET /admin/settings now returns subject (token name or SSO identity) and the caller's scopes with can_write, for both token and proxy auth. Scopes and a name are not credentials. Closes #85
The console built its requests-over-time chart from the last 1000 usage rows,
so above a thousand calls in the window the chart was a sample, and every
client reimplemented the same bucketing.
GET /admin/stats?window=24h&bucket=1h now returns a series of {from, requests,
cost_usd, failures, p95_latency_ms}. Buckets are 1m, 5m, 1h or 1d; an invalid
value is a 400 naming the accepted ones. Empty buckets are zero-filled, the
window is clamped to 500 points, and the whole series comes from one query.
Omitting bucket leaves the response unchanged.
Closes #84
The only provider alert was provider.circuit_open, which needs consecutive failures. A provider failing 30% of calls with a success in between never tripped it, so requests were silently retried onto a fallback while cost and latency rose. A one-minute check now reads the routing EWMA, aggregates it per provider by sample count, and fires provider.error_rate_high above GATEWAY_ALERT_ERROR_RATE (0.25) with at least GATEWAY_ALERT_MIN_SAMPLES (20) recent calls. One alert per provider per hour; provider.error_rate_normal closes it once the rate falls under half the threshold. Closes #83
The metrics existed but no recording or alerting rules did, so every operator had to derive the PromQL, and most did not. - deploy/prometheus/path-rules.yml: fallback >20%, cache hit ratio under half its 7-day average, spend on track for 2x yesterday, error rate >2%, provider parked >15m; each with a for-duration and a summary/runbook annotation - Helm prometheusRule.enabled renders the same rules as a PrometheusRule - CI runs promtool check rules and renders the toggle - new path_fallbacks_total counter and path_provider_circuit_open gauge supply the two rules that had no metric - docs/OPERATIONS.md explains each alert and what to do Closes #86
AlertService now has a second constructor for the bounded-queue test, so the container needs to be told which one to use.
The EWMA is what makes the rate recent; the sample count only proves enough traffic has been seen to trust it. Say so instead of implying a sliding window that does not exist.
path_requests_total only knows success/error, so the 5xx alert shipped in #86 was really an overall error-rate rule: a route returning 400s and a provider falling over with 500s looked identical. recordError now takes the status that produced the failure and increments path_errors_total{provider,class=4xx|5xx|other,code}. The alert rule divides the 5xx series by all requests, so a client-side 400 can no longer page. Closes #88
Email started one virtual thread per alert, so a hung SMTP server grew one thread per alert with no ceiling and no way to see that mail was lost. Email now shares the webhook's shape: two workers, a 64-entry queue, daemon threads, and a drop that is counted. path_alerts_sent_total gains a channel tag (webhook|email); email reports delivered/dropped only, because retrying an SMTP send can duplicate a message the server already accepted. Closes #89
/admin/stats read cache.semantic_hits from a LongAdder in the JVM: it reset on
every restart, differed per replica and never reached Prometheus. Every hit
was already a usage row, but nothing recorded which tier served it.
- usage_records.cache_tier (V19): exact or semantic, null otherwise
- recordCacheHit sets it and tags path_cache_hits_total{model,tier}
- /admin/stats and per-project stats count semantic hits from the window
- the in-process counter is gone; the row is the source of truth
Verified against the smoke upstream: semantic_hits=1, and
path_cache_hits_total{tier="semantic"}=1 for the same window.
Closes #90
Threshold alerts only fire when something is already wrong, and nobody opens a dashboard every morning, so a slow spend drift is noticed on the invoice. One email a day (GATEWAY_DIGEST_CRON, default 08:00 gateway time) summarises yesterday: requests with the day-over-day delta, spend, tokens with the cache hit ratio and savings, failures with the top error code, the slowest model's p95, the top three models by spend, and any provider parked at send time. - silently off without the alert email channel; nothing is sent for a day with no traffic, and a mailbox failure only logs a warning - GATEWAY_DIGEST=false disables it - POST /admin/digest/test?date=... sends it on demand and returns the text, so the schedule can be checked before the first morning - rendered text is pinned by a unit test for the documented numbers Closes #82
The workflow pushed the version tag and latest before Trivy ran, so a failed scan still published the version: v0.2.0 and v0.2.1 each burned a number that way. - build and push only ghcr.io/<repo>:sha-<commit> - Trivy gates that tag - on success, docker buildx imagetools create copies the same digest to <tag> and latest; nothing is rebuilt - the recorded digest is the one that was scanned - sha- is kept for traceability Closes #80
usage_records grows forever, and the nightly retention delete competes with the indexes that /admin/stats scans. - deploy/postgres/partitions.sql converts usage_records to monthly range partitions (same columns, identity and indexes, plus a DEFAULT partition) and ships path_ensure_usage_partitions for a monthly cron; documented as a maintenance-window step, not a Flyway migration, because it rewrites the table - /admin/usage/export accepts from/to, so the exact slice a purge is about to delete can be archived for finance; window still works when they are absent - docs/OPERATIONS.md covers retention, the archive loop, the dump-first procedure and detaching an archived month Retention itself already existed (GATEWAY_USAGE_RETENTION, nightly purge) and is unchanged. Closes #61
With three replicas one parked a dead provider while the others kept sending it traffic, and per-key concurrency caps multiplied by the replica count. - CircuitBreakerStore: memory (default) or Valkey, one hash per provider with state/open_until/failures and a TTL, so an OPEN entry expires even if the replica that opened it dies - OPEN -> HALF_OPEN is a Lua compare-and-set: exactly one replica claims the probe, and the claim itself expires after GATEWAY_BREAKER_PROBE_LEASE - ConcurrencyStore: semaphores (default) or a Valkey sorted set of leased permits; releasing pops one member, a killed pod's permits age out - ChatService/StreamService count availability with isParked() so the probe claim is not consumed by the pre-flight count - GATEWAY_SHARED_STATE_BACKEND=valkey opts in; the chart sets it when its Valkey is enabled - docs/LIMITS.md no longer lists the gap Tests: two breaker instances over one store park together, exactly one claims the probe, success closes and failure reopens for both; two limiters over one store enforce a single cap. The Valkey scripts need a real server and are not covered by the H2 suite; the memory path is unchanged. Closes #74
A Java team using Spring AI could not point its ChatModel at the gateway without hand-writing an HTTP client, which is why the Maven Central artifact was hard to justify: there was no library-shaped thing to add. - the build becomes a reactor: path-parent (pom), path (the gateway, still io.github.an1ndra:path) and path-spring-ai - PathChatModel implements ChatModel: messages, options, tool definitions and tool responses, non-streaming tool calls and SSE content streaming - PathEmbeddingModel implements EmbeddingModel over /v1/embeddings - X-Path-Cache, X-Gateway-Provider and X-Gateway-Cost-Usd land in the ChatResponse metadata; errors surface as PathException with status and code - idempotency(true) sends Idempotency-Key so a caller retry is not charged twice - the gateway never depends on Spring AI; only the adapter does - Dockerfile builds -pl path-gateway only, so the image stays the same size - CI builds and tests both modules; release publishes all three artifacts - docs/SPRING-AI.md and a README entry Tests run against a stub HTTP server: request shape, tool round-trip, streaming assembly, PathException, embeddings. Closes #62
The compatible Gemini endpoint covers chat and streaming but silently drops thinkingConfig, safetySettings and cached-content pricing, which is the opposite of the 422 contract the Anthropic adapter keeps. - GEMINI and VERTEX provider types with a native generateContent adapter - thinkingConfig (or reasoning_effort as a shortcut) and safetySettings are honoured; a parameter Gemini cannot honour is 422 unsupported_parameter, never dropped - thinking tokens are counted in completion (billable output) and broken out as reasoning_tokens, with cachedContentTokenCount priced as cache reads - a blocked prompt maps to 400 content_blocked naming the category - Vertex: service-account JSON as the encrypted credential, RS256 assertion exchanged for a cached OAuth token, project/location columns (V20) - tool calls and tool responses round-trip; images are rejected, not ignored - model sync and the probe handle the native types instead of pretending they are OpenAI-shaped Tests: request translation, 422, response normalisation, the exact cost arithmetic (thinking priced as output), blocked category, streaming, and a Vertex OAuth exchange against a stub Google that verifies the RS256 signature and asserts the token is cached. Closes #75
GitHub now forces Node 20 actions onto Node 24 and warns on every Helm job: 'azure/setup-helm@1a275c3 ... is being forced to run on Node.js 24'. v5.1.0 is the same action on node24. The version input is unchanged, so Helm stays pinned at v3.16.3. Every other action in the workflows already targets node24 or is composite.
Bumps the reactor to 0.2.3 and adds the release notes: the Spring AI adapter, native Gemini (AI Studio and Vertex), the daily digest, bucketed stats, archive slices, Prometheus rules, shared Valkey state, the alerting changes and the security fixes since v0.2.2. unreleased.md is reset to the empty template.
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.OpenSSF ScorecardScorecard details
Scanned Files
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release v0.2.3.
Notes in
changelog/v0.2.3.md. Highlights since v0.2.2:path-spring-ai: a Spring AIChatModel/EmbeddingModelover the gateway, as a sibling module under a newpath-parentPOM/admin/stats;from/toon the usage exportpath_errors_total,path_fallbacks_total,path_provider_circuit_open,path_alerts_sent_total/admin/settings; email alerting and console mail settings; SSO proxy authMigrations V19 and V20 run on startup. The Postgres partitioning script is opt-in and is not run by the upgrade.
Verified locally:
./mvnw verify(251 tests, both modules), the full smoke suite (157/157),promtool check rules, andhelm templatewith every toggle.