Skip to content

Release v0.2.0 - #76

Merged
an1ndra merged 52 commits into
mainfrom
Develop
Nov 17, 2026
Merged

an1ndra merged 52 commits into
mainfrom
Develop

Conversation

@an1ndra

@an1ndra an1ndra commented Nov 17, 2026

Copy link
Copy Markdown
Owner

Release v0.2.0.

Cuts the notes in changelog/v0.2.0.md, bumps the pom and the Helm chart to 0.2.0, resets unreleased.md, and points CHANGELOG.md at the new file.

Verified before tagging: 164 tests and 157 smoke checks against the 0.2.0 jar with prompt logging, moderation and the semantic cache enabled.

- README leads with multi-provider support; OpenAI is the client contract, not
  the product identity
- add a Providers section (OPENAI and ANTHROPIC wire formats, how to add one)
- rename OpenAiController to ChatCompletionsController and OpenAiAdapter to
  OpenAiCompatibleAdapter, since they serve every compatible provider
- update the Maven description
- docs/API.md: every public and admin endpoint, error codes, metering fields
- docs/path.postman_collection.json: 26 requests in 8 folders, with tests that
  mint a key and chain created ids; verified end to end with Newman (49 assertions)
- docs/path.postman_environment.json: baseUrl and adminToken
- AGENTS.md: layout, request flow, conventions, where to add things, gotchas
Tomcat starts listening during context refresh, before ApplicationRunner
execution, so /actuator/health could return UP against a database with no
providers or routes. That made CI's smoke test race the seeder and fail
intermittently with 'provider has no API key configured'.

SeedState records completion; SeedReadinessIndicator reports OUT_OF_SERVICE
until then, so health checks (and load balancers) wait for a usable gateway.
CI additionally polls the catalogue before running the suite.
Clears the Node 20 deprecation warnings and the setup-java v4 deprecation:
checkout v7, setup-java v6, upload-artifact v7, qemu v4, buildx v4,
login v4, build-push v7, codeql v4, gh-release v3.
Circuit breaker (#11)
- ProviderCircuitBreaker: CLOSED/OPEN/HALF_OPEN per provider, 5 consecutive
  failures parks it for 30s, a successful probe closes it
- upstream Retry-After on 429 parks the provider for exactly that long
- open providers are skipped before any network call; max-attempts bounds the walk

Idempotency (#12)
- Idempotency-Key on POST /v1/chat/completions
- first request claims a PENDING row, repeat replays with X-Idempotent-Replay
- different body -> 409 idempotency_conflict, concurrent duplicate -> 409
  idempotency_in_progress
- streaming responses are captured and replayed as the same SSE sequence;
  >2MB responses are not stored
- keys hashed at rest, TTL 24h, hourly purge

Tests: 28 unit/integration (6 new for idempotency, 5 for the breaker), 72
smoke checks (4 new).

Closes #11
Closes #12
Limits (#19)
- per-key in-flight cap (default 8, overridable per key, 0 = unlimited),
  enforced before the call; excess returns 429 concurrency_limit_exceeded
- RequestSizeFilter rejects bodies above GATEWAY_MAX_REQUEST_BYTES with 413
  before Jackson reads them (Tomcat's maxPostSize only covers form posts)

Parameter fidelity (#14)
- AnthropicAdapter rejects seed, logit_bias, penalties, logprobs, modalities,
  audio, prediction, n > 1 and non-text response_format with 422
  unsupported_parameter instead of silently dropping them
- user maps to metadata.user_id

Tokenizer (#17)
- jtokkit counts real tokens when the upstream model has a known encoding,
  chars/4 otherwise; used only for pre-authorisation, billing still uses the
  provider-reported usage

Tests: 40 (12 new), 74 smoke checks (2 new).

Closes #14
Closes #17
Closes #19
- flyway-core + flyway-database-postgresql; ddl-auto moves from update to validate
- V1__baseline.sql covers all five tables and indexes in portable SQL
  (identity columns, timestamptz, enums as varchar + check so H2 and Postgres share
  one script)
- enums pinned to VARCHAR via @JdbcTypeCode, cost_usd widened to 8 decimals
- baseline-on-migrate handles existing pre-Flyway databases without data loss
- tests run the real migrations against in-memory H2

Verified: fresh DB applies V1 and validates; a pre-Flyway schema baselines at V1 and
boots; 40 tests, 74 smoke checks.

Closes #13
- opt-in per route (cache_enabled, cache_ttl_seconds)
- only explicit temperature:0, tool-free requests qualify
- keys scoped per client key; expired entries purged hourly
- non-streaming and streaming (SSE replay) both supported
- hits are recorded at zero cost with the avoided cost as savings,
  finish_reason=cache, X-Path-Cache: hit|miss|off
- V2 migration adds the table and the two route columns

Tests: 48 (8 new), 82 smoke checks (8 new).

Closes #10
Billing (#16)
- client keys gain daily_limit_usd, monthly_limit_usd and max_output_tokens
- windows are checked against stored spend (UTC day/month), before the call
- max_tokens clamped per key and globally (GATEWAY_MAX_OUTPUT_TOKENS)

Timeouts (#21)
- providers gain request_timeout_ms and stream_idle_timeout_ms, overriding defaults

Shutdown (#22)
- DrainState: readiness DOWN on SIGTERM, new requests 503 shutting_down,
  in-flight streams drain for GATEWAY_DRAIN_TIMEOUT

Operations (#23, #18)
- POST /admin/providers/{id}/probe and GET /admin/providers/health
- GET /admin/usage/export CSV statement, oldest first, per key or all

V3 migration adds the five columns. Tests: 55 (7 new), 87 smoke checks (5 new).

Closes #16
Closes #18
Closes #21
Closes #22
Closes #23
Metrics (#25)
- micrometer-registry-prometheus; /actuator/prometheus
- meters: path_requests_total (model/provider/status/streamed), latency timer,
  tokens by type, cost, cache hits
- micrometer-tracing-bridge-otel + OTLP exporter; one span per chat request,
  sampling 0 by default so nothing is exported unless configured

Logging (#26)
- RequestIdFilter: echoes or generates X-Request-Id, puts it in the MDC
- prod profile emits ECS JSON including requestId, traceId and spanId

Docs (#32)
- springdoc publishes /v3/api-docs, Swagger UI behind GATEWAY_SWAGGER_UI
- CI captures the spec as a build artifact

Tests: 57 (2 new), 90 smoke checks (3 new).

Closes #25
Closes #26
Closes #32
Aliases (#29)
- cheap / fast / long-context / vision resolved from the live route table
- advertised in /v1/models with resolves_to; no alias table to drift
- RouteResolver resolves aliases before the normal chain lookup

Canary (#30)
- per-route weight (default 100); starting route chosen by a stable hash of
  (client key, model), rest of the chain kept as fallback
- admin can set weight on any route

Tests: 61 (4 new), 93 smoke checks (3 new).

Closes #29
Closes #30
Audit (#27)
- admin_audit table records every provider/route/key mutation with actor,
  action, target and payload
- GET /admin/audit?action=&limit= to read it back
- failures are swallowed so auditing can never break the operation

Alerts (#28)
- AlertService posts Slack-compatible JSON to GATEWAY_ALERT_WEBHOOK
- events: provider.circuit_open, provider.rate_limited, key.budget_low
- per-event cooldown (default 1h), fire-and-forget on a virtual thread

V5 migration adds the audit table. Tests: 65 (4 new), 94 smoke checks (1 new).

Closes #27
Closes #28
Embeddings (#31)
- POST /v1/embeddings routed through the same chain (aliases, weights, fallback)
- OPENAI-type providers only; Anthropic returns 501 unsupported_endpoint
- metered on input tokens with the usual headers

Load test (#24)
- tools/load-test.js: k6 steady + streaming scenarios with latency/error thresholds

Property tests (#36)
- 200 generated conversations assert translation invariants
- found a real bug: system-only conversations produced an empty Anthropic
  message list; ChatRequestSpec now rejects them with 400

Tests: 66 (5 new), 97 smoke checks (3 new).

Closes #24
Closes #31
Closes #36
- prompt_logs table, hourly purge, GATEWAY_PROMPT_LOG_RETENTION
- disabled by default; enabled per gateway instance
- redacts sk-/gsk_/sk-ant-/Bearer tokens before storage, caps body size
- GET /admin/prompt-logs?keyId=&limit=
- non-streaming chat only for now

V6 migration. Tests: 70 (4 new), 98 smoke checks (1 new).

Closes #34
Allowlist (#15)
- client_keys.allowed_models: comma-separated ids and globs (gpt-*, claude-*)
- enforced in CreditGuard before any provider call; 403 model_not_allowed

Canary (#35)
- nightly workflow sends one 8-token call per configured provider
- skipped cleanly when no provider secrets exist
- opens an issue on failure so drift is visible

V7 migration. Tests: 72 (2 new), 99 smoke checks (1 new).

Closes #15
Closes #35
README keeps the pitch, the diagram, a quick start and a documentation index.
Details move to docs/:

- ARCHITECTURE.md   request flow, providers, routing, metering fields
- CONFIGURATION.md  every env var, timeouts, credentials, profiles
- OPERATIONS.md     admin API, observability, migrations, alerts, load testing
- LIMITS.md         design notes and known gaps
- TESTING.md        unit/integration suites and the mock upstream

README: 421 -> 93 lines. AGENTS.md and the PR template now point at the new files.
docs/ROADMAP.md lists #20 (Redis limiter + budget reservation), #33
(org/project/key hierarchy + scoped tokens) and #37 (fuzz the request parser)
with why-it-matters and effort, plus what has shipped and how to pick one up.
Linked from the README and the AGENTS.md documentation map.
…uzzing

Shared limiter (#20)
- RateLimitStore abstraction: in-memory default, Valkey backend for clusters
  (Valkey speaks the Redis protocol; Spring Data Redis client unchanged)
- budget reservation in the database: an atomic UPDATE reserves the worst-case
  cost before the call, released in a finally; stale reservations expire after 15m

Tenancy (#33)
- projects table; keys carry project_id and can be filtered
- admin_tokens with read / read+write scopes, hashed at rest, revocable
- the bootstrap GATEWAY_ADMIN_TOKEN keeps full access; read tokens get 403 on writes

Fuzzing (#37)
- Jazzer 0.30 harness over ChatRequestSpec + Anthropic translation, 10s per build
- only ApiException may escape; a regression test covers deep nesting

Also fixed: weighting only applies when a route weight differs from the default,
so synced fallback routes no longer turn routing into a lottery. The smoke suite is
now idempotent (unique cache prompts, all matching routes patched) and passes twice
against a warm database.

Tests: 85 (13 new), 104 smoke checks (5 new).
V8 and V9 migrations.

Closes #20
Closes #33
Closes #37
Security workflow (runs on PRs, pushes and weekly)
- gitleaks over the full history, with a reviewed allowlist for test placeholders
- dependency-review fails a PR that adds a high-severity dependency
- Trivy scans the tree for secrets and misconfiguration

Release
- CycloneDX SBOM embedded in the jar and attached to the release
- SHA-256 checksums for the jar and SBOM, verified with sha256sum -c
- the published image is scanned with Trivy; HIGH/CRITICAL fails the release
- the image digest is written to the workflow summary for pinning

Supply chain
- maven-wrapper.properties pins distributionSha256Sum for Maven 3.10.1
  (verified against the published .sha512)

Adds SECURITY.md and documents the gates in docs/OPERATIONS.md.
- Keep a Changelog format with an Unreleased section and a full 0.1.0 entry
- tools/changelog.py extracts a version's section and fails when it is missing
  or empty, so a tag without notes cannot publish a blank release
- release workflow uses body_path from that extraction
- CI validates the extractor; docs/OPERATIONS.md gains a release checklist
- fixed the action versions in the security workflow (trivy @v0.36.0,
  gitleaks @V3, dependency-review @v5)
- changelog/v0.1.0.md is the 0.1.0 entry and is published verbatim as the
  GitHub release body
- changelog/unreleased.md is the working file for the next release
- changelog/README.md documents the release steps
- CHANGELOG.md becomes a thin index linking the version files
- tools/changelog.py reads changelog/<tag>.md (accepts with or without the v),
  and exits non-zero when the file is missing or still says 'Nothing yet.'
- CI validates the extractor; the release checklist in docs/OPERATIONS.md and
  AGENTS.md points at the folder
- Dockerfile creates uid 1001 and switches to USER path, fixing Trivy DS-0002
  (HIGH) which was failing the Security workflow
- /app/data stays writable for the H2 file and the generated master key
- release image scan blocks on CRITICAL only; HIGH stays visible but the Debian
  base rotates too often to gate on
- changelog extractor requires at least one real bullet, not just placeholders
ubuntu-latest is being migrated to Ubuntu 26 in October 2026; pinning the label
keeps the build reproducible and silences the migration warning. Documented in
docs/OPERATIONS.md and AGENTS.md.
Project budgets
- projects carry lifetime/daily/monthly USD limits, checked against the spend of
  every key in the project before the call
- GET /admin/projects/{id}/usage rolls up today, month and all time
- PATCH /admin/projects/{id} updates the limits

Prompt logging
- streaming responses are captured through the same wrapper as idempotency, so the
  SSE payload is stored (redacted, capped) instead of being skipped

Anthropic prompt caching
- cache_control on text and image parts is forwarded unchanged

V10 migration. Tests: 86 (1 new), 106 smoke checks (2 new). The remaining gaps are
now grouped as per-node or provider-inherent rather than open TODOs.
Gemini exposes an OpenAI-compatible surface at
https://generativelanguage.googleapis.com/v1beta/openai (Bearer auth, chat,
streaming, tools, images, embeddings), so it needs no new adapter.

- GEMINI_API_KEY in config, .env.example and provider credentials resolution
- gemini seeded as an OPENAI-type provider; routes are added per model after
  setting a price (model sync imports the list but leaves unknown prices at 0)
- docs updated: provider table, configuration recipe with a curl example

Verified: fresh boot seeds the provider at the correct base URL and reports
credential_source=config when GEMINI_API_KEY is set.
- bruno/ contains bruno.json, a Local environment and 26 .bru requests in the
  same seven folders as the Postman collection
- create requests carry script:post-response to capture apiKey, keyId,
  createdProviderId and createdRouteId for later requests
- tools/postman_to_bruno.py regenerates it from the Postman collection and
  preserves the hand-written README
- bruno/README.md documents opening, the setup order and the CLI command

Verified: 'bru run . -r --env Local' against a local gateway with the mock
upstream -> 26 requests, 26 passed.
- usage_records grew forever: nightly purge honouring GATEWAY_USAGE_RETENTION
  (180d default, 0 keeps history)
- a crashed instance left an Idempotency-Key PENDING for 24h: claims older than
  GATEWAY_IDEMPOTENCY_PENDING (2m) are reclaimed, with a flush so the unique
  constraint is not hit on reinsert
- /v1/embeddings bypassed DrainState and the per-key concurrency cap
- traces stopped at the gateway: traceparent is injected into every upstream call

Tests: 87 (1 new for stale claims), 106 smoke checks. Docs and changelog updated.
Twelve issues filed from the audit (#38-#46, #49-#51) and grouped by area in
docs/ROADMAP.md. #47 and #48 were duplicates of shipped work and are closed
with pointers to the implementations.
Billing
- #38 models/sync imports unknown-priced routes disabled instead of serving them free
- #44 reservation uses the expected cost (512 output tokens) while still refusing when
  the estimate exceeds the balance; the cap no longer blocks small balances

Security
- #39 500s return a reference id; the exception is logged, not sent
- #40 chunked bodies are counted while reading and rejected with 413
- #41 demo key defaults off and is refused in the prod profile
- #42 admin auth failures are rate limited per address (10 / 5m)
- #43 POST /admin/rotate-master-key re-encrypts credentials before swapping keys
- #51 /v3/api-docs is off in the prod profile

API and operations
- #45 projectId filters on /admin/usage, /admin/stats and the CSV export
- #46 the in-memory limiter evicts stale windows again
- #49 release tags must be ancestors of main
- #50 shared ObjectMapper for error mapping

Tests: 96 (9 new), 113 smoke checks (7 new), verified twice against a warm database.
- ProviderHealthIndicator reports circuit state and retry-after per provider
- a health group exposes it at /actuator/health/providers without leaking the rest
- overall health only goes OUT_OF_SERVICE when every enabled provider is parked, so a
  single failure still leaves the gateway in rotation
- clearer 'could not reach' message when the network exception has no message

Tests: 98 (2 new).
- docs/WHY-PATH.md compares Path with LiteLLM, Portkey and OpenRouter, and names
  where Path is weaker (model coverage, hosted option, guardrails, dashboards)
- README hero leads with the JVM-native positioning and links to it
- tools/demo.sh runs catalogue, chat, streaming, cache hit and usage against the
  mock upstream with no API keys; verified end to end
- pom metadata required by Central (url, licences, developers, scm)
- release profile: plain library jar (repackage skipped), sources and javadoc jars,
  GPG signing and the Sonatype central-publishing plugin with autoPublish=false
- source/javadoc plugins live only in the profile, so the default build still
  produces a single fat jar for Docker and CI
- optional CI step gated on the PUBLISH_TO_CENTRAL repository variable
- docs/PUBLISHING.md covers namespace verification, key creation, the Portal token
  and the exact commands, plus the immutability warning

Verified locally with a throwaway key in a temporary GNUPGHOME: jar, sources,
javadoc and pom are each signed, and the main jar is the plain 338K library.
- <groupId> is io.github.an1ndra; anindra.com does not resolve, so the GitHub
  namespace (a public repo of that exact name) is the verification route
- the Java package stays com.anindra.path; only the Maven coordinates change
- docs/PUBLISHING.md rewritten for the GitHub namespace, with the keyserver email
  confirmation and the plain-vs-runnable jar caveat
- smoke_test.py no longer trusts a demo key scraped from a stale log: it checks the
  key authenticates and creates a fresh one otherwise

Verified: 98 tests, 113/113 smoke checks, release profile still emits the signed
jar/sources/javadoc/pom set.
The open-stories table still listed #38-#51, all closed and shipped in 93ae278.
It now lists the five genuinely open stories (#59, #61, #62, #63, #64) and moves the
audit work into Delivered.
Grouped by where the value is (product, billing and limits, reach and operations)
rather than by severity, so the next pick is obvious.
Eleven labels (seven areas, three effort bands, blocked) applied to all sixteen open
stories; the rule and the per-label meaning now live in AGENTS.md so the next issue is
tagged the same way.
…alerts

#66 POST /v1/tokens/count answers 'will this prompt fit?' without spending a call.
  Free but authenticated: model allowlist, drain state, concurrency and RPM limits all
  apply. Counts tool definitions, which is where naive estimates are wrong, and reports
  estimated=true when the chars/4 heuristic was used.

#67 /v1/embeddings honours Idempotency-Key with the same semantics as chat: replay,
  409 on a different body, 409 while in flight. A retried embedding batch is no longer
  billed twice.

#69 project budgets warn before they run out. AlertService.notifyOnce dedupes per
  project, window and period, so a busy project emits one event, not thousands.

Tests: 109 (11 new), smoke 118 (5 new).
#65 A project that owns forty keys could push 4,800 rpm with nothing to cap it. Projects
  now carry rpm_limit and max_concurrent_requests, enforced on top of the per-key caps
  (a request must pass both) and shared across replicas through the Valkey backend.
  The rate limiter is keyed by bucket (key:<id> / project:<id>) rather than a bare key
  id; the in-memory implementation is unchanged in behaviour.

0 means unlimited, negatives are rejected at the admin API, and the 429 names the cap
that was hit.

Tests: 112 (3 new), smoke 123 (5 new).
#68 'cheap' is recomputed from prices on every request, so it can move under a product's
  feet. /admin/aliases stores an explicit ordered chain that keeps meaning what it meant
  when it was created, expands in RouteResolver in place, and appears in /v1/models.

Chains are validated at write time: an unknown model, a self-reference or more than ten
hops is a 400, so a broken alias cannot become a 500 on a customer request. Shadowing a
computed alias is allowed but flagged with shadows_computed and logged.

Tests: 121 (8 new), smoke 130 (7 new).
#63 A degraded-but-alive provider kept the traffic until it actually 5xxed, because
  routing was a static priority list and the breaker only reacts to consecutive failures.

  Routes now carry strategy (priority by default, so nothing changes until opted in):
  - fastest       order the chain by EWMA latency, tie-broken on price
  - cheapest-viable  cheapest route under latency_slo_ms, the rest demoted to fallback

  RoutingMetrics keeps an EWMA per (provider, model) with a ~5-sample half-life, updated
  from both the streaming and non-streaming paths. A route with no samples counts as
  viable, so a cold start cannot deadlock routing. Route views expose ewma_latency_ms and
  ewma_error_rate so an operator can see why traffic moved; provider.order still wins over
  any strategy.

Tests: 131 (10 new), smoke 134 (4 new), verified cold and warm.
#72 Only docker-compose existed, so every operator rewrote the Deployment and got the
  subtle parts wrong. The chart encodes the two that matter:

  - terminationGracePeriodSeconds must exceed GATEWAY_DRAIN_TIMEOUT, or a rolling update
    truncates live SSE streams. The template fails to render when it does not.
  - readiness stays on /actuator/health, because one parked provider still has fallback;
    removing the pod from the Service would be wrong.

Also: HPA (with custom-metric hook), PodDisruptionBudget, optional Postgres StatefulSet
and Valkey, ServiceMonitor, and no inline Secret by default — existingSecret is required
unless secrets are passed explicitly.

Verified with helm v3.16.3: lint clean, every toggle renders, the drain guard fails closed,
and no Secret object is rendered when existingSecret is set. A CI job runs the same checks.
Seven of sixteen shipped in 0.2 (routing, aliases, project limits and alerts, token
counting, idempotent embeddings, Helm); the rest are grouped by where the value is.
A tag is mutable: whoever controls the action repository can repoint @v7 at any commit,
and the workflow would run it with this repository's token. Every uses: now names a
commit SHA, with the tag kept in a comment for readability.

  actions/checkout                3d3c42e5...  v7.0.1
  actions/setup-java              d1374f8a...  v6
  actions/upload-artifact         cf430e03...  v7
  actions/dependency-review       a1d282b3...  v5
  docker/*                        ...          v4 / v7
  github/codeql-action/*          33d7b85d...  v4
  aquasecurity/trivy-action       ed142fd0...  v0.36.0
  azure/setup-helm                1a275c3b...  v4
  gitleaks/gitleaks-action        e0c47f4f...  v3
  softprops/action-gh-release     8a0cd651...  v3

Each SHA was resolved from the tag today, so behaviour is identical to what the floating
tags resolved to; the difference is that it can no longer change under us. Dependabot
already watches the github-actions ecosystem, so the pins still get bumped weekly.

All five workflows still parse (yaml.safe_load) and no uses: is left floating.
#64 A key and a project are operator concepts. Without an end-user axis the gateway could
  not answer 'which of my customers spent this', which is the whole point of embedding it
  in a product.

  - end_user (or OpenAI's 'user') is stored on the usage row, indexed with the key and
    timestamp, and rejected past 128 characters
  - /v1/usage returns an end_users[] breakdown; /admin/usage and the CSV export filter by
    endUser and the CSV carries the column
  - client_keys.end_user_daily_limit_usd reserves against that end user with an atomic
    conditional update on (key, end_user), so two concurrent calls cannot both take the
    last of the budget

Two fixes found while testing this:
  - the cap used to sit behind the remaining == null early return, so a key with no
    credit limit skipped it entirely; the estimates now run before that branch
  - CreditGuard returns a Reservation (key + end-user) instead of a bare BigDecimal, so
    both are released together

Tests: 141 (10 new), smoke 142 (8 new), verified cold and warm.
#70 Nothing scrubbed prompt content, and docs/LIMITS.md already admitted prompt logging
was a PII store.

- PiiRedactor is a pure function: emails, phones, Luhn-valid card numbers, Aadhaar and
  PAN, replaced by numbered tokens so a stored conversation stays readable
- prompt logs redact credentials first, then PII (on by default)
- GATEWAY_GUARDRAILS_REDACT also redacts what the provider receives, before the body is
  hashed or stored, so an idempotent replay cannot resurrect the original text
- GATEWAY_GUARDRAILS_MODERATION calls the provider's /moderations first and refuses a
  flagged prompt with 400 content_flagged plus the categories; a moderation outage is
  logged and the request proceeds rather than becoming a gateway outage
- GET /admin/settings reports which of these are active

False positives cost real time here: the first cut redacted the 10-digit epoch in every
response body as a phone number, and read a 16-digit order id as an Aadhaar. Contiguous
digits are now a phone only when they look like a mobile (10-13 digits, starting 6-9),
and Aadhaar cannot match inside a longer digit run.

Tests: 153 (11 new), smoke 148 with guardrails on and 144 with defaults.
#73 The adapter assumed the public OpenAI shape: bearer auth, /v1/chat/completions, model
in the body. Azure puts the deployment in the path, the version in a query parameter and
the credential in an api-key header, so every request failed.

A provider with api_version set is now an Azure resource:

  {base}/openai/deployments/{upstream_model_id}/chat/completions?api-version=...

so the route table maps a public model to a deployment, which is what Azure requires.
ProviderUrls owns the URL and header shapes and is used by chat, embeddings, moderation
and the probe, so the four cannot drift apart.

The probe uses the deployment listing rather than /models, which Azure does not have, and
an unsupported version surfaces Azure's own message instead of a generic 502.

Found while testing: the PATCH path ignored api_version entirely, because the update was
nested inside the "if base_url was supplied" branch.

Tests: 159 (6 new in ProviderUrlsTest), smoke 148 (5 new: create, chat, embeddings, and a
bad api-version surfacing the provider message), verified cold and warm.
#71 The exact cache only catches literal repeats; a one-word difference missed. A second
tier embeds the prompt and serves a stored completion when cosine similarity clears a
per-route threshold.

It stays inside the exact cache's rules, because a hit here answers a different prompt:
the route opted into caching, the request must be temperature 0 and tool-free, and entries
are scoped per client key. The tier itself is opt-in per route (semantic_cache_enabled,
semantic_threshold) and needs GATEWAY_SEMANTIC_MODEL.

Candidates are bounded by semantic-max-entries, so the comparison is a small scan rather
than a table scan; docs/LIMITS.md says so, and says an ANN index is what a large
deployment wants. Embedding failures are swallowed: a failure degrades to the exact tier
rather than failing the request.

Scope: non-streaming only. Streaming keeps the exact tier. /admin/stats reports
cache.semantic_hits.

The mock upstream now emits a deterministic bag-of-words vector, so the smoke suite can
assert real geometry: a near duplicate hits, an unrelated prompt does not.

Tests: 164 (5 new geometry tests), smoke 153 (5 new), verified cold and warm.
Eleven of sixteen shipped. The rest are blocked on a decision (#62, #75), on an
environment this box does not have (#74 needs Valkey, #61 needs Postgres), or on the
maintainer's account (#59).
- version 0.2.0 in the pom and the Helm chart (appVersion too)
- changelog/v0.2.0.md: routing, billing, safety, providers and the twelve audit fixes
- unreleased.md reset to the empty template
- CHANGELOG.md index points at the new notes

Verified before tagging: 164 tests, 157 smoke checks against the 0.2.0 jar with prompt
logging, moderation and the semantic cache all enabled.
@github-actions

Copy link
Copy Markdown

Dependency Review

The following issues were found:
  • ✅ 0 vulnerable package(s)
  • ✅ 0 package(s) with incompatible licenses
  • ✅ 0 package(s) with invalid SPDX license definitions
  • ⚠️ 1 package(s) with unknown licenses.
See the Details below.

License Issues

.github/workflows/security.yml

PackageVersionLicenseIssue Type
gitleaks/gitleaks-actione0c47f4f8be36e29cdc102c57e68cb5cbf0e8d1eNullUnknown License

OpenSSF Scorecard

Scorecard details
PackageVersionScoreDetails
actions/actions/checkout 3d3c42e5aac5ba805825da76410c181273ba90b1 🟢 5.8
Details
CheckScoreReason
Code-Review🟢 10all changesets reviewed
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Binary-Artifacts🟢 10no binaries found in the repo
Maintained⚠️ 00 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Pinned-Dependencies🟢 3dependency not pinned by hash detected -- score normalized to 3
Fuzzing⚠️ 0project is not fuzzed
License🟢 10license file detected
Packaging⚠️ -1packaging workflow not detected
Signed-Releases⚠️ -1no releases found
Security-Policy🟢 9security policy file detected
Branch-Protection🟢 5branch protection is not maximal on development and all release branches
SAST🟢 10SAST tool is run on all commits
actions/actions/dependency-review-action a1d282b36b6f3519aa1f3fc636f609c47dddb294 UnknownUnknown
actions/aquasecurity/trivy-action ed142fd0673e97e23eac54620cfb913e5ce36c25 🟢 6.7
Details
CheckScoreReason
Code-Review🟢 9Found 15/16 approved changesets -- score normalized to 9
Maintained🟢 1014 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 10
Binary-Artifacts🟢 10no binaries found in the repo
Packaging⚠️ -1packaging workflow not detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Token-Permissions🟢 7detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies🟢 8dependency not pinned by hash detected -- score normalized to 8
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Fuzzing⚠️ 0project is not fuzzed
License🟢 10license file detected
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ -1internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
Security-Policy⚠️ 0security policy file not detected
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
actions/gitleaks/gitleaks-action e0c47f4f8be36e29cdc102c57e68cb5cbf0e8d1e 🟢 5
Details
CheckScoreReason
Code-Review🟢 3Found 6/16 approved changesets -- score normalized to 3
Maintained⚠️ 00 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
Packaging⚠️ -1packaging workflow not detected
Binary-Artifacts🟢 10no binaries found in the repo
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Security-Policy🟢 10security policy file detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies🟢 10all dependencies are pinned
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
License🟢 9license file detected
Signed-Releases⚠️ -1no releases found
Fuzzing⚠️ 0project is not fuzzed
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
Branch-Protection🟢 6branch protection is not maximal on development and all release branches
maven/com.code-intelligence:jazzer-junit 0.30.0 UnknownUnknown
maven/com.knuddels:jtokkit 1.1.0 UnknownUnknown
maven/io.micrometer:micrometer-registry-prometheus UnknownUnknown
maven/io.micrometer:micrometer-tracing-bridge-otel UnknownUnknown
maven/io.opentelemetry:opentelemetry-exporter-otlp UnknownUnknown
maven/org.apache.maven.plugins:maven-gpg-plugin 3.2.8 🟢 3.8
Details
CheckScoreReason
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Packaging⚠️ -1packaging workflow not detected
Code-Review⚠️ 0Found 0/22 approved changesets -- score normalized to 0
Maintained🟢 1023 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10
Binary-Artifacts⚠️ 0binaries present in source code
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Fuzzing⚠️ 0project is not fuzzed
License🟢 10license file detected
Signed-Releases⚠️ -1no releases found
Branch-Protection⚠️ -1internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
Security-Policy🟢 10security policy file detected
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
maven/org.apache.maven.plugins:maven-javadoc-plugin UnknownUnknown
maven/org.apache.maven.plugins:maven-source-plugin UnknownUnknown
maven/org.cyclonedx:cyclonedx-maven-plugin 2.9.3 UnknownUnknown
maven/org.flywaydb:flyway-core UnknownUnknown
maven/org.flywaydb:flyway-database-postgresql UnknownUnknown
maven/org.sonatype.central:central-publishing-maven-plugin 0.12.0 UnknownUnknown
maven/org.springdoc:springdoc-openapi-starter-webmvc-ui 2.8.17 🟢 5
Details
CheckScoreReason
Maintained🟢 1030 commit(s) and 12 issue activity found in the last 90 days -- score normalized to 10
Code-Review⚠️ 2Found 5/25 approved changesets -- score normalized to 2
Binary-Artifacts🟢 10no binaries found in the repo
Security-Policy🟢 10security policy file detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Fuzzing⚠️ 0project is not fuzzed
License🟢 10license file detected
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
Packaging🟢 10packaging workflow detected
Branch-Protection⚠️ 0branch protection not enabled on development/release branches
Signed-Releases⚠️ -1no releases found
maven/org.springframework.boot:spring-boot-starter-data-redis UnknownUnknown

Scanned Files

  • .github/workflows/security.yml
  • pom.xml

@an1ndra
an1ndra merged commit f45aa07 into main Nov 17, 2026
16 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant