diff --git a/README.md b/README.md index de5c205..eae70e6 100644 --- a/README.md +++ b/README.md @@ -46,7 +46,7 @@ Once started, modelrelay is accessible at `http://localhost:7352/`. Router endpoint: - Base URL: `http://127.0.0.1:7352/v1` -- API key: any string +- API key: any string — unless the proxy is protected with `MODELRELAY_PROXY_TOKEN`, in which case send that value (see [Security](https://schaetzkc.com/modelrelay/security/#proxy-token)) - Model: `auto-fastest` (router picks actual backend) ## 🚀 Install via Docker diff --git a/docker-compose.yml b/docker-compose.yml index 5420046..3253686 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -13,5 +13,6 @@ services: # Recommended if the dashboard is reachable beyond this machine: require sign-in for the admin API. # - MODELRELAY_ADMIN_TOKEN=change-me-to-16-or-more-random-characters # - MODELRELAY_ALLOWED_HOSTS=modelrelay.example.com # only if you open the dashboard through a host name + # - MODELRELAY_PROXY_TOKEN=change-me-to-16-or-more-random-characters # require an API key on /v1 volumes: modelrelay_config: diff --git a/docs/configuration.md b/docs/configuration.md index 3f068b9..97caf9d 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -26,6 +26,7 @@ nav_order: 10 - Security and server settings (see [Security](security.md)): - `MODELRELAY_ADMIN_TOKEN` (require sign-in for the admin API and dashboard) + - `MODELRELAY_PROXY_TOKEN` (require `Authorization: Bearer ` on the `/v1` proxy; unset leaves it open) - `MODELRELAY_ALLOWED_HOSTS` (extra host names the admin API answers to; `*` disables the check) - `MODELRELAY_ALLOWED_ORIGINS` (extra browser origins, for reverse proxies that rewrite `Host`) - `MODELRELAY_HOST` (interface to listen on; same as `--host`) diff --git a/docs/endpoints.md b/docs/endpoints.md index c285c33..84df0cb 100644 --- a/docs/endpoints.md +++ b/docs/endpoints.md @@ -5,6 +5,10 @@ nav_order: 5 # Endpoints +## Authentication + +`/v1` is open by default: any `Authorization` value is accepted and ignored. If the router is reachable by other people, set `MODELRELAY_PROXY_TOKEN`; callers must then send `Authorization: Bearer `, which is the value they already put in their "API key" field. A missing or wrong credential gets `401` with an OpenAI-shaped `invalid_api_key` error. CORS preflights stay open. See [Security](security.md#proxy-token). + ## `/v1/chat/completions` `POST /v1/chat/completions` is an OpenAI-compatible chat completions endpoint. diff --git a/docs/security.md b/docs/security.md index 266c79a..d9ddf9f 100644 --- a/docs/security.md +++ b/docs/security.md @@ -9,7 +9,7 @@ modelrelay has two kinds of endpoints with different jobs, and they are protecte | Endpoint | What it is | Protection | |---|---|---| -| `/v1/*` | The OpenAI-compatible proxy your tools talk to | Open by design (any `Authorization` value is accepted and ignored). CORS is allowed so browser-based clients work. | +| `/v1/*` | The OpenAI-compatible proxy your tools talk to | Open by default; an optional bearer token (`MODELRELAY_PROXY_TOKEN`) requires callers to send it. CORS is allowed so browser-based clients work. | | `/api/*` | The admin API behind the dashboard: settings, provider keys, tags, logs | Host and origin checks, plus an optional admin token. **No CORS.** | The admin API can read and change your provider API keys, so it is the part to protect. Anyone who can use `/v1` can spend your free-tier quota, but only the admin API can see or redirect your keys. @@ -41,7 +41,24 @@ modelrelay ``` - After 10 wrong guesses a client is locked out for 15 minutes. - Use at least 16 random characters. The router warns at startup if the token is shorter. -- The token is read from the environment only; it is never written to the config file. The `/v1` proxy is not affected. +- The token is read from the environment only; it is never written to the config file. The `/v1` proxy is not affected by the admin token; see [Proxy token](#proxy-token). + +## Proxy token + +Anyone who can reach `/v1` can spend the quota of the providers you configured. On a laptop that is fine; on a VPS, a shared host or behind a public reverse proxy, set a proxy token: + +```bash +export MODELRELAY_PROXY_TOKEN="$(openssl rand -hex 24)" +modelrelay +``` + +- Every `/v1` request must then send it as the API key: `Authorization: Bearer `. Clients that already have an "API key" field use it unchanged — for modelrelay the value stops being ignored. +- Without the header, or with a wrong one, the proxy answers `401` with an OpenAI-shaped error (`invalid_api_key`). Comparison is constant-time. +- **Unset means open**, exactly as before, so existing local setups keep working. +- `OPTIONS` preflights are still answered without the token, because a browser must be able to ask before sending `Authorization`. +- The token is read from the environment only and never written to the config file. It is independent of `MODELRELAY_ADMIN_TOKEN`: the admin token protects the dashboard, this one protects the proxy. +- Use at least 16 random characters. The router warns at startup if the token is shorter. Changing it requires a restart. +- The startup banner shows the proxy as protected and tells clients which API key to use. ## Host names, reverse proxies and Docker @@ -63,8 +80,8 @@ export MODELRELAY_ALLOWED_ORIGINS="https://modelrelay.example.com" | Local use at `http://localhost:7352` | Nothing. | | LAN access by IP address (`http://192.168.1.20:7352`) | Nothing for the checks. Set a token, because the dashboard is reachable by every device on the network. | | LAN access by machine name (`http://mybox.local:7352`) | Nothing (the machine's own name is allowed). | -| Docker Compose with a published port | Nothing for `localhost` or IP access. Add `MODELRELAY_ADMIN_TOKEN` to the service's environment. | -| Behind a reverse proxy or Kubernetes ingress | Set `MODELRELAY_ALLOWED_HOSTS` (and `MODELRELAY_ALLOWED_ORIGINS` if the proxy changes `Host`), and a token. The `/v1` traffic from other services is not affected. | +| Docker Compose with a published port | Nothing for `localhost` or IP access. Add `MODELRELAY_ADMIN_TOKEN` to the service's environment, and `MODELRELAY_PROXY_TOKEN` if other hosts can reach the published port. | +| Behind a reverse proxy or Kubernetes ingress | Set `MODELRELAY_ALLOWED_HOSTS` (and `MODELRELAY_ALLOWED_ORIGINS` if the proxy changes `Host`), and an admin token. If the proxy is publicly reachable, also set `MODELRELAY_PROXY_TOKEN` so `/v1` callers must send it. | Health checks that only open a TCP connection are not affected by any of this. @@ -90,7 +107,7 @@ The default has **not** been changed, because making loopback the default would | The Docker image | Would have to set `MODELRELAY_HOST=0.0.0.0` for published ports to work. | | Kubernetes | Same as Docker. | -Recommendation: if everything runs on one machine, use `--host 127.0.0.1`. Otherwise keep the default and set an admin token. +Recommendation: if everything runs on one machine, use `--host 127.0.0.1`. Otherwise keep the default, set an admin token, and set a proxy token if `/v1` is reachable by people you do not trust. ## Reporting a problem diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 6a55873..1e610ff 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -5,6 +5,10 @@ nav_order: 11 # Troubleshooting +## `/v1` returns `401 invalid_api_key` + +`MODELRELAY_PROXY_TOKEN` is set, so the proxy requires that token as the API key. Send it as `Authorization: Bearer ` and use the same value wherever the client asks for an API key. Unset the variable (and restart) to make the proxy open again. Check the startup banner: it says when the proxy is protected. See [Security](security.md#proxy-token). + ## Clicking the update button or running `modelrelay` won't perform an update To trigger a manual npm update and restart the service, run: diff --git a/lib/proxy-guard.js b/lib/proxy-guard.js new file mode 100644 index 0000000..913b179 --- /dev/null +++ b/lib/proxy-guard.js @@ -0,0 +1,57 @@ +// Protection for the OpenAI-compatible proxy (/v1/*): an optional bearer token. +// +// The proxy is open by default because it is usually reached over loopback by local tools, and its CORS headers +// are what browser-based clients rely on. When the router is published beyond this machine (a VPS, a reverse +// proxy, a shared host), anyone who knows the URL can spend the quota configured for the providers. Setting +// MODELRELAY_PROXY_TOKEN requires `Authorization: Bearer ` on every /v1 request; without it nothing +// changes, so existing clients keep working. +// +// CORS preflights stay open: a browser must be able to ask for permission before it sends the header. + +import { createHmac, randomBytes, timingSafeEqual } from 'node:crypto'; + +export const PROXY_TOKEN_ENV = 'MODELRELAY_PROXY_TOKEN'; +export const MIN_PROXY_TOKEN_LENGTH = 16; + +/** Reads the proxy token from the environment, trimmed. Empty means the proxy stays open. */ +export function getProxyToken(env = process.env) { + const raw = env?.[PROXY_TOKEN_ENV]; + return typeof raw === 'string' ? raw.trim() : ''; +} + +/** Returns the credential of an `Authorization: Bearer ` header, or '' for any other shape. */ +export function parseBearerToken(header) { + if (typeof header !== 'string') return ''; + const match = /^Bearer[ \t]+(\S+)$/i.exec(header.trim()); + return match ? match[1] : ''; +} + +/** + * Builds the token check. With no token the proxy is open; with one, only an exact bearer credential passes. + * Comparison runs over HMACs under a random per-process key, so it is constant-time and does not leak the + * token length. + */ +export function createProxyAuth({ token } = {}) { + const normalized = typeof token === 'string' ? token.trim() : ''; + const enabled = normalized.length > 0; + const macKey = randomBytes(32); + const mac = (value) => createHmac('sha256', macKey).update(String(value)).digest(); + const expected = enabled ? mac(normalized) : null; + + const verify = (header) => { + if (!enabled) return true; + const candidate = parseBearerToken(header); + return candidate.length > 0 && timingSafeEqual(mac(candidate), expected); + }; + + return { enabled, verify }; +} + +/** Express middleware for /v1. Mount it after the CORS middleware so preflights are answered first. */ +export function createProxyMiddleware({ token } = {}) { + const auth = createProxyAuth({ token }); + return (req, res, next) => { + if (!auth.enabled || req.method === 'OPTIONS' || auth.verify(req.headers.authorization)) return next(); + return res.status(401).json({ error: { message: 'Invalid API key', type: 'invalid_request_error', code: 'invalid_api_key' } }); + }; +} diff --git a/lib/server.js b/lib/server.js index 353fd2f..774d4f8 100644 --- a/lib/server.js +++ b/lib/server.js @@ -13,6 +13,7 @@ import { getPreferredLanIpv4Address } from './network.js'; import { capLength, isSafeObjectKey } from './text.js'; import { rateLimit } from 'express-rate-limit'; import { createAdminAuth, createAdminMiddleware, createAuthRoutes, parseAllowedHosts, parseAllowedOrigins } from './admin-guard.js'; +import { createProxyAuth, createProxyMiddleware, getProxyToken, MIN_PROXY_TOKEN_LENGTH } from './proxy-guard.js'; import { classifyFailureStatus, createTelemetryStore, normalizeTelemetryStore, recordFailure, recordSuccess, reliabilityMultiplier, summarizeTelemetry, summarizeTelemetryEntry } from './telemetry.js'; import { createHash, randomInt, randomUUID } from 'crypto'; import { getAutostartStatus } from './autostart.js'; @@ -3625,6 +3626,12 @@ export async function runServer(config, port, enableLog = true, bannedModels = [ next(); }); + // Optional bearer token for the proxy (see lib/proxy-guard.js). Off unless MODELRELAY_PROXY_TOKEN is set, so + // local clients keep working; when set, every /v1 request must send it as the API key. + const proxyToken = getProxyToken(); + const proxyAuth = createProxyAuth({ token: proxyToken }); + app.use('/v1', createProxyMiddleware({ token: proxyToken })); + app.get('/api/auth/status', authRoutes.status); app.post('/api/auth/login', loginLimiter, authRoutes.login); app.post('/api/auth/logout', authRoutes.logout); @@ -5165,13 +5172,17 @@ export async function runServer(config, port, enableLog = true, bannedModels = [ console.log(chalk.yellow(' ⚠️ The admin API (which can read your provider API keys) is reachable from other devices on your network without a token.')); console.log(chalk.yellow(' Set MODELRELAY_ADMIN_TOKEN to require sign-in, or start with --host 127.0.0.1 to keep it on this machine.')); } + if (proxyAuth.enabled) { + console.log(chalk.green(' 🔒 Proxy API protected by MODELRELAY_PROXY_TOKEN (send it as the API key).')); + if (proxyToken.length < MIN_PROXY_TOKEN_LENGTH) console.log(chalk.yellow(' ⚠️ The proxy token is short. Use at least 16 random characters.')); + } if (lanIp) { console.log(chalk.green(` ✅ Visit ${chalk.bold(`http://${lanIp}:${port}`)} to access the Web UI from another computer on your network.`)); } console.log(chalk.green(` ✅ Router proxy active at ${chalk.bold(`http://localhost:${port}/v1`)}`)); console.log(chalk.dim(` Usage in OpenCode/Cursor:`)); console.log(chalk.dim(` - Provider Base URL: http://localhost:${port}/v1`)); - console.log(chalk.dim(` - API Key: (anything, ignored)`)); + console.log(chalk.dim(` - API Key: ${proxyAuth.enabled ? 'the MODELRELAY_PROXY_TOKEN value' : '(anything, ignored)'}`)); console.log(chalk.dim(` - Model: (anything, ignored)`)); console.log(); }); diff --git a/package-lock.json b/package-lock.json index 1abf3db..c28d0d8 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@schaetzkc/modelrelay", - "version": "1.29.0", + "version": "1.30.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@schaetzkc/modelrelay", - "version": "1.29.0", + "version": "1.30.0", "license": "MIT", "dependencies": { "chalk": "^5.4.1", diff --git a/package.json b/package.json index 8d06025..49bf96c 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@schaetzkc/modelrelay", - "version": "1.29.0", + "version": "1.30.0", "description": "OpenAI-compatible local router that benchmarks free coding models across providers and forwards requests to the best available model.", "keywords": [ "nvidia", diff --git a/test/test.js b/test/test.js index 5c47e26..5cbb206 100644 --- a/test/test.js +++ b/test/test.js @@ -41,6 +41,7 @@ import { classifyFailureStatus, createTelemetryStore, getReliability, normalizeT import http from 'node:http' import express from 'express' import { SESSION_COOKIE, buildClearedCookie, buildSessionCookie, createAdminAuth, createAdminMiddleware, createAuthRoutes, evaluateRequestOrigin, isAllowedHost, parseAllowedHosts, parseAllowedOrigins, parseCookies, parseHostHeader } from '../lib/admin-guard.js' +import { PROXY_TOKEN_ENV, createProxyMiddleware, getProxyToken } from '../lib/proxy-guard.js' import '../public/model-search.js' import { capLength, isSafeObjectKey, stripTrailingSuffixes, trimEndWhile, trimStartWhile, trimWhile } from '../lib/text.js' import { OPENCLAW_ROUTING_PRESETS, applyOpenClawConfig, buildOpenClawProviderConfig } from '../lib/onboard.js' @@ -4298,3 +4299,110 @@ describe('OpenCode free-tier SSE stream (stateful) split-chunk', () => { }) }) + +describe('proxy token: end to end over HTTP', () => { + const TOKEN = 'proxy-token-for-tests-0123456789' + + const startApp = async (proxyToken) => { + const app = express() + app.use('/v1', (req, res, next) => { + res.header('Access-Control-Allow-Origin', '*') + res.header('Access-Control-Allow-Headers', 'Content-Type, Authorization') + if (req.method === 'OPTIONS') return res.sendStatus(204) + next() + }) + app.use('/v1', createProxyMiddleware({ token: proxyToken })) + app.use(express.json()) + app.get('/v1/models', (req, res) => res.json({ object: 'list', data: [] })) + app.post('/v1/chat/completions', (req, res) => res.json({ ok: true, model: req.body?.model ?? null })) + app.get('/public', (req, res) => res.json({ ok: true })) + const server = await new Promise(resolve => { const s = app.listen(0, '127.0.0.1', () => resolve(s)) }) + const port = server.address().port + const call = (options = {}) => new Promise((resolve, reject) => { + const body = options.body === undefined ? undefined : JSON.stringify(options.body) + const req = http.request({ host: '127.0.0.1', port, method: options.method || 'GET', path: options.path || '/v1/models', + headers: { host: `localhost:${port}`, ...(body ? { 'content-type': 'application/json', 'content-length': Buffer.byteLength(body) } : {}), ...(options.headers || {}) } }, (res) => { + let data = '' + res.on('data', c => { data += c }) + res.on('end', () => { let json = null; try { json = JSON.parse(data) } catch { /* not json */ } resolve({ status: res.statusCode, headers: res.headers, json }) }) + }) + req.on('error', reject) + if (body) req.write(body) + req.end() + }) + return { call, close: () => new Promise(resolve => server.close(resolve)) } + } + + it('leaves the proxy open when no token is configured', async () => { + const app = await startApp('') + try { + assert.equal((await app.call()).status, 200) + assert.equal((await app.call({ path: '/public' })).status, 200) + } finally { + await app.close() + } + }) + + it('rejects a request without an Authorization header', async () => { + const app = await startApp(TOKEN) + try { + const res = await app.call() + assert.equal(res.status, 401) + assert.deepEqual(res.json, { error: { message: 'Invalid API key', type: 'invalid_request_error', code: 'invalid_api_key' } }) + } finally { + await app.close() + } + }) + + it('rejects a wrong token and a non-Bearer scheme', async () => { + const app = await startApp(TOKEN) + try { + assert.equal((await app.call({ headers: { authorization: 'Bearer wrong-token' } })).status, 401) + assert.equal((await app.call({ headers: { authorization: `Basic ${TOKEN}` } })).status, 401) + assert.equal((await app.call({ headers: { authorization: TOKEN } })).status, 401) + assert.equal((await app.call({ headers: { authorization: `Bearer ${TOKEN}x` } })).status, 401) + } finally { + await app.close() + } + }) + + it('accepts the configured bearer token on every /v1 route', async () => { + const app = await startApp(TOKEN) + try { + const models = await app.call({ headers: { authorization: `Bearer ${TOKEN}` } }) + assert.equal(models.status, 200) + assert.deepEqual(models.json, { object: 'list', data: [] }) + const chat = await app.call({ method: 'POST', path: '/v1/chat/completions', headers: { authorization: `bearer ${TOKEN}` }, body: { model: 'auto-fastest' } }) + assert.equal(chat.status, 200) + assert.equal(chat.json.model, 'auto-fastest') + } finally { + await app.close() + } + }) + + it('keeps the CORS preflight open', async () => { + const app = await startApp(TOKEN) + try { + const res = await app.call({ method: 'OPTIONS', headers: { origin: 'http://browser.example', 'access-control-request-method': 'POST' } }) + assert.equal(res.status, 204) + assert.equal(res.headers['access-control-allow-origin'], '*') + } finally { + await app.close() + } + }) + + it('does not touch routes outside /v1', async () => { + const app = await startApp(TOKEN) + try { + assert.equal((await app.call({ path: '/public' })).status, 200) + } finally { + await app.close() + } + }) + + it('reads the token from the environment, trimmed, and treats blank as off', () => { + assert.equal(getProxyToken({}), '') + assert.equal(getProxyToken({ [PROXY_TOKEN_ENV]: ' ' }), '') + assert.equal(getProxyToken({ [PROXY_TOKEN_ENV]: ' secret-token-value ' }), 'secret-token-value') + }) +})