Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Once started, modelrelay is accessible at `http://localhost:7352/`.
Router endpoint:

- Base URL: `http://127.0.0.1:7352/v1`
- API key: any string
- API key: any string — unless the proxy is protected with `MODELRELAY_PROXY_TOKEN`, in which case send that value (see [Security](https://schaetzkc.com/modelrelay/security/#proxy-token))
- Model: `auto-fastest` (router picks actual backend)

## 🚀 Install via Docker
Expand Down
1 change: 1 addition & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,6 @@ services:
# Recommended if the dashboard is reachable beyond this machine: require sign-in for the admin API.
# - MODELRELAY_ADMIN_TOKEN=change-me-to-16-or-more-random-characters
# - MODELRELAY_ALLOWED_HOSTS=modelrelay.example.com # only if you open the dashboard through a host name
# - MODELRELAY_PROXY_TOKEN=change-me-to-16-or-more-random-characters # require an API key on /v1
volumes:
modelrelay_config:
1 change: 1 addition & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ nav_order: 10

- Security and server settings (see [Security](security.md)):
- `MODELRELAY_ADMIN_TOKEN` (require sign-in for the admin API and dashboard)
- `MODELRELAY_PROXY_TOKEN` (require `Authorization: Bearer <token>` on the `/v1` proxy; unset leaves it open)
- `MODELRELAY_ALLOWED_HOSTS` (extra host names the admin API answers to; `*` disables the check)
- `MODELRELAY_ALLOWED_ORIGINS` (extra browser origins, for reverse proxies that rewrite `Host`)
- `MODELRELAY_HOST` (interface to listen on; same as `--host`)
Expand Down
4 changes: 4 additions & 0 deletions docs/endpoints.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ nav_order: 5

# Endpoints

## Authentication

`/v1` is open by default: any `Authorization` value is accepted and ignored. If the router is reachable by other people, set `MODELRELAY_PROXY_TOKEN`; callers must then send `Authorization: Bearer <token>`, which is the value they already put in their "API key" field. A missing or wrong credential gets `401` with an OpenAI-shaped `invalid_api_key` error. CORS preflights stay open. See [Security](security.md#proxy-token).

## `/v1/chat/completions`

`POST /v1/chat/completions` is an OpenAI-compatible chat completions endpoint.
Expand Down
27 changes: 22 additions & 5 deletions docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ modelrelay has two kinds of endpoints with different jobs, and they are protecte

| Endpoint | What it is | Protection |
|---|---|---|
| `/v1/*` | The OpenAI-compatible proxy your tools talk to | Open by design (any `Authorization` value is accepted and ignored). CORS is allowed so browser-based clients work. |
| `/v1/*` | The OpenAI-compatible proxy your tools talk to | Open by default; an optional bearer token (`MODELRELAY_PROXY_TOKEN`) requires callers to send it. CORS is allowed so browser-based clients work. |
| `/api/*` | The admin API behind the dashboard: settings, provider keys, tags, logs | Host and origin checks, plus an optional admin token. **No CORS.** |

The admin API can read and change your provider API keys, so it is the part to protect. Anyone who can use `/v1` can spend your free-tier quota, but only the admin API can see or redirect your keys.
Expand Down Expand Up @@ -41,7 +41,24 @@ modelrelay
```
- After 10 wrong guesses a client is locked out for 15 minutes.
- Use at least 16 random characters. The router warns at startup if the token is shorter.
- The token is read from the environment only; it is never written to the config file. The `/v1` proxy is not affected.
- The token is read from the environment only; it is never written to the config file. The `/v1` proxy is not affected by the admin token; see [Proxy token](#proxy-token).

## Proxy token

Anyone who can reach `/v1` can spend the quota of the providers you configured. On a laptop that is fine; on a VPS, a shared host or behind a public reverse proxy, set a proxy token:

```bash
export MODELRELAY_PROXY_TOKEN="$(openssl rand -hex 24)"
modelrelay
```

- Every `/v1` request must then send it as the API key: `Authorization: Bearer <token>`. Clients that already have an "API key" field use it unchanged — for modelrelay the value stops being ignored.
- Without the header, or with a wrong one, the proxy answers `401` with an OpenAI-shaped error (`invalid_api_key`). Comparison is constant-time.
- **Unset means open**, exactly as before, so existing local setups keep working.
- `OPTIONS` preflights are still answered without the token, because a browser must be able to ask before sending `Authorization`.
- The token is read from the environment only and never written to the config file. It is independent of `MODELRELAY_ADMIN_TOKEN`: the admin token protects the dashboard, this one protects the proxy.
- Use at least 16 random characters. The router warns at startup if the token is shorter. Changing it requires a restart.
- The startup banner shows the proxy as protected and tells clients which API key to use.

## Host names, reverse proxies and Docker

Expand All @@ -63,8 +80,8 @@ export MODELRELAY_ALLOWED_ORIGINS="https://modelrelay.example.com"
| Local use at `http://localhost:7352` | Nothing. |
| LAN access by IP address (`http://192.168.1.20:7352`) | Nothing for the checks. Set a token, because the dashboard is reachable by every device on the network. |
| LAN access by machine name (`http://mybox.local:7352`) | Nothing (the machine's own name is allowed). |
| Docker Compose with a published port | Nothing for `localhost` or IP access. Add `MODELRELAY_ADMIN_TOKEN` to the service's environment. |
| Behind a reverse proxy or Kubernetes ingress | Set `MODELRELAY_ALLOWED_HOSTS` (and `MODELRELAY_ALLOWED_ORIGINS` if the proxy changes `Host`), and a token. The `/v1` traffic from other services is not affected. |
| Docker Compose with a published port | Nothing for `localhost` or IP access. Add `MODELRELAY_ADMIN_TOKEN` to the service's environment, and `MODELRELAY_PROXY_TOKEN` if other hosts can reach the published port. |
| Behind a reverse proxy or Kubernetes ingress | Set `MODELRELAY_ALLOWED_HOSTS` (and `MODELRELAY_ALLOWED_ORIGINS` if the proxy changes `Host`), and an admin token. If the proxy is publicly reachable, also set `MODELRELAY_PROXY_TOKEN` so `/v1` callers must send it. |

Health checks that only open a TCP connection are not affected by any of this.

Expand All @@ -90,7 +107,7 @@ The default has **not** been changed, because making loopback the default would
| The Docker image | Would have to set `MODELRELAY_HOST=0.0.0.0` for published ports to work. |
| Kubernetes | Same as Docker. |

Recommendation: if everything runs on one machine, use `--host 127.0.0.1`. Otherwise keep the default and set an admin token.
Recommendation: if everything runs on one machine, use `--host 127.0.0.1`. Otherwise keep the default, set an admin token, and set a proxy token if `/v1` is reachable by people you do not trust.

## Reporting a problem

Expand Down
4 changes: 4 additions & 0 deletions docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ nav_order: 11

# Troubleshooting

## `/v1` returns `401 invalid_api_key`

`MODELRELAY_PROXY_TOKEN` is set, so the proxy requires that token as the API key. Send it as `Authorization: Bearer <token>` and use the same value wherever the client asks for an API key. Unset the variable (and restart) to make the proxy open again. Check the startup banner: it says when the proxy is protected. See [Security](security.md#proxy-token).

## Clicking the update button or running `modelrelay` won't perform an update

To trigger a manual npm update and restart the service, run:
Expand Down
57 changes: 57 additions & 0 deletions lib/proxy-guard.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
// Protection for the OpenAI-compatible proxy (/v1/*): an optional bearer token.
//
// The proxy is open by default because it is usually reached over loopback by local tools, and its CORS headers
// are what browser-based clients rely on. When the router is published beyond this machine (a VPS, a reverse
// proxy, a shared host), anyone who knows the URL can spend the quota configured for the providers. Setting
// MODELRELAY_PROXY_TOKEN requires `Authorization: Bearer <token>` on every /v1 request; without it nothing
// changes, so existing clients keep working.
//
// CORS preflights stay open: a browser must be able to ask for permission before it sends the header.

import { createHmac, randomBytes, timingSafeEqual } from 'node:crypto';

export const PROXY_TOKEN_ENV = 'MODELRELAY_PROXY_TOKEN';
export const MIN_PROXY_TOKEN_LENGTH = 16;

/** Reads the proxy token from the environment, trimmed. Empty means the proxy stays open. */
export function getProxyToken(env = process.env) {
const raw = env?.[PROXY_TOKEN_ENV];
return typeof raw === 'string' ? raw.trim() : '';
}

/** Returns the credential of an `Authorization: Bearer <token>` header, or '' for any other shape. */
export function parseBearerToken(header) {
if (typeof header !== 'string') return '';
const match = /^Bearer[ \t]+(\S+)$/i.exec(header.trim());
return match ? match[1] : '';
}

/**
* Builds the token check. With no token the proxy is open; with one, only an exact bearer credential passes.
* Comparison runs over HMACs under a random per-process key, so it is constant-time and does not leak the
* token length.
*/
export function createProxyAuth({ token } = {}) {
const normalized = typeof token === 'string' ? token.trim() : '';
const enabled = normalized.length > 0;
const macKey = randomBytes(32);
const mac = (value) => createHmac('sha256', macKey).update(String(value)).digest();
const expected = enabled ? mac(normalized) : null;

const verify = (header) => {
if (!enabled) return true;
const candidate = parseBearerToken(header);
return candidate.length > 0 && timingSafeEqual(mac(candidate), expected);
};

return { enabled, verify };
}

/** Express middleware for /v1. Mount it after the CORS middleware so preflights are answered first. */
export function createProxyMiddleware({ token } = {}) {
const auth = createProxyAuth({ token });
return (req, res, next) => {
if (!auth.enabled || req.method === 'OPTIONS' || auth.verify(req.headers.authorization)) return next();
return res.status(401).json({ error: { message: 'Invalid API key', type: 'invalid_request_error', code: 'invalid_api_key' } });
};
}
13 changes: 12 additions & 1 deletion lib/server.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import { getPreferredLanIpv4Address } from './network.js';
import { capLength, isSafeObjectKey } from './text.js';
import { rateLimit } from 'express-rate-limit';
import { createAdminAuth, createAdminMiddleware, createAuthRoutes, parseAllowedHosts, parseAllowedOrigins } from './admin-guard.js';
import { createProxyAuth, createProxyMiddleware, getProxyToken, MIN_PROXY_TOKEN_LENGTH } from './proxy-guard.js';
import { classifyFailureStatus, createTelemetryStore, normalizeTelemetryStore, recordFailure, recordSuccess, reliabilityMultiplier, summarizeTelemetry, summarizeTelemetryEntry } from './telemetry.js';
import { createHash, randomInt, randomUUID } from 'crypto';
import { getAutostartStatus } from './autostart.js';
Expand Down Expand Up @@ -3625,6 +3626,12 @@ export async function runServer(config, port, enableLog = true, bannedModels = [
next();
});

// Optional bearer token for the proxy (see lib/proxy-guard.js). Off unless MODELRELAY_PROXY_TOKEN is set, so
// local clients keep working; when set, every /v1 request must send it as the API key.
const proxyToken = getProxyToken();
const proxyAuth = createProxyAuth({ token: proxyToken });
app.use('/v1', createProxyMiddleware({ token: proxyToken }));

app.get('/api/auth/status', authRoutes.status);
app.post('/api/auth/login', loginLimiter, authRoutes.login);
app.post('/api/auth/logout', authRoutes.logout);
Expand Down Expand Up @@ -5165,13 +5172,17 @@ export async function runServer(config, port, enableLog = true, bannedModels = [
console.log(chalk.yellow(' ⚠️ The admin API (which can read your provider API keys) is reachable from other devices on your network without a token.'));
console.log(chalk.yellow(' Set MODELRELAY_ADMIN_TOKEN to require sign-in, or start with --host 127.0.0.1 to keep it on this machine.'));
}
if (proxyAuth.enabled) {
console.log(chalk.green(' 🔒 Proxy API protected by MODELRELAY_PROXY_TOKEN (send it as the API key).'));
if (proxyToken.length < MIN_PROXY_TOKEN_LENGTH) console.log(chalk.yellow(' ⚠️ The proxy token is short. Use at least 16 random characters.'));
}
if (lanIp) {
console.log(chalk.green(` ✅ Visit ${chalk.bold(`http://${lanIp}:${port}`)} to access the Web UI from another computer on your network.`));
}
console.log(chalk.green(` ✅ Router proxy active at ${chalk.bold(`http://localhost:${port}/v1`)}`));
console.log(chalk.dim(` Usage in OpenCode/Cursor:`));
console.log(chalk.dim(` - Provider Base URL: http://localhost:${port}/v1`));
console.log(chalk.dim(` - API Key: (anything, ignored)`));
console.log(chalk.dim(` - API Key: ${proxyAuth.enabled ? 'the MODELRELAY_PROXY_TOKEN value' : '(anything, ignored)'}`));
console.log(chalk.dim(` - Model: (anything, ignored)`));
console.log();
});
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@schaetzkc/modelrelay",
"version": "1.29.0",
"version": "1.30.0",
"description": "OpenAI-compatible local router that benchmarks free coding models across providers and forwards requests to the best available model.",
"keywords": [
"nvidia",
Expand Down
108 changes: 108 additions & 0 deletions test/test.js
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ import { classifyFailureStatus, createTelemetryStore, getReliability, normalizeT
import http from 'node:http'
import express from 'express'
import { SESSION_COOKIE, buildClearedCookie, buildSessionCookie, createAdminAuth, createAdminMiddleware, createAuthRoutes, evaluateRequestOrigin, isAllowedHost, parseAllowedHosts, parseAllowedOrigins, parseCookies, parseHostHeader } from '../lib/admin-guard.js'
import { PROXY_TOKEN_ENV, createProxyMiddleware, getProxyToken } from '../lib/proxy-guard.js'
import '../public/model-search.js'
import { capLength, isSafeObjectKey, stripTrailingSuffixes, trimEndWhile, trimStartWhile, trimWhile } from '../lib/text.js'
import { OPENCLAW_ROUTING_PRESETS, applyOpenClawConfig, buildOpenClawProviderConfig } from '../lib/onboard.js'
Expand Down Expand Up @@ -4298,3 +4299,110 @@ describe('OpenCode free-tier SSE stream (stateful) split-chunk', () => {
})

})

describe('proxy token: end to end over HTTP', () => {
const TOKEN = 'proxy-token-for-tests-0123456789'

const startApp = async (proxyToken) => {
const app = express()
app.use('/v1', (req, res, next) => {
res.header('Access-Control-Allow-Origin', '*')
res.header('Access-Control-Allow-Headers', 'Content-Type, Authorization')
if (req.method === 'OPTIONS') return res.sendStatus(204)
next()
})
app.use('/v1', createProxyMiddleware({ token: proxyToken }))
app.use(express.json())
app.get('/v1/models', (req, res) => res.json({ object: 'list', data: [] }))
app.post('/v1/chat/completions', (req, res) => res.json({ ok: true, model: req.body?.model ?? null }))
app.get('/public', (req, res) => res.json({ ok: true }))
const server = await new Promise(resolve => { const s = app.listen(0, '127.0.0.1', () => resolve(s)) })
const port = server.address().port
const call = (options = {}) => new Promise((resolve, reject) => {
const body = options.body === undefined ? undefined : JSON.stringify(options.body)
const req = http.request({ host: '127.0.0.1', port, method: options.method || 'GET', path: options.path || '/v1/models',
headers: { host: `localhost:${port}`, ...(body ? { 'content-type': 'application/json', 'content-length': Buffer.byteLength(body) } : {}), ...(options.headers || {}) } }, (res) => {
let data = ''
res.on('data', c => { data += c })
res.on('end', () => { let json = null; try { json = JSON.parse(data) } catch { /* not json */ } resolve({ status: res.statusCode, headers: res.headers, json }) })
})
req.on('error', reject)
if (body) req.write(body)
req.end()
})
return { call, close: () => new Promise(resolve => server.close(resolve)) }
}

it('leaves the proxy open when no token is configured', async () => {
const app = await startApp('')
try {
assert.equal((await app.call()).status, 200)
assert.equal((await app.call({ path: '/public' })).status, 200)
} finally {
await app.close()
}
})

it('rejects a request without an Authorization header', async () => {
const app = await startApp(TOKEN)
try {
const res = await app.call()
assert.equal(res.status, 401)
assert.deepEqual(res.json, { error: { message: 'Invalid API key', type: 'invalid_request_error', code: 'invalid_api_key' } })
} finally {
await app.close()
}
})

it('rejects a wrong token and a non-Bearer scheme', async () => {
const app = await startApp(TOKEN)
try {
assert.equal((await app.call({ headers: { authorization: 'Bearer wrong-token' } })).status, 401)
assert.equal((await app.call({ headers: { authorization: `Basic ${TOKEN}` } })).status, 401)
assert.equal((await app.call({ headers: { authorization: TOKEN } })).status, 401)
assert.equal((await app.call({ headers: { authorization: `Bearer ${TOKEN}x` } })).status, 401)
} finally {
await app.close()
}
})

it('accepts the configured bearer token on every /v1 route', async () => {
const app = await startApp(TOKEN)
try {
const models = await app.call({ headers: { authorization: `Bearer ${TOKEN}` } })
assert.equal(models.status, 200)
assert.deepEqual(models.json, { object: 'list', data: [] })
const chat = await app.call({ method: 'POST', path: '/v1/chat/completions', headers: { authorization: `bearer ${TOKEN}` }, body: { model: 'auto-fastest' } })
assert.equal(chat.status, 200)
assert.equal(chat.json.model, 'auto-fastest')
} finally {
await app.close()
}
})

it('keeps the CORS preflight open', async () => {
const app = await startApp(TOKEN)
try {
const res = await app.call({ method: 'OPTIONS', headers: { origin: 'http://browser.example', 'access-control-request-method': 'POST' } })
assert.equal(res.status, 204)
assert.equal(res.headers['access-control-allow-origin'], '*')
} finally {
await app.close()
}
})

it('does not touch routes outside /v1', async () => {
const app = await startApp(TOKEN)
try {
assert.equal((await app.call({ path: '/public' })).status, 200)
} finally {
await app.close()
}
})

it('reads the token from the environment, trimmed, and treats blank as off', () => {
assert.equal(getProxyToken({}), '')
assert.equal(getProxyToken({ [PROXY_TOKEN_ENV]: ' ' }), '')
assert.equal(getProxyToken({ [PROXY_TOKEN_ENV]: ' secret-token-value ' }), 'secret-token-value')
})
})