Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions site/docs/bins.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,18 +87,33 @@ A failed ping sends HTTP 503:
"connected": false,
"uriPresent": true,
"scheme": "mongodb+srv",
"uri": {
"length": 78,
"leadingWhitespace": false,
"trailingWhitespace": false,
"newline": false,
"wrappedQuotes": false,
"atCount": 2,
"host": "cluster0.example.invalid",
"username": "user",
"passwordLength": 8,
"passwordReserved": ["@"],
"passwordHasPercent": false
},
"error": {
"name": "MongoParseError",
"message": "Invalid connection string. Percent-encode reserved characters in the username and password (@ : / ? # [ ]).",
"message": "Invalid connection string. Percent-encode reserved characters in the username and password (@ : / ? # [ ] $).",
"code": null
}
}
}
```

The response does not contain the connection string, the username, the password, or the host. The logs do not contain them. `uriPresent` is a boolean. `scheme` is `mongodb`, `mongodb+srv`, or `null`.
A MongoDB result includes `bins.uri` on success and on failure. A memory result omits `bins.uri`. `uri.host` is the text after the last `@` before the first `/` or `?`. An `@` in the query string does not change the host. `uri.username` is the username. `uri.passwordLength` is the password length.

`Invalid connection string` means that the URI has a second `@`. It also means that you did not encode a reserved character in the username or the password. Encode these characters:
`uri.passwordReserved` lists raw reserved characters in the password. `uri.passwordLength` is null when the password has only reserved characters. `uri.passwordHasPercent` is true when the password contains `%`. The response does not contain the connection string or the password. The logs do not contain the connection string, the username, the password, or the host. `uriPresent` is a boolean, and `scheme` is `mongodb`, `mongodb+srv`, or `null`.

`Invalid connection string` is the driver text. MockHTTP adds the percent-encode hint only for that text or for `unescaped characters`. A DNS error stays a DNS error. `Invalid connection string` means that the URI has a second `@`. It also means that you did not encode a reserved character in the username or the password. Encode these characters:

- `@` as `%40`
- `:` as `%3A`
Expand All @@ -107,6 +122,7 @@ The response does not contain the connection string, the username, the password,
- `#` as `%23`
- `[` as `%5B`
- `]` as `%5D`
- `$` as `%24`

A correct `mongodb+srv://` URI does not fail at parse time. A DNS failure or a network failure has a different error name.

Expand Down
6 changes: 5 additions & 1 deletion site/docs/how-to-deploy/cloudflare.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,11 @@ Bins stay in memory in the isolate when `MONGODB_URI` is not set. A bin from one

Percent-encode reserved characters in the username and the password. `@` is `%40`. An unencoded `@` makes the driver throw `Invalid connection string` before it contacts Atlas.

`GET /healthz` reports the bin store. A successful admin ping returns HTTP 200 and `bins.connected: true`. A failed ping returns HTTP 503 with the driver error name, a redacted message, and a code. The body has `uriPresent` (boolean) and `scheme` (`mongodb` or `mongodb+srv`). The body does not have the URI, the username, the password, or the host. On the Node server, this route reports the memory store. Connection failures are also one JSON line in the Worker logs (`mockhttp.bin-store`). Those logs use the same redaction.
`GET /healthz` reports the bin store. A successful admin ping returns HTTP 200 and `bins.connected: true`. A failed ping returns HTTP 503 with the driver error name, a redacted message, and a code. The body has `uriPresent` (boolean) and `scheme` (`mongodb` or `mongodb+srv`). On the Node server, this route reports the memory store.

When `MONGODB_URI` is set, `bins.uri` has the URI length and the `@` count. It also has flags for whitespace, a newline, and quote marks. It has the host, the username, and the password length. The host is the text after the last `@` before `/` or `?`. An `@` in the query string does not change the host. The body does not have the password or the full URI.

The percent-encode hint is added only for `Invalid connection string` or `unescaped characters`. A DNS error keeps the driver text. `uri.passwordLength` is null when the password has only reserved characters. Worker logs still omit the URI, the username, the password, and the host.

Taps are a library API (`mock.taps.inject`). They have no HTTP routes, so a deployed Worker has no way to inject one.

Expand Down
158 changes: 145 additions & 13 deletions src/bin-store-diagnostics.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
/**
* Diagnostics for the bin store.
* Logs and HTTP bodies can show the error name and a safe message.
* They can show if the URI is present.
* They cannot show the URI, the username, the password, or the host.
* Logs can show the error name, a safe message, and URI shape flags.
* Logs cannot show the URI, the username, the password, or the host.
* `GET /healthz` can show the host, the username, and the password length.
* It cannot show the password or the full URI.
*/

export type BinStoreScheme = "mongodb" | "mongodb+srv";
Expand All @@ -22,6 +23,7 @@ export type BinHealth =
latencyMs: number;
uriPresent: boolean;
scheme: BinStoreScheme | null;
uri?: UriDiagnostics;
};
}
| {
Expand All @@ -31,6 +33,7 @@ export type BinHealth =
connected: false;
uriPresent: boolean;
scheme: BinStoreScheme | null;
uri?: UriDiagnostics;
error: BinStoreErrorInfo;
};
};
Expand All @@ -41,8 +44,25 @@ export type UriShape = {
extraAt: boolean;
};

/** Redacted URI facts for `GET /healthz`. No password and no full URI. */
export type UriDiagnostics = {
length: number;
leadingWhitespace: boolean;
trailingWhitespace: boolean;
newline: boolean;
wrappedQuotes: boolean;
atCount: number;
host: string | null;
username: string | null;
passwordLength: number | null;
passwordReserved: string[];
passwordHasPercent: boolean;
};

const PASSWORD_RESERVED = ["@", ":", "/", "?", "#", "[", "]", "$"] as const;

const ENCODING_HINT =
"Percent-encode reserved characters in the username and password (@ : / ? # [ ]).";
"Percent-encode reserved characters in the username and password (@ : / ? # [ ] $).";

const loggedPhases = new WeakMap<object, Set<string>>();

Expand Down Expand Up @@ -98,6 +118,118 @@ export function uriShape(uri: string): UriShape {
};
}

function hasWrappedQuotes(uri: string): boolean {
const trimmed = uri.trim();
return (
trimmed.startsWith('"') ||
trimmed.startsWith("'") ||
trimmed.endsWith('"') ||
trimmed.endsWith("'")
);
}

function looksLikeHost(value: string): boolean {
if (value === "localhost" || /^localhost:\d+$/.test(value)) {
return true;
}
if (/^\d{1,3}(?:\.\d{1,3}){3}(?::\d+)?$/.test(value)) {
return true;
}
if (/^\[[0-9a-fA-F:.]+\](?::\d+)?$/.test(value)) {
return true;
}
const name = value.replace(/:\d+$/, "");
return name.includes(".") && /^[A-Za-z0-9._-]+$/.test(name);
}

/**
* Keep the authority section. Stop at the `/` or `?` that follows the host.
* An `@` in the query string or the path is outside that section.
*/
function authorityBody(body: string): string {
const firstAt = body.indexOf("@");
if (firstAt < 0) {
return body;
}
for (let i = firstAt + 1; i < body.length; i++) {
const mark = body[i];
if (mark !== "/" && mark !== "?") {
continue;
}
const at = body.lastIndexOf("@", i - 1);
const between = body.slice(at + 1, i);
if (looksLikeHost(between)) {
return body.slice(0, i);
}
}
return body;
}

function userinfoAndHost(uri: string): {
userinfo: string | null;
host: string | null;
} {
const cleaned = uri.trim().replace(/^['"]+|['"]+$/g, "");
const scheme = cleaned.match(/^[a-z][a-z0-9+.-]*:\/\//i);
const body = scheme ? cleaned.slice(scheme[0].length) : cleaned;
const authority = authorityBody(body);
const at = authority.lastIndexOf("@");
if (at < 0) {
return { userinfo: null, host: null };
}
const host = authority.slice(at + 1);
return {
userinfo: authority.slice(0, at),
host: host.length > 0 ? host : null,
};
}

/**
* Host is the text after the last `@` in the authority section.
* The search stops at the first `/` or `?` after that host.
* Username is the text before the first `:` in the userinfo.
* The password is not returned.
* When every password character is reserved, the length and the list stay empty.
* Those two values rebuild the password.
*/
export function uriDiagnostics(uri: string): UriDiagnostics {
const parts = userinfoAndHost(uri);
let username: string | null = null;
let password: string | null = null;
if (parts.userinfo !== null) {
const colon = parts.userinfo.indexOf(":");
if (colon < 0) {
username = parts.userinfo.length > 0 ? parts.userinfo : null;
} else {
username = parts.userinfo.slice(0, colon) || null;
password = parts.userinfo.slice(colon + 1);
}
}
const secret = password;
const onlyReserved =
secret !== null &&
secret.length > 0 &&
[...secret].every((char) =>
(PASSWORD_RESERVED as readonly string[]).includes(char),
);
return {
length: uri.length,
leadingWhitespace: /^\s/.test(uri),
trailingWhitespace: /\s$/.test(uri),
newline: /[\r\n]/.test(uri),
wrappedQuotes: hasWrappedQuotes(uri),
atCount: uri.split("@").length - 1,
host: parts.host,
username,
passwordLength: secret === null || onlyReserved ? null : secret.length,
passwordReserved:
Comment on lines +221 to +225

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid password reconstruction from public diagnostics

For mongodb://user:@@@@@host, the public response gives atCount: 5, passwordLength: 4, and passwordReserved: ["@"]. An unauthenticated caller can therefore reconstruct the complete @@@@ password, although this change states that the endpoint never returns it.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When every password character is reserved, passwordLength and passwordReserved stay empty. The body does not rebuild that password. atCount still counts each @. Commit 8c26806.

secret === null || onlyReserved
? []
: PASSWORD_RESERVED.filter((char) => secret.includes(char)),
passwordHasPercent: secret?.includes("%") === true,
};
}

export function isBinStoreConnectionFailure(error: unknown): boolean {
if (!(error instanceof Error)) {
return false;
Expand Down Expand Up @@ -174,13 +306,14 @@ export function redactSecrets(message: string, uri?: string): string {
return out;
}

function withEncodingHint(
message: string,
shape: UriShape | undefined,
): string {
const needsHint =
shape?.extraAt === true ||
/invalid connection string|unescaped characters/i.test(message);
/**
* Add the encode hint only when the driver message is already a parse error.
* A DNS error or a scheme error keeps its own text.
*/
function withEncodingHint(message: string): string {
const needsHint = /invalid connection string|unescaped characters/i.test(
message,
);
if (!needsHint || message.includes("Percent-encode")) {
return message;
}
Expand All @@ -205,10 +338,9 @@ export function redactStoreError(
typeof codeValue === "string" || typeof codeValue === "number"
? codeValue
: null;
const shape = uri ? uriShape(uri) : undefined;
return {
name,
message: withEncodingHint(redactSecrets(raw, uri), shape),
message: withEncodingHint(redactSecrets(raw, uri)),
code,
};
}
Expand Down
33 changes: 32 additions & 1 deletion src/routes/health.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,17 +37,48 @@ export async function checkMemoryBinStore(keyv: {
}
}

const uriDiagnosticsSchema = {
type: "object",
properties: {
length: { type: "integer" },
leadingWhitespace: { type: "boolean" },
trailingWhitespace: { type: "boolean" },
newline: { type: "boolean" },
wrappedQuotes: { type: "boolean" },
atCount: { type: "integer" },
host: { type: ["string", "null"] },
username: { type: ["string", "null"] },
passwordLength: { type: ["integer", "null"] },
passwordReserved: { type: "array", items: { type: "string" } },
passwordHasPercent: { type: "boolean" },
},
required: [
"length",
"leadingWhitespace",
"trailingWhitespace",
"newline",
"wrappedQuotes",
"atCount",
"host",
"username",
"passwordLength",
"passwordReserved",
"passwordHasPercent",
],
};

const binHealthFields = {
store: { type: "string", enum: ["mongo", "memory"] },
connected: { type: "boolean" },
latencyMs: { type: "integer" },
uriPresent: { type: "boolean" },
scheme: { type: ["string", "null"], enum: ["mongodb", "mongodb+srv", null] },
uri: uriDiagnosticsSchema,
};

const healthSchema: FastifySchema = {
description:
"Report whether bin storage is reachable. MongoDB failures include the driver error with credentials removed. The URI, username, password, and host are never returned.",
"Report whether bin storage is reachable. MongoDB failures include the driver error. The password and the full URI are never returned. A MongoDB result can include the host, the username, the URI length, and the password length.",
tags: ["Health"],
response: {
200: {
Expand Down
Loading
Loading