feat(spec): treat nested datasource-config credential positions identically to the top-level keys they mirror (#13604)

A credential under the very spelling the top level refuses and redacts -
one object level down (options.auth.token, options.pool.password,
tunnel.password on a contract-less driver) - was accepted at publish and
served by every datasource read door in cleartext with
redactedConfigKeys: []. The nested judgment was a hand-enumerated
per-driver path table on the read side and absent on the write side,
while the top level was derived from the driver contract.

Both sides now consume one derivation: the canonical spellings and
former aliases move to driver/common.zod.ts (CREDENTIAL_KEY_SPELLINGS,
the bottom of the import graph); the read scrub applies the name
judgment and the URL composite at every object depth for every driver;
the write door's passthrough walk refuses the same spellings at any
depth; refusedCredentialPaths walks nested object shapes for z.never
leaves; arrays stay off the walk on both doors (row-shaped data is not
config). passthroughSecretPaths remains only as the client-measured
residue. restoreRedactedConfig is now derived from the redactor's own
redactedPaths, so every current and future redaction source is mirrored
on the untouched-Save round trip by construction.

Semantic migration entry
datasource-config-options-nested-credential-spelling-refused (major 18)
carries the authored-artifact upgrade.


Claude-Session: https://claude.ai/code/session_01PBjwYLS6BciTQW3c9xQiD2

Co-authored-by: Claude <noreply@anthropic.com>

Author: os-warren

.changeset/nested-credential-redaction-class.md

@@ -0,0 +1,59 @@
+---
+'@objectstack/spec': minor
+'@objectstack/service-datasource': patch
+---
+
+feat(spec): treat a nested datasource-config credential position identically to the top-level key it mirrors — derived at every depth, on both doors (#13405)
+
+**BREAKING** accept-set narrowing, the nested closure of the #9040 family. A
+credential under the very spelling the top level refuses and redacts — one
+object level down (`options.auth.token`, `options.pool.password`, a
+`tunnel.password` on a contract-less driver) — was accepted at publish and
+served back by every datasource read door in cleartext with
+`redactedConfigKeys: []`, because the read side's nested judgment was only the
+hand-enumerated `passthroughSecretPaths` table and the write side had no nested
+name judgment at all.
+
+Both sides are now derived from ONE source instead of hand-maintained:
+
+- The canonical credential spellings and former aliases moved to
+  `driver/common.zod.ts` (`CREDENTIAL_KEY_SPELLINGS`) — the bottom of the
+  driver-schema import graph — so the write door's passthrough walk and the
+  read redactor consume the same list (#8300's no-second-copy posture applied
+  to the list itself).
+- **Read door** (`redactDatasourceConfig`, behind both consumers — the
+  datasource-admin routes and the kernel per-type redaction hook): the
+  credential-name judgment and the URL composite (userinfo + query params) now
+  run at EVERY object depth, for every driver, contract-less included. Nested
+  removals are reported as dotted paths in `redactedKeys`, plus a new
+  `redactedPaths` field carrying exact segments. `passthroughSecretPaths`
+  remains only as the residue it always should have been: CLIENT-MEASURED
+  secret spellings (`proxyPassword`, `key`, `passphrase`, …) that mirror no
+  top-level key.
+- **Write door** (`credentialFreeMongoOptions`): a non-empty string under a
+  credential-spelled key is refused at any object depth of the mongodb
+  `options` passthrough, with a prescription that does not inherit the
+  `auth.password`-only "wins over" reassurance. The measured `auth.password`
+  refusal keeps its own message; nothing is double-reported.
+- **Schema derivation walked at depth**: `refusedCredentialPaths` /
+  `refusedCredentialPathsOfSchema` extend the `z.never()` derivation below the
+  top level, so a driver contract that refuses a key inside a nested object
+  shape is covered the day it lands (none exists today — pinned per driver).
+- **Arrays are off the walk** on both doors — the same structural line
+  `valueAtPath`/`withoutPath` already drew — so row-shaped data (memory's
+  `initialData` seeds) keeps its own fields without a per-driver exclusion
+  list.
+- `restoreRedactedConfig` (service-datasource) is now DERIVED from the
+  redactor instead of mirroring it rule by rule: it grafts stored material
+  back wherever the patch is indistinguishable from what the read path served,
+  so an untouched "Save" on an affected legacy row keeps its stored material
+  for every current and future redaction source, and an author's edit always
+  wins. The metadata write door's generic `carryForwardRedactedValues` already
+  walks the dotted paths and needs no change.
+
+Semantic migration entry
+`datasource-config-options-nested-credential-spelling-refused` (protocol major
+18) carries the authored-artifact upgrade: remove the nested key, or bind the
+real secret through `external.credentialsRef` / the connection form.
+
+<!-- adr-0087: registered datasource-config-options-nested-credential-spelling-refused -->

content/docs/references/data/driver-mongo.mdx

@@ -48,7 +48,7 @@ MongoDB Connection Configuration
 | **username** | `string` | optional | Authentication user |
 | **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
 | **authSource** | `string` | optional | Authentication database |
-| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Only `auth.password` is refused inline — bind it via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |
+| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Credential-spelled keys (`password`, `authToken`, and their former aliases) are refused inline at any depth — bind the secret via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |
 
 
 ---

packages/services/service-datasource/src/__tests__/datasource-config-redaction.test.ts

@@ -475,3 +475,78 @@ describe('#9040 — the passthrough spelling, both halves at the service door',
     ).rejects.toThrow(/options\.auth\.password/);
   });
 });
+
+describe('nested credential positions OFF the passthrough table — the class control, at this door', () => {
+  /**
+   * The nested-position finding, at the admin door: a credential spelling one
+   * object level down from the key the top level hides — deliberately NOT a
+   * `passthroughSecretPaths` row — used to be served by `getDatasource()` in
+   * cleartext with `redactedConfigKeys: []`, and accepted by the write gate in
+   * silence (measured on the pre-fix build). Regression cases against table
+   * rows were already green and prove nothing about this class.
+   */
+  const OFF_TABLE_MONGO: StoredDatasource = {
+    name: 'off_table_mongo',
+    driver: 'mongodb',
+    origin: 'runtime',
+    config: {
+      database: 'events',
+      options: { replicaSet: 'rs0', auth: { username: 'app', token: 'eyJhbGci.OFFTABLE.y' } },
+    },
+  };
+
+  it('read path: the off-table nested credential does not reach the caller, and is named', async () => {
+    const { service } = makeService([OFF_TABLE_MONGO]);
+    const read = await service.getDatasource('off_table_mongo');
+    expect(JSON.stringify(read!.config)).not.toContain('OFFTABLE');
+    expect(read!.config!.options).toEqual({ replicaSet: 'rs0', auth: { username: 'app' } });
+    expect(read!.redactedConfigKeys).toContain('options.auth.token');
+  });
+
+  it('an untouched round-trip keeps the stored off-table credential — the restore mirrors the recursion', async () => {
+    const { service, records } = makeService([OFF_TABLE_MONGO]);
+    const read = await service.getDatasource('off_table_mongo');
+    await service.updateDatasource('off_table_mongo', { config: read!.config, label: 'Renamed' });
+    expect(records[0].label).toBe('Renamed');
+    expect((records[0].config!.options as any).auth).toEqual({
+      username: 'app',
+      token: 'eyJhbGci.OFFTABLE.y',
+    });
+  });
+
+  it('a TYPED-IN off-table nested credential is refused at the write gate on its own merits', async () => {
+    const { service } = makeService([OFF_TABLE_MONGO]);
+    await expect(
+      service.updateDatasource('off_table_mongo', {
+        config: {
+          database: 'events',
+          options: { auth: { username: 'app', token: 'typed-new-secret' } },
+        },
+      }),
+    ).rejects.toThrow(/options\.auth\.token/);
+  });
+
+  it('a NESTED URL string is redacted and restored like a top-level one — contract-less driver included', async () => {
+    const stored: StoredDatasource = {
+      name: 'vendor_ds',
+      driver: 'com.vendor.custom',
+      origin: 'runtime',
+      config: { endpoint: 'x', replication: { url: 'postgresql://svc:hunter2@replica/db' } },
+    };
+    const { service, records } = makeService([stored]);
+    const read = await service.getDatasource('vendor_ds');
+    expect((read!.config!.replication as any).url).toBe('postgresql://svc@replica/db');
+    expect(read!.redactedConfigKeys).toContain('replication.url');
+    // Untouched round-trip: the served (redacted) URL is indistinguishable
+    // from the stored one once redacted, so the stored value is carried back.
+    await service.updateDatasource('vendor_ds', { config: read!.config });
+    expect((records[0].config!.replication as any).url).toBe('postgresql://svc:hunter2@replica/db');
+    // An author who rewrites the nested URL by hand still WINS.
+    const edited = {
+      ...read!.config,
+      replication: { url: 'postgresql://svc@other-replica/db' },
+    };
+    await service.updateDatasource('vendor_ds', { config: edited });
+    expect((records[0].config!.replication as any).url).toBe('postgresql://svc@other-replica/db');
+  });
+});

packages/services/service-datasource/src/datasource-config-redaction.ts

@@ -31,11 +31,7 @@
  * #8154's, deliberately not built here.
  */
 
-import {
-  passthroughSecretPaths,
-  redactableConfigKeys,
-  redactUrlCredentials,
-} from '@objectstack/spec/data';
+import { redactDatasourceConfig } from '@objectstack/spec/data';
 
 export {
   refusedCredentialKeys,
@@ -56,6 +52,24 @@ export {
  * where the patch is indistinguishable from what the read path served — an
  * absent key, or a URL that matches the stored URL once redacted. Anything the
  * author actually changed wins, including clearing a URL's password by hand.
+ * A patch whose CONTAINER for a nested leaf is removed is the author's word
+ * too (they deleted the block), so nothing is grafted there.
+ *
+ * ## Derived from the redactor, not restated beside it
+ *
+ * This function used to mirror the read path rule by rule — one loop per
+ * redaction source, each a copy that could silently fall behind (the docblock
+ * threat on every one of them: "a redaction the restore side did not mirror
+ * turns an untouched Save into silent credential deletion"). The nested-
+ * position fix made the read path recursive, which would have added two more
+ * loops — so the mirroring is now structural instead: compute what the read
+ * path SERVES for the stored row (`redactDatasourceConfig(driver, stored)`),
+ * and for every redacted path graft the stored value back exactly where the
+ * patch still matches the served projection. A future redaction source is
+ * mirrored here by construction, with nothing to forget. (Same inversion the
+ * metadata door's generic `carryForwardRedactedValues` performs; this one
+ * consumes the redactor's exact `redactedPaths` segments, so a stored key
+ * with a literal dot cannot be mis-split.)
  *
  * What this does NOT do is let a patch set a refused key: `assertValidConfig`
  * still runs on the merged record, so a caller that types `password` into the
@@ -69,43 +83,25 @@ export function restoreRedactedConfig(
   if (!patch || typeof patch !== 'object') return patch;
   if (!stored || typeof stored !== 'object') return patch;
 
-  const hidden = new Set(redactableConfigKeys(driver));
+  const served = redactDatasourceConfig(driver, stored);
   const out: Record<string, unknown> = { ...patch };
 
-  for (const key of hidden) {
-    // Only when the patch does not speak to the key at all. A patch that DOES
-    // carry it is the author's word, and (for a refused spelling) is about to
-    // be refused on its own merits rather than quietly overwritten here.
-    if (!(key in out) && stored[key] !== undefined) out[key] = stored[key];
-  }
-
-  for (const [key, storedValue] of Object.entries(stored)) {
-    if (hidden.has(key) || typeof storedValue !== 'string') continue;
-    // The SAME composite the read path applies (userinfo password + #8337
-    // credential query parameters) — a redaction this compare did not mirror
-    // would make the untouched "Save" it exists for delete the credential.
-    const redactedStored = redactUrlCredentials(storedValue);
-    // Unchanged by redaction ⇒ it carried no credential ⇒ nothing to restore.
-    if (redactedStored === storedValue) continue;
-    if (out[key] === redactedStored) out[key] = storedValue;
-  }
-
-  // The passthrough spellings (#9040) — the nested material the read path
-  // drops by PATH (`options.auth.password`, `options.proxyPassword`, …). The
-  // same narrow rule as the top-level keys, translated per leaf: restore ONLY
-  // when the patch's container for the leaf exists but does not speak to the
-  // leaf at all — exactly what the read path served. A patch carrying the leaf
-  // is the author's word (a typed-in `auth.password` is then refused by the
-  // #9040 write gate on its own merits); a patch with the CONTAINER removed is
-  // the author's word too (they deleted the block), so nothing is grafted.
-  for (const path of passthroughSecretPaths(driver)) {
+  for (const path of served.redactedPaths) {
     const storedLeaf = valueAt(stored, path);
     if (storedLeaf === undefined) continue;
     const parentPath = path.slice(0, -1);
     const leafKey = path[path.length - 1] as string;
-    const patchParent = valueAt(out, parentPath);
+    const patchParent = parentPath.length === 0 ? out : valueAt(out, parentPath);
     if (!patchParent || typeof patchParent !== 'object' || Array.isArray(patchParent)) continue;
-    if (leafKey in (patchParent as Record<string, unknown>)) continue;
+    // What the read path served at this position: `undefined` for a dropped
+    // key, the rewritten string for a URL redaction. The patch speaks for the
+    // author exactly where it DIFFERS from that projection.
+    const servedParent = parentPath.length === 0 ? served.config : valueAt(served.config, parentPath);
+    const servedLeaf =
+      servedParent && typeof servedParent === 'object' && !Array.isArray(servedParent)
+        ? (servedParent as Record<string, unknown>)[leafKey]
+        : undefined;
+    if ((patchParent as Record<string, unknown>)[leafKey] !== servedLeaf) continue;
     graftAt(out, path, storedLeaf);
   }
 

packages/spec/api-surface/data.json

@@ -56,12 +56,14 @@
     "BulkWriteHookPhase (type)",
     "BulkWritePayloadScope (type)",
     "CALENDAR_DATE_TYPES (const)",
+    "CANONICAL_CREDENTIAL_KEYS (const)",
     "CLOCK_TIME_TYPES (const)",
     "COMPUTED_VALUE_TYPES (const)",
     "CONTEXT_TOKENS (const)",
     "CONTEXT_TOKEN_DESCRIPTIONS (const)",
     "CONTEXT_TOKEN_SUGGESTIONS (const)",
     "CONTEXT_TOKEN_WRAPPED_RE (const)",
+    "CREDENTIAL_KEY_SPELLINGS (const)",
     "CREDENTIAL_URL_QUERY_PARAMS (const)",
     "CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
     "CalendarDateValue (type)",
@@ -258,6 +260,7 @@
     "FILTER_TEXT_CASES (const)",
     "FILTER_TEXT_ROWS (const)",
     "FILTER_TOKEN_WRAPPED_RE (const)",
+    "FORMER_CREDENTIAL_ALIASES (const)",
     "FeedFilterMode (type)",
     "FeedItemType (type)",
     "Field (type)",
@@ -452,6 +455,7 @@
     "PAGINATION_UNORDERED_CASES (const)",
     "PAGINATION_ZERO_LIMIT_CASES (const)",
     "PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
+    "PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
     "PaginationConformanceCase (interface)",
     "PaginationConformanceRow (interface)",
     "PerOperationRequiredPermissions (type)",
@@ -750,6 +754,8 @@
     "referenceTargetOf (function)",
     "referencedFields (function)",
     "refusedCredentialKeys (function)",
+    "refusedCredentialPaths (function)",
+    "refusedCredentialPathsOfSchema (function)",
     "refusedInlineCredentialKey (function)",
     "refusedPassthroughSecretPaths (function)",
     "renderAutonumber (function)",

packages/spec/export-origins/data.json

@@ -56,12 +56,14 @@
     "BulkWriteHookPhase": "src/data/bulk-write-hook-conformance.ts#BulkWriteHookPhase (type)",
     "BulkWritePayloadScope": "src/data/bulk-write-hook-conformance.ts#BulkWritePayloadScope (type)",
     "CALENDAR_DATE_TYPES": "src/data/field-value.zod.ts#CALENDAR_DATE_TYPES (const)",
+    "CANONICAL_CREDENTIAL_KEYS": "src/data/driver/common.zod.ts#CANONICAL_CREDENTIAL_KEYS (const)",
     "CLOCK_TIME_TYPES": "src/data/field-value.zod.ts#CLOCK_TIME_TYPES (const)",
     "COMPUTED_VALUE_TYPES": "src/data/field-value.zod.ts#COMPUTED_VALUE_TYPES (const)",
     "CONTEXT_TOKENS": "src/data/context-tokens.zod.ts#CONTEXT_TOKENS (const)",
     "CONTEXT_TOKEN_DESCRIPTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_DESCRIPTIONS (const)",
     "CONTEXT_TOKEN_SUGGESTIONS": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_SUGGESTIONS (const)",
     "CONTEXT_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#CONTEXT_TOKEN_WRAPPED_RE (const)",
+    "CREDENTIAL_KEY_SPELLINGS": "src/data/driver/common.zod.ts#CREDENTIAL_KEY_SPELLINGS (const)",
     "CREDENTIAL_URL_QUERY_PARAMS": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAMS (const)",
     "CREDENTIAL_URL_QUERY_PARAM_NAMES": "src/data/driver/common.zod.ts#CREDENTIAL_URL_QUERY_PARAM_NAMES (const)",
     "CalendarDateValue": "src/data/field-value.zod.ts#CalendarDateValue (type)",
@@ -258,6 +260,7 @@
     "FILTER_TEXT_CASES": "src/data/filter-text-conformance.ts#FILTER_TEXT_CASES (const)",
     "FILTER_TEXT_ROWS": "src/data/filter-text-conformance.ts#FILTER_TEXT_ROWS (const)",
     "FILTER_TOKEN_WRAPPED_RE": "src/data/context-tokens.zod.ts#FILTER_TOKEN_WRAPPED_RE (const)",
+    "FORMER_CREDENTIAL_ALIASES": "src/data/driver/common.zod.ts#FORMER_CREDENTIAL_ALIASES (const)",
     "FeedFilterMode": "src/data/feed.zod.ts#FeedFilterMode (type)",
     "FeedItemType": "src/data/feed.zod.ts#FeedItemType (type)",
     "Field": "src/data/field.zod.ts#Field (type)",
@@ -452,6 +455,7 @@
     "PAGINATION_UNORDERED_CASES": "src/data/pagination-conformance.ts#PAGINATION_UNORDERED_CASES (const)",
     "PAGINATION_ZERO_LIMIT_CASES": "src/data/pagination-conformance.ts#PAGINATION_ZERO_LIMIT_CASES (const)",
     "PASSTHROUGH_INLINE_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_INLINE_CREDENTIAL_REFUSED (const)",
+    "PASSTHROUGH_NESTED_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#PASSTHROUGH_NESTED_CREDENTIAL_REFUSED (const)",
     "PaginationConformanceCase": "src/data/pagination-conformance.ts#PaginationConformanceCase (interface)",
     "PaginationConformanceRow": "src/data/pagination-conformance.ts#PaginationConformanceRow (interface)",
     "PerOperationRequiredPermissions": "src/data/object.zod.ts#PerOperationRequiredPermissions (type)",
@@ -750,6 +754,8 @@
     "referenceTargetOf": "src/data/field-value.zod.ts#referenceTargetOf (function)",
     "referencedFields": "src/data/autonumber-format.ts#referencedFields (function)",
     "refusedCredentialKeys": "src/data/datasource-credential-redaction.ts#refusedCredentialKeys (function)",
+    "refusedCredentialPaths": "src/data/datasource-credential-redaction.ts#refusedCredentialPaths (function)",
+    "refusedCredentialPathsOfSchema": "src/data/datasource-credential-redaction.ts#refusedCredentialPathsOfSchema (function)",
     "refusedInlineCredentialKey": "src/data/driver/common.zod.ts#refusedInlineCredentialKey (function)",
     "refusedPassthroughSecretPaths": "src/data/datasource-credential-redaction.ts#refusedPassthroughSecretPaths (function)",
     "renderAutonumber": "src/data/autonumber-format.ts#renderAutonumber (function)",