refactor(types): move the 5xx fault-log rule to @objectstack/types and land it on sendError

Re-homed under the domain seat's serial fence: `packages/rest/src/package-routes.ts`
is held by the open PR #14499, so this no longer edits that file. The REST
direct-mount doors are covered from the producer side instead.

`sendError` (`@objectstack/types`) is the single writer for every
nested-envelope error in the repo, and every catch in the package registrar
ends there — so wiring the rule at that one exit covers those doors with no
per-door call, and covers any door added later by construction. That also puts
the helper in the same package as `resolveThrownHttpError`, on the same
argument: a rule two doors must agree on cannot live inside one of them.

`@objectstack/observability` and `packages/rest` are back to origin/main
byte-for-byte, as is the system-context census page (its line rot was caused
by the package-routes import this drops).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Author: hotlong

.changeset/log-every-5xx-server-fault.md

@@ -1,10 +1,9 @@
 ---
-"@objectstack/observability": patch
+"@objectstack/types": patch
 "@objectstack/runtime": patch
-"@objectstack/rest": patch
 ---
 
-fix(observability,runtime,rest): log every 5xx at `error` level instead of answering it silently (#14310)
+fix(types,runtime): log every 5xx at `error` level instead of answering it silently (#14310)
 
 A 500 that leaves no server-side line is diagnosed from the browser or not at
 all. Measured on `main`, through the real plugin and the real route handlers: a
@@ -27,21 +26,24 @@ reasons:
   so even a wired reporter never saw those.
 
 **The rule now has one definition.** `logServerFault` (new, in
-`@objectstack/observability`, which owns the operator-facing `Logger` /
-`ErrorReporter` channel and is already a dependency of both consumers) emits
-exactly one `error`-level record carrying method, path, request id, the
-message and — where the door still holds the throw — the stack. It could not
-live in either consumer: `@objectstack/runtime` depends on `@objectstack/rest`,
-so an import could only ever point one way — the same argument that put
-`resolveThrownHttpError` in `@objectstack/types` rather than in one of its two
-doors.
-
-Wired at each transport's own single exit, so a fault costs one line and never
-two: the dispatcher's thrown exit (`errorResponseBase`) and returned exit
-(`sendResultBase`), the AI-route mount that writes its own result, and the REST
-direct-mount package registrar's `sendThrownError` plus its two reported
-driver faults. `packages/rest`'s `/data` doors were already loud
-(`logUnexpectedRouteError`) and are untouched.
+`@objectstack/types`) emits exactly one `error`-level record carrying method,
+path, request id, the message and — where the door still holds the throw — the
+stack. It shares a home with `resolveThrownHttpError` for the same reason that
+rule was moved there in #8016: a rule two doors must agree on cannot live
+inside one of them, because `@objectstack/runtime` depends on
+`@objectstack/rest` and an import could only ever point one way.
+
+Wired at each transport's single exit, so a fault costs one line and never two:
+
+- `sendError` — the one writer for every nested-envelope error in the repo. The
+  REST direct-mount registrars (the `/api/v1/packages` door that mounts first
+  in production) become loud through it with no per-door call, so a door added
+  later cannot forget one.
+- The dispatcher's thrown exit (`errorResponseBase`), its returned exit
+  (`sendResultBase`) and the AI-route mount that writes its own result.
+
+`packages/rest`'s `/data` doors were already loud via `logUnexpectedRouteError`
+and are untouched.
 
 `error` level is load-bearing: the CLI's default is `warn` and `error` (40)
 outranks `warn` (30), so the record clears `--log-level`'s default without
@@ -51,7 +53,8 @@ deliberate instruction rather than the default this fixes.
 **4xx stays quiet**, decided once inside the helper rather than at each call
 site — client mistakes are already explained by the response, and logging them
 is how a `?state=draft` probe once printed 45 stack traces in one browsing
-session.
+session. The wire body is byte-identical at every door: this adds a side
+effect, never a field.
 
 ⚠️ Behaviour change worth knowing before upgrading: a deployment that answers
 a *declared* 5xx on a polled route — `501 NOT_IMPLEMENTED` from an uninstalled

content/docs/permissions/system-context.mdx

@@ -160,9 +160,9 @@ The largest single consumer — **20 of the 109 sites**.
 | 49 | Action `requiredPermissions` bypassed | runtime | Get: engine self-invocation runs any action | `action-execution.ts:399` |
 | 50 | `manage_metadata` bypassed on metadata writes | runtime, rest | Get: schema writes without the capability | `domains/meta.ts:471`, `:874`, `rest-server.ts:4716`, `:6079`, `:6327`, `:6758`, `:6951` |
 | 51 | The shared metadata-write verdict itself returns `allowed` | metadata-core | Get: the one function all of row 50's doors consult answers yes before any capability is examined | `meta-write-capability.ts:134` |
-| 52 | Anonymous-deny seam satisfied on the domain dispatchers and the package/federation routes | runtime, rest | Get: passes with no `userId` | `domains/actions.ts:411`, `domains/ai.ts:60`, `domains/automation.ts:989`, `domains/meta.ts:232`, `domains/security.ts:78`, `domains/packages.ts:276`, `external-datasource-routes.ts:302`, `package-routes.ts:98` |
+| 52 | Anonymous-deny seam satisfied on the domain dispatchers and the package/federation routes | runtime, rest | Get: passes with no `userId` | `domains/actions.ts:411`, `domains/ai.ts:60`, `domains/automation.ts:989`, `domains/meta.ts:232`, `domains/security.ts:78`, `domains/packages.ts:276`, `external-datasource-routes.ts:302`, `package-routes.ts:97` |
 | 53 | MCP principal check satisfied | runtime | Get: MCP surface reachable with no user | `domains/mcp.ts:61` |
-| 54 | Package REST route capability gate bypassed | rest | Get: package read/write over REST without `manage_metadata` / `studio.access` / `setup.access` | `package-routes.ts:103` |
+| 54 | Package REST route capability gate bypassed | rest | Get: package read/write over REST without `manage_metadata` / `studio.access` / `setup.access` | `package-routes.ts:102` |
 | 55 | Package domain capability gates bypassed | runtime | Get: package management and package-inventory reads without the capability | `domains/packages.ts:95`, `:128` |
 | 56 | Activation write / authoring refusals do not fire | runtime | Get: activation artifacts writable and authorable without the activation-authoring capability | `activation-gate.ts:139`, `:190` |
 | 57 | Automation run-state read, flow-authoring write and unrelated-screen read all pass | runtime | Get: run state, flow writes and screen reads with no grant | `domains/automation.ts:254`, `:545`, `:635` |

packages/observability/src/index.ts

@@ -42,20 +42,6 @@ export {
     ConsoleErrorReporter,
 } from './error-exporters.js';
 
-// [#14310] The shared "a 5xx is never silent" rule, read by every transport
-// that turns a fault into an HTTP envelope (REST's direct-mount doors and the
-// runtime dispatcher's two exits).
-export {
-    logServerFault,
-    isServerFault,
-    serverFaultLogMessage,
-    serverFaultLogMeta,
-    describeFaultRequest,
-    SERVER_FAULT_LOG_PREFIX,
-    type ServerFaultLogInput,
-    type ServerFaultRequest,
-} from './server-fault-log.js';
-
 // Loggers
 export {
     NoopLogger,

packages/rest/src/package-routes.ts

@@ -7,7 +7,6 @@ import { IHttpServer, shouldDenyAnonymous, ANONYMOUS_DENY_STATUS, ANONYMOUS_DENY
 // gate's cohort was ruled separately (#7033 / #7023) and pins write-only callers
 // OUT. Same value it read before — no re-ruling by side effect.
 import { OBJECT_SCHEMA_READ_ONLY_EXEMPT_CAPABILITIES } from '@objectstack/metadata-core';
-import { logServerFault } from '@objectstack/observability';
 import type { PackageService } from '@objectstack/service-package';
 // The declared envelope is written in ONE place for the whole platform (#3973),
 // and so (#8016) is the rule that reads an HTTP answer off a THROWN error.
@@ -266,16 +265,6 @@ function sendThrownError(res: any, error: unknown): void {
     ...(declaredCode !== undefined ? { declaredCode } : {}),
     ...(thrown.userMessage !== undefined ? { userMessage: thrown.userMessage } : {}),
   };
-  // [#14310] This registrar's 5xx were silent, and it is the door that mounts
-  // FIRST in the production stack — so for `/api/v1/packages` the silent
-  // answer was the live one, which is how the fault this card was filed on
-  // stayed invisible for a week. `logServerFault` owns the 5xx test, so the
-  // coded 4xx refusals this exit exists to carry (#8016's `409
-  // DESTRUCTIVE_CHANGE`, the `[tenant_scope_required]` 400) still cost no
-  // line. It logs the UNSANITISED `error`: the withhold above is scoped to
-  // what the CLIENT reads, and an operator losing the driver text to the same
-  // rule would trade a leak for the blind spot #5437 already refused.
-  logServerFault({ status: thrown.status, error, code: thrown.code });
   sendError(
     res,
     thrown.status,
@@ -647,14 +636,6 @@ export function registerPackageRoutes(
       // for a `PackageService` implementation that reports failure without
       // saying why, which is the one thing the old `error?: string` could not
       // distinguish from a driver dump.
-      // [#14310] A REPORTED driver fault never passes through
-      // {@link sendThrownError} — nothing was thrown — so it owes its own
-      // line, or this 5xx stays as silent as the thrown ones were.
-      logServerFault({
-        status: 500,
-        code: 'PACKAGE_PUBLISH_FAILED',
-        message: result.driverFault?.message ?? `Failed to publish ${manifest.id}.`,
-      });
       sendError(
         res,
         500,
@@ -1014,12 +995,6 @@ export function registerPackageRoutes(
       // producer returns a bare flag with no message channel at all
       // (`PackageDeleteResult`), which is what keeps that true — this route is
       // a status-classification defect only, never a disclosure.
-      // [#14310] Same shape as the publish fault above: reported, not thrown.
-      logServerFault({
-        status: 500,
-        code: 'PACKAGE_DELETE_FAILED',
-        message: `Failed to delete ${packageId}${version ? `@${version}` : ''}.`,
-      });
       sendError(
         res,
         500,

packages/runtime/src/dispatcher-plugin.ts

@@ -1,9 +1,9 @@
 // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
 
 import { Plugin, PluginContext, IHttpServer, ANONYMOUS_DENY_BODY, ANONYMOUS_DENY_STATUS } from '@objectstack/core';
-import { looksLikeInternalErrorLeak, INTERNAL_ERROR_MESSAGE, resolveThrownHttpError, serverFaultProvenance, demotedDeclaredCode } from '@objectstack/types';
+import { looksLikeInternalErrorLeak, INTERNAL_ERROR_MESSAGE, resolveThrownHttpError, serverFaultProvenance, demotedDeclaredCode, logServerFault, describeFaultRequest } from '@objectstack/types';
 import { DispatcherErrorCode } from '@objectstack/spec/api';
-import type { IAuthService, IMetadataService } from '@objectstack/spec/contracts';
+import type { IAuthService, IMetadataService, Logger } from '@objectstack/spec/contracts';
 import type { CounterStore } from '@objectstack/plugin-auth/rate-limit-storage';
 import { HttpDispatcher, HttpDispatcherResult, type HttpProtocolContext } from './http-dispatcher.js';
 import { isServiceServeable } from './service-serveable.js';
@@ -20,11 +20,6 @@ import {
 } from './security/index.js';
 import { resolveSessionData, resolveSessionPrincipalId } from './security/resolve-session-principal.js';
 import { buildActorUser } from './security/actor-user.js';
-import {
-    logServerFault,
-    describeFaultRequest,
-    type Logger,
-} from '@objectstack/observability';
 import {
     NoopMetricsRegistry,
     NoopErrorReporter,

packages/types/src/index.ts

@@ -18,6 +18,10 @@ export * from './response-envelope.js';
 // direct-mount REST registrar, which used to answer 500 INTERNAL_ERROR for a
 // coded 4xx the dispatcher mapped correctly.
 export * from './thrown-http-error.js';
+// [#14310] The sibling rule to the one above, for the same two doors: "is this
+// answer worth an operator's attention?". `resolveThrownHttpError` decides what
+// the CLIENT is told; this decides what the LOG says — 5xx always, 4xx never.
+export * from './server-fault-log.js';
 export * from './validation-failure.js';
 // [#6615] The one home for Postgres' `«sub-object» "x" of relation "y"` phrase,
 // whose missing-COLUMN spelling contains a legal missing-TABLE phrase as a

packages/types/src/response-envelope.ts

@@ -51,6 +51,7 @@
  */
 
 import type { ApiError, ErrorCode } from '@objectstack/spec/api';
+import { logServerFault } from './server-fault-log.js';
 
 /**
  * The only thing an envelope builder needs from a response object.
@@ -207,5 +208,22 @@ export function sendError(
   message: string,
   extra?: Pick<ApiError, 'category' | 'httpStatus' | 'details' | 'requestId' | 'declaredCode' | 'userMessage'>,
 ): void {
+  // [#14310] A 5xx is never silent. This writer is the single exit for every
+  // nested-envelope error in the repo, so the rule is applied ONCE here rather
+  // than at each registrar's catch block — a per-door call is a thing a new
+  // door can forget, and the `/api/v1/packages` 500 that motivated the card
+  // went unlogged for a week through exactly such a door.
+  //
+  // ⛔ Not a second opinion about the answer: `logServerFault` reads the same
+  // `status` this call is about to write, and its own 5xx gate keeps every
+  // deliberate 4xx refusal — the coded `409 DESTRUCTIVE_CHANGE`, the
+  // `403 FORBIDDEN` capability denials above it — as quiet as they were.
+  //
+  // The thrown value is not available here (callers resolve it into `message`
+  // before arriving), so this line carries the message and code rather than a
+  // stack. A door still holding the throw can call `logServerFault` itself for
+  // the stack-bearing line; none does today, and the transports that DO hold
+  // it log at their own exits instead.
+  logServerFault({ status, code, message, ...(extra?.requestId ? { request: { requestId: extra.requestId } } : {}) });
   res.status(status).json({ success: false, error: { code, message, ...extra } });
 }

…y/src/__tests__/server-fault-log.test.ts …kages/types/src/server-fault-log.test.tspackages/observability/src/__tests__/server-fault-log.test.ts renamed to packages/types/src/server-fault-log.test.ts packages/observability/src/__tests__/server-fault-log.test.ts renamed to packages/types/src/server-fault-log.test.ts

@@ -22,7 +22,8 @@ import {
     serverFaultLogMeta,
     describeFaultRequest,
     SERVER_FAULT_LOG_PREFIX,
-} from '../server-fault-log.js';
+} from './server-fault-log.js';
+import { sendError } from './response-envelope.js';
 
 const spyLogger = () => ({
     debug: vi.fn(),
@@ -119,6 +120,70 @@ describe('serverFaultLogMessage / serverFaultLogMeta', () => {
     });
 });
 
+describe('sendError — the funnel every nested-envelope 5xx exits through', () => {
+    /**
+     * This is the half that makes the REST direct-mount registrars loud
+     * without any per-door call: `packages/rest`'s package routes end every
+     * catch in `sendError`, so wiring the rule HERE covers them (and any door
+     * added later) by construction rather than by remembering.
+     *
+     * The sink is `console.error` because `sendError` takes no logger — it is
+     * a pure envelope writer reached from ~50 sites that have no logger to
+     * pass. Spying it is the only way to observe this seam, and it is done
+     * ONLY here: the behavioural pins that matter (level, exact count, the
+     * message and stack) assert against an INJECTED logger above and in
+     * `packages/runtime/src/dispatcher-5xx-always-logged.test.ts`, where a
+     * console spy would have been the weaker instrument.
+     */
+    const makeRes = () => {
+        const res: any = {
+            statusCode: undefined as number | undefined,
+            body: undefined as any,
+            status(c: number) { res.statusCode = c; return res; },
+            json(b: any) { res.body = b; return res; },
+        };
+        return res;
+    };
+
+    it('logs a 5xx', () => {
+        const spy = vi.spyOn(console, 'error').mockImplementation(() => { });
+        try {
+            sendError(makeRes(), 500, 'INTERNAL_ERROR', 'the driver fell over');
+            const lines = spy.mock.calls.filter((c) => String(c[0]).startsWith(SERVER_FAULT_LOG_PREFIX));
+            expect(lines).toHaveLength(1);
+            expect(String(lines[0][0])).toContain('the driver fell over');
+        } finally {
+            spy.mockRestore();
+        }
+    });
+
+    it('stays quiet on a 4xx — the coded refusals this door exists to carry', () => {
+        const spy = vi.spyOn(console, 'error').mockImplementation(() => { });
+        try {
+            sendError(makeRes(), 409, 'DESTRUCTIVE_CHANGE', 'that change drops a column');
+            sendError(makeRes(), 403, 'FORBIDDEN', 'Managing packages requires `manage_metadata`.');
+            expect(spy.mock.calls.filter((c) => String(c[0]).startsWith(SERVER_FAULT_LOG_PREFIX))).toHaveLength(0);
+        } finally {
+            spy.mockRestore();
+        }
+    });
+
+    it('leaves the wire body byte-identical — this change adds a side effect, not a field', () => {
+        const spy = vi.spyOn(console, 'error').mockImplementation(() => { });
+        try {
+            const res = makeRes();
+            sendError(res, 500, 'INTERNAL_ERROR', 'boom', { details: { a: 1 } });
+            expect(res.statusCode).toBe(500);
+            expect(res.body).toEqual({
+                success: false,
+                error: { code: 'INTERNAL_ERROR', message: 'boom', details: { a: 1 } },
+            });
+        } finally {
+            spy.mockRestore();
+        }
+    });
+});
+
 describe('describeFaultRequest', () => {
     it('reads the spellings adapters actually use', () => {
         expect(describeFaultRequest({ method: 'GET', path: '/a', requestId: 'r1' }))

…es/observability/src/server-fault-log.ts packages/types/src/server-fault-log.tspackages/observability/src/server-fault-log.ts renamed to packages/types/src/server-fault-log.ts packages/observability/src/server-fault-log.ts renamed to packages/types/src/server-fault-log.ts

@@ -28,15 +28,18 @@
  *
  * ## Why it lives here
  *
- * `@objectstack/rest` and `@objectstack/runtime` both emit 5xx envelopes and
- * both already depend on this package, which owns the operator-facing channel
- * (`Logger`, `LOG_LEVELS`, `ErrorReporter`). The rule cannot live in either
- * consumer: `runtime` depends on `rest`, so an import could only ever point
- * one way — the same argument that put `resolveThrownHttpError` in
- * `@objectstack/types` rather than in one of its two doors. Centralising it
- * is also what keeps the two doors serving `/api/v1/packages` from printing
- * two lines for one fault: each door logs at its own single exit, and the
- * predicate that decides "is this worth a line" has one definition.
+ * Same argument, and the same package, as `resolveThrownHttpError` one file
+ * over: a rule two doors must agree on cannot live inside one of them.
+ * `@objectstack/runtime` depends on `@objectstack/rest`, so an import between
+ * the two doors could only ever point one way — which is exactly why the
+ * "what status does this throw mean" rule was moved here in #8016. "Is this
+ * answer worth an operator's attention" is the same kind of rule, read by the
+ * same two doors, so it gets the same home rather than a second one.
+ *
+ * Living beside {@link sendError} is what makes the REST side automatic: that
+ * writer is the single exit for every nested-envelope 5xx, so the direct-mount
+ * registrars need no per-door call and cannot forget one. Each transport logs
+ * at its own single exit, so a fault costs one line and never two.
  *
  * ## `error` level, and why that clears the default
  *
@@ -56,7 +59,7 @@
  * whole rule: at or above 500.
  */
 
-import type { Logger } from './contracts.js';
+import type { Logger } from '@objectstack/spec/contracts';
 
 /** The request coordinates an operator needs to find the failing call. */
 export interface ServerFaultRequest {