merge origin/main into claude/issue-14744-multi-update-per-row-value-census

Author: claude

.changeset/cli-option-b-config-load-boundaries.md

@@ -0,0 +1,45 @@
+---
+"@objectstack/cli": patch
+---
+
+fix(cli): `os serve` / `os dev` / `os build` / `os migrate` resolve `packages[]` when a stack carries no flattened top level
+
+The CLI holds four independent config-load boundaries, and every read of a
+package-owned collection behind them was an inline expression against the
+FLATTENED top level. A multi-package stack that carries each definition once
+under `packages[]` — the shape ADR-0130 D4's option B produces — reached those
+expressions with the key simply absent, and nothing threw:
+
+- `os serve` / `os dev` auto-register the ObjectQL engine and the storage driver
+  when the stack declares objects. Both gates read `config.objects`, so the app
+  booted with **no query engine and no storage driver** and reported healthy.
+  Nothing between the artifact and the gate could notice: the standalone stack
+  omits the `objects` key entirely when the array is absent rather than setting
+  `[]`, and the boot-config merge is a plain spread.
+- `os serve` auto-registers the i18n service plugin when the stack carries
+  translations. `translations` is package-owned while `i18n` is an envelope key a
+  translations-only stack never sets, so the REST i18n routes silently did not
+  exist.
+- `os dev` diffs the artifact's object inventory across recompiles to name a
+  newly added `*.object.ts`. It went permanently empty, so every recompile read
+  as all-green.
+- `os build` runs the author-time rule table twice — once over the union, once
+  per package. The per-package run already read `packages[]`; the union run,
+  which is the only one of the two that can see a finding spanning packages,
+  judged an empty stack and published green.
+
+All of them now resolve through one seam, in the dependency-topological order
+`resolveArtifactPackageOrder` gives. Each answer starts from the expression it
+replaced, so every stack that boots or builds today takes the identical branch —
+including a stack declaring an empty `objects: []`, which stays a stack that gets
+an engine — and `packages[]` is consulted only where the old read returned
+nothing. A malformed `packages` list is refused with its ADR-0112 envelope on
+that leg instead of resolving to the silent empty.
+
+The predicate `os serve` and `os migrate` each carried their own copy of — "does
+this config carry app metadata that needs an `AppPlugin` wrap" — is now one
+function. Measured, it does not lose under the new shape; it is folded in because
+it is the master gate for everything `AppPlugin` then reads.
+
+No command emits anything different: the compiled artifact still carries both
+copies, and the folded stack is a rule INPUT that reaches no writer.

.changeset/plugin-security-default-profile-packages-reader.md

@@ -0,0 +1,50 @@
+---
+"@objectstack/plugin-security": patch
+---
+
+fix(plugin-security): the app default permission set resolves from `packages[]`, not only the flattened top level (#15007)
+
+`appSecurityPluginOptions(config)` read `config.permissions` and nothing else.
+For a multi-package artifact under the ADR-0130 D4 option-B shape — where
+`packages[]` carries each definition exactly once and the flattened top-level
+copy is gone — that read returns `undefined`, the reader concludes "this app
+declared no default profile", and the boot continues. Nothing throws and
+nothing logs.
+
+That silence has a security posture attached. The name this resolves becomes
+the `SecurityPlugin`'s `fallbackPermissionSet`, i.e. the app's half of every
+authenticated human principal's additive baseline
+(`composeHumanBaselinePermissionSets`, ADR-0090 D5). Losing it does not deny
+anyone the boot — the deployment simply runs on the platform floor alone, and
+every member of a multi-package app quietly holds less access than the app
+declared for them. #7555 measured what that looks like from the outside: nav
+entries served, 403 behind them.
+
+The resolution now reads the flattened top level FIRST and then each package
+body, in the order `resolveArtifactPackageOrder` (`@objectstack/core`,
+ADR-0130 D4+D5) registers them:
+
+- **Every artifact the platform emits today answers bit-identically.** The
+  flattened level still answers first, so the `packages[]` pass can only supply
+  a set where the top level had none. This is the reader half of the ruled
+  order (readers first, emitter last, artifact stays additive throughout), so
+  it lands with no change to what any command emits.
+- **Order is the platform's one package order, not the array's.**
+  `appDefaultPermissionSetName` resolves the FIRST `isDefault` set, so with two
+  packages declaring one, "first" has to mean here what it means at every other
+  artifact reader: dependency-topological, so a package that extends another is
+  read after it whichever array slot it occupies.
+- **The singular `manifest` is still not consulted** (#7001 — the harness must
+  not honour a declaration `serve.ts` ignores). That is not a special case: an
+  artifact carrying no `packages` key makes `resolveArtifactPackageOrder`
+  return the caller's own object as the single package body, so that branch
+  reads `permissions` from exactly where the old code read it.
+- **A malformed `packages` is refused, not skipped.** A non-array `packages`,
+  an entry inlined instead of wrapped under `manifest:`, or a duplicate package
+  id raises the same ADR-0112 envelope (`code` + `status: 422`) the manifest
+  service raises when it registers that artifact. Catching it would resolve a
+  permission surface out of an artifact the loader refuses to load.
+
+Every boot path that already funnelled through this one function picks the fix
+up unchanged: `objectstack serve`'s artifact and from-source paths, and
+`@objectstack/verify`'s `bootStack` / RLS harness.

.changeset/service-storage-test-tsc-program.md

@@ -0,0 +1,50 @@
+---
+"@objectstack/service-storage": patch
+---
+
+fix(service-storage): put the test layer in front of tsc, and repair what it was hiding (#15050)
+
+`packages/services/service-storage` had **no `typecheck` script at all** — its
+scripts were `build` and `test` — so no tsc program anywhere read this
+package's test layer, and its errors were carried instead as a 51-error DEBT
+entry in `scripts/check-type-check-coverage.mjs`. Gives it the #14062 /
+#14181 "checked test zone" shape: a sibling `tsconfig.test.json` (module
+semantics only — `esnext` / `bundler` / `lib: ES2022` — matching how vitest
+actually executes these files; strictness inherited and untouched) plus a
+`tsconfig.scripts.json` for `scripts/i18n-extract.config.ts` (the ninth
+instance of #11351, previously excluded from that ledger only because this
+package had no `typecheck` script to hang it on), both named by a new
+`typecheck` script.
+
+Measured before repair: 51 errors under BUILD semantics (`tsc --noEmit -p
+tsconfig.json`, which already includes the tests — matching the DEBT entry's
+recorded number exactly), 10 under the split. Unlike `service-cluster`
+(#14181), this package's BUILD reading was *not* already clean, so both
+programs needed genuine repair, not just the test-only split: 23 `TS2835`
+(relative imports missing their `.js` extension, required under BUILD's
+NodeNext resolution) were fixed by *adding* the extension — which resolves
+correctly under both NodeNext and the split's bundler mode — and clearing
+that also cleared all 15 `TS7006` "implicitly any" as a downstream cascade
+from the same unresolved imports (the shape `@objectstack/core` reported at
+98 → 4). The remaining 3 `TS2550` (`Array.prototype.at` needing `lib`
+es2022) are rewritten to indexed access rather than widening the shared
+BUILD `tsconfig.json`. The 8 code-tier errors (`TS2339` × 4 — a test
+helper's object-spread dropped its `Record<string, unknown>` index
+signature, fixed with an explicit return-shape annotation; `TS2347` × 4 — a
+fake `ctx: any`'s `getService<T>(...)` calls converted to `getService(...)
+as T`, the pattern one call site in the same file had already adopted for
+exactly this reason) are genuine test-file fixes. Both readings now agree at
+0/0 — the same result `service-cluster` reported, reached by a longer road.
+
+The package's DEBT entry (51 errors) is **deleted**, not lowered — the
+graduation this ratchet's invariant requires. No `test-typecheck-debt.json`
+is added: residue is 0, so none is owed (#5286, maintainer-only to open).
+`check:type-source-resolution` went red from onboarding the two new
+programs (the documented onboarding-limb case): a registry entry is added
+rather than `paths`, measured both ways — `paths` takes this package's test
+layer from 0 errors to 306, all in other packages' source.
+
+No runtime code changes: `src/**` excluding tests is byte-identical, so no
+shipped behaviour moves. The `patch` level reflects the published
+`package.json` gaining `typecheck` / `check:test-typecheck` scripts and a
+`tsx` devDependency.

.changeset/sys-email-error-description-widen.md

@@ -0,0 +1,22 @@
+---
+"@objectstack/platform-objects": patch
+---
+
+fix(platform-objects): `sys_email.error` field help now covers pre-delivery rejections, not only transport failures
+
+`sys_email.error` was declared as *"Transport error message when status=failed"*.
+Since `EmailService.recordRejectedMessage` landed, the same column also carries
+the reason a message was rejected by `normalizeMessage` **before** it reached a
+transport (an unsendable `from`, no recipient, no subject, no body) — those rows
+are written with `status: 'failed'` too, prefixed `rejected before delivery: `.
+
+Nothing was misleading in the *data*: the row prefixes its own reason, so an
+operator reading a failed row is never sent chasing an SMTP host for a message
+that never reached one. What was stale was the field's declared `description`,
+which Studio surfaces as the field's help text — it named only the transport
+case, narrower than what the column has held since that change landed.
+
+The description now reads: *"Why the message failed — a transport error, or the
+validation that rejected it before delivery."* It stays true under both row
+shapes and deliberately does not name the row's own `rejected before delivery:`
+prefix, so it will not go stale again if that prefix's wording changes.

packages/cli/src/commands/compile.ts

@@ -15,6 +15,7 @@ import {
 } from '@objectstack/spec';
 import { loadConfig } from '../utils/config.js';
 import { lowerCallables } from '../utils/lower-callables.js';
+import { authoringRuleUnionStack } from '../utils/stack-collections.js';
 import { buildAccessMatrix, diffAccessMatrix } from '@objectstack/lint';
 import { runAuthoringRules, splitBySeverity, authoringRulesFor } from '@objectstack/lint';
 import { resolveSduiManifest } from '../utils/sdui-manifest.js';
@@ -356,9 +357,24 @@ export default class Compile extends Command {
       //     is declared in `lint/authoring-rules.ts`. Do not add a call site here.
       const registered = authoringRulesFor('build');
       if (!flags.json) printStep(`Running author-time rules (${registered.length})...`);
+      //     [ADR-0130 D4 / option B, #15006] The UNION run judges the flattened
+      //     top level. Under option B that top level is gone — `packages[]`
+      //     carries every definition once — so this run's input would be an
+      //     EMPTY stack and `os build` would publish green having judged
+      //     nothing. `authoringRuleUnionStack` folds each absent collection
+      //     back in from `packages[]`, in `resolveArtifactPackageOrder`'s
+      //     dependency order. It changes what the rules JUDGE and nothing this
+      //     command EMITS: the artifact is written from `lowering.lowered` /
+      //     `result.data`, which this does not touch, and a stack that still
+      //     carries its collections is returned by identity.
+      //
+      //     The per-package run below needs no such fold — it already reads
+      //     `packages[]`. The union run is the only one of the two that can see
+      //     a finding spanning packages, which is exactly what an empty input
+      //     silently stops reporting.
       const findings = runAuthoringRules('build', {
-        normalized: normalized as Record<string, unknown>,
-        parsed: result.data as Record<string, unknown>,
+        normalized: authoringRuleUnionStack(normalized as Record<string, unknown>),
+        parsed: authoringRuleUnionStack(result.data as Record<string, unknown>),
         sduiManifest: resolveSduiManifest(),
       });
       const { errors: ruleErrors, advisories } = splitBySeverity(findings);

packages/cli/src/commands/dev.ts

@@ -17,6 +17,7 @@ import {
   formatMtimeGap,
 } from '../utils/dev-restart.js';
 import { childEnvWithResolvedArtifact } from '../utils/internal-artifact-channel.js';
+import { artifactObjectNames } from '../utils/stack-collections.js';
 import { readEnvWithDeprecation, isMcpServerEnabled } from '@objectstack/types';
 // The ONE port contract, shared with `start` and with the `serve` child this
 // command spawns (#12673). ⛔ Nothing about ports is declared in this file —
@@ -650,16 +651,16 @@ export default class Dev extends Command {
       // newly added *.object.ts is called out explicitly (15.1 third-party
       // eval: "recompiled" alone read as all-green while the new object's
       // table/seed sync was invisible to the user).
+      // [ADR-0130 D4 / option B, #15006] The envelope unwrap and the object
+      // read are `artifactObjectNames` — one of this package's four reads of a
+      // PACKAGE-OWNED collection, and the only one whose loss is non-fatal: with
+      // the flattened top level gone this inventory went permanently EMPTY, so
+      // `os dev` stopped naming a newly added *.object.ts and every recompile
+      // read as all-green. The file read and the `null`-on-failure contract stay
+      // here; the seam is what the acceptance probe can call.
       const readArtifactObjects = (): Set<string> | null => {
         try {
-          const raw = JSON.parse(fs.readFileSync(opts.artifactPath, 'utf8'));
-          const meta = raw?.metadata ?? raw?.data?.metadata ?? raw;
-          const objects = Array.isArray(meta?.objects) ? meta.objects : [];
-          return new Set(
-            objects
-              .map((o: any) => o?.name)
-              .filter((n: any): n is string => typeof n === 'string'),
-          );
+          return new Set(artifactObjectNames(JSON.parse(fs.readFileSync(opts.artifactPath, 'utf8'))));
         } catch {
           return null;
         }

packages/cli/src/commands/serve.ts

@@ -63,6 +63,17 @@ import {
 } from '../utils/port-contract.js';
 import { BootLogCapture, isVerboseBootLevel } from '../utils/boot-log-capture.js';
 import { graftAuthoredRuntimeMembers, isAppPluginLike } from '../utils/graft-runtime-hooks.js';
+// [ADR-0130 D4 / option B, #15006] Every read below that keys off a
+// PACKAGE-OWNED collection goes through this seam, so an option-B artifact
+// (flattened top level gone, `packages[]` carrying everything once) reaches
+// the same decision — and so the acceptance probe can CALL the decision
+// instead of re-implementing it.
+import {
+  shouldAutoRegisterObjectQL,
+  shouldAutoRegisterStorageDriver,
+  stackDeclaresMetadata,
+  bundleDeclaresTranslations,
+} from '../utils/stack-collections.js';
 import { redactConnectionUrl, describeDriverConnection } from '../utils/connection-display.js';
 // The posture prose `os serve` and `os doctor` BOTH print, declared once
 // (#12492) — and, since #12579, the multi-org runtime SPELLING those two
@@ -2628,8 +2639,12 @@ export default class Serve extends Command {
       }
 
       // 1. Auto-register ObjectQL Plugin if objects define but plugins missing
-      const hasObjectQL = plugins.some((p: any) => p.name?.includes('objectql') || p.constructor?.name?.includes('ObjectQL'));
-      if (config.objects && !hasObjectQL) {
+      // [#15006] The whole gate — the `objects` read AND the already-composed
+      // check — is `shouldAutoRegisterObjectQL`. It answers exactly as the two
+      // inline expressions did for every stack that boots today, and resolves
+      // `packages[]` when the flattened top level is absent, which is the shape
+      // that used to boot with NO QUERY ENGINE and throw nothing.
+      if (shouldAutoRegisterObjectQL(config, plugins)) {
          try {
            const { ObjectQLPlugin } = await import('@objectstack/objectql');
            await kernel.use(new ObjectQLPlugin());
@@ -2661,12 +2676,9 @@ export default class Serve extends Command {
       // at boot through the datasource connection service, so building a
       // storage driver here would construct a duplicate pool the engine then
       // discards as already-registered.
-      const hasDriver = plugins.some((p: any) =>
-        p.name?.includes('driver') ||
-        p.constructor?.name?.includes('Driver') ||
-        p.name === 'com.objectstack.runtime.default-datasource' ||
-        p.constructor?.name === 'DefaultDatasourcePlugin');
-      if (!hasDriver && config.objects) {
+      // [#15006] Same seam, same reason — see the ObjectQL gate above. The
+      // driver-provider duck-typing moved into it with the read it guards.
+      if (shouldAutoRegisterStorageDriver(config, plugins)) {
          const databaseUrl = process.env.OS_DATABASE_URL;
          const driverType = resolveDriverType(process.env.OS_DATABASE_DRIVER, databaseUrl);
          // libSQL/Turso's credential is the only one that does NOT ride inside the
@@ -2799,9 +2811,10 @@ export default class Serve extends Command {
       // already holds an AppPlugin instance — and never on a named app, so it
       // is checked structurally below.
       const hasAppPluginAlready = plugins.some(isAppPluginLike);
-      const configHasMetadata = !!(
-        config.objects || config.manifest || config.apps || config.flows || config.apis
-      );
+      // [#15006] The same predicate `schema-migration-plugins.ts` runs after its
+      // own second `loadConfig` (B4) — folded into one seam rather than left as
+      // two copies whose comment already said they were the same.
+      const configHasMetadata = stackDeclaresMetadata(config);
 
       // ── Decide the dev-only artifact door BEFORE the wrap (#14397) ────
       // On a HOST config `os dev` composes TWO writers over ONE stack: the
@@ -2973,24 +2986,19 @@ export default class Serve extends Command {
       // `plugins` array — a host/aggregator config may define no translations
       // of its own and instead compose several `new AppPlugin(...)` entries,
       // each carrying its own. Keyed on that shape, not on a named app.
-      const pluginBundleHasTranslations = (bundle: any): boolean => {
-        if (!bundle || typeof bundle !== 'object') return false;
-        if (Array.isArray(bundle.translations) && bundle.translations.length > 0) return true;
-        if (bundle.i18n) return true;
-        if (bundle.manifest && (
-          (Array.isArray(bundle.manifest.translations) && bundle.manifest.translations.length > 0)
-          || bundle.manifest.i18n
-        )) return true;
-        return false;
-      };
+      // [#15006] `bundleDeclaresTranslations` is that same shape-keyed check plus
+      // the `packages[]` leg: `translations` is package-owned and `i18n` is an
+      // envelope key a translations-only stack never sets, so an option-B artifact
+      // reached this gate with neither and the REST i18n routes silently did not
+      // exist. MEASURED on the acceptance probe, not inferred.
       const anyAppPluginHasTranslations = plugins.some((p: any) => {
         if (!p) return false;
         // AppPlugin instances expose their bundle on `.bundle`
-        if (p.bundle && pluginBundleHasTranslations(p.bundle)) return true;
+        if (p.bundle && bundleDeclaresTranslations(p.bundle)) return true;
         return false;
       });
       const configHasTranslations = (
-        pluginBundleHasTranslations(config)
+        bundleDeclaresTranslations(config)
         || anyAppPluginHasTranslations
       );
       if (!hasI18nPlugin && configHasTranslations && tierEnabled('i18n')) {