docs(skills): objectstack-ai factual sweep — 11 false behavioral facts corrected

[huangyiirene]

Fixes #13841

Part of #13658 — flight ⑩ of the published-skills factual sweep (skills/objectstack-ai, 3 files, 691 lines).

Every corrected claim was settled against the implementing code, never against another document, with executed parse probes wherever the claim is behaviour-bearing. Governed surface: this PR stays DRAFT for human merge.

---

Result

11 distinct false facts across 21 landing sites. (The branch's single commit message says 9 — the count as it stood mid-sweep, before F9 / F10 / F11 were settled. This body carries the final tally; the branch is not force-pushed to repair a message.) Package inventory: ~255 behavioral claims across the three files (SKILL.md 602 · references/_index.md 45, generator-owned · evals/README.md 44) — 4.3% distinct / 8.2% by landing site, inside the 1.5–10% working range set after flight ⑧.

~59 claims recorded NOT MEASURABLE (the flight-④ ai:routes class — the whole in-product AI runtime is out-of-repo: git ls-files | grep -ic service-ai returns 0 in this repo). None silently skipped; they are listed at the bottom.

Net: −15 tokens, −10 lines. Both ratcheted files stay under their existing ceilings; no ceiling touched.

The three standing probe classes all fired: enumeration-vs-schema-member-set (F1, F2, F7), cross-file contradiction (F5 — the document contradicted itself 12 lines apart, and the implementation settled it), and the wrong-default class (F6). One correction has a spec-side twin, filed rather than fixed here (see Spin-off).

---

Per-item: landing site | before | after

#	Landing site	Before	After	Settled against
F1a	SKILL.md tier diagram	"Capability bundle with instructions & trigger phrases"	"… & trigger conditions"	skill.zod.ts — triggerPhrases is a retiredKey tombstone
F1b	Skill "Important Optional Properties" table	row `triggerPhrases` — "Natural language phrases that activate the skill"	row deleted	probe: authoring it is a parse error
F1c	Pitfall 5	"Not testing trigger phrases. Ambiguous trigger phrases cause skill conflicts…"	"Expecting a phrase to activate a skill. Nothing matches phrases; write routing as triggerConditions…"	same
F1d	App AI Blueprint table	"defineSkill — trigger phrases + trigger conditions + bounded toolset"	"defineSkill — instructions + trigger conditions + bounded toolset"	same
F1e	evals/README.md planned structure	skills/test-trigger-phrases.md	skills/test-trigger-conditions.md	same
F2	Agent "Important Optional Properties" table	row `tools` — "Direct tool references — legacy fallback"	struck row: "REMOVED in protocol 17 — the inline slot resolved names against the full registry with no surface check. A parse error now; move each reference into a skill"	agent.zod.ts — retiredKey; probe rejects
F3	Whole section "Inline Agent tools[] (legacy)"	taught AIToolSchema as the live shape of that slot	section deleted	AIToolSchema was removed with agent.tools (#3894); zero live references in the tree
F4	Skill permissions callout	"removed in 16.x"	"removed in 17.0.0"	packages/spec/CHANGELOG.md, under the 17.0.0 heading: "remove SkillSchema.permissions — it never gated anything (#3686)". It is tool.requiresConfirmation that went in the 16.x line
F5a	Built-in skills table, Edition column	schema_reader / data_explorer / actions_executor marked OSS	column removed	PLATFORM_TOOLS_BY_PACKAGE puts every one of those tools in a cloud package (service-ai, service-ai-studio)
F5b	Callout under that table	"The open single-env framework ships only the ask agent… a build-intent turn on OSS degrades gracefully"	"The open framework ships no in-product agent (there is no service-ai package)… what a skill DOES serve on the open edition is its instructions, projected onto the MCP prompts primitive"	this document's own edition box 90 lines earlier ("On the open edition there is no in-product agent"); ADR-0063's cloud-owned banner; packages/mcp/src/skill-prompts.ts
F6a	Section heading + lead paragraph	"### Auto-Exposed Actions … Every Action you attach … is auto-exposed as a tool"	"### Actions as AI Tools — opt-in … becomes a tool only when it opts in — ai.exposed: true (default false) plus an ai.description of at least 40 chars"	ActionAiSchema.exposed is .default(false); ADR-0011 §2 "opt-in gate. Skips unless ai.exposed === true"
F6b	Skip list header	"Skipped automatically:"	"Skipped even when opted in:"	same
F7	Skip list bullet	"Owner opt-outs (aiExposed: false)"	bullet deleted; the lead paragraph now says "there is no opt-out key"	ADR-0011: "The old flat aiExposed boolean is removed" (clean break). Probe rejects it at both the action and the ai block. The live aiExposed is on KnowledgeSource only — untouched
F8	Dangerous-variants bullet	"(confirmText set, mode: 'delete', variant: 'danger')"	"the declared mode: 'delete' / variant: 'danger' only (confirmText is dialog copy, not a destructive signal; ai.requiresConfirmation overrides either way)"	actionLooksDestructive in packages/runtime/src/action-execution.ts — maintainer ruling #7828, Option A; probe both directions
F9	Tool "gate what a tool can do" paragraph	"(action.ai.category — a live, enforced surface)"	"(action.ai.category — live, but listing-only: carried onto the tool, never sent to the model)"	AIToolDefinition.category in packages/spec/src/contracts/ai-service.ts: "Surfaced by tool-listing routes. Not sent to the model."
F10a-c	Agent optional-properties table	guardrails / structuredOutput / memory presented as working config	each row suffixed "— declared only"	the schema marks all three [EXPERIMENTAL — not enforced] Parsed but no runtime consumer yet; packages/spec/liveness/agent.json records "no runtime reader" for each
F10d	Agent example	// token budget per invocation · // wall-clock cap per invocation (asserting caps)	one comment on the block: // declared only — nothing reads these yet	same
F10e	Pitfall 3	"Missing guardrails and approval gates. Define blockedTopics (plus the token / time budgets) in agent guardrails" — prescribing an unenforced control as the remedy	"Mistaking guardrails for a gate. guardrails / memory / structuredOutput are declared only — no runtime reads them, and real limits come from the quota service." The three enforced gates it already listed are kept	same; guardrails' own describe: "real limits come from the quota service"
F11	evals/README.md planned structure	tools/test-requires-confirmation.md — an eval planned for a key that no longer parses	tools/test-strict-unknown-keys.md	tool.requiresConfirmation is in TOOL_RETIRED_KEY_GUIDANCE; probe rejects it

Zero out-of-list changes. The diff is exactly these sites.

---

Executed evidence

Two probe files, run against the built dist (spec + runtime dependency closure rebuilt first, so nothing was read off a stale .d.ts).

Probe 1 — both error directions through the published schemas (27 assertions)

Non-vacuity controls first — the three happy paths must PARSE, and did:

OK   CONTROL SkillSchema accepts a minimal valid skill   ->  parsed
OK   CONTROL AgentSchema accepts a minimal valid agent   ->  parsed
OK   CONTROL ToolSchema accepts a minimal valid tool     ->  parsed

Then the retired-key direction — every key this document taught as authorable:

OK   skill.triggerPhrases is REJECTED   ->  "`skill.triggerPhrases` was removed in @objectstack/spec 17.0.0 (audit close-out)
                                             — phrases were never matched against the user's message…"
OK   agent.tools is REJECTED            ->  "`agent.tools` was removed in @objectstack/spec 17 — use `skills`…"
OK   skill.permissions is REJECTED      ->  "Unrecognized key(s) on this skill: `permissions`…"
OK   tool.category / tool.permissions / tool.active / tool.builtIn / tool.requiresConfirmation — all 5 REJECTED
OK   agent.knowledge is REJECTED · agent.temperature (top level) is REJECTED

And the defaults / member sets the document asserts:

OK   skill.surface default 'ask' · skill.active default true
OK   agent.planning.maxIterations default 10 · agent.surface default 'ask'
OK   structuredOutput.strict false · maxRetries 3 · retryOnValidationFailure true · no `retry` object
OK   knowledge refresh.onRecordChange default true · aiExposed default true
OK   trigger operator set == the doc's table rows  ->  ["eq","neq","in","not_in","contains"]

Probe 2 — the action-exposure and destructive-signal claims (12 assertions, 12 pass / 0 fail)

OK   inline agent model.provider accepts exactly the narrow set     ->  ["openai","azure_openai","anthropic","local"]
OK   ModelProviderSchema accepts narrow + google/cohere/huggingface/custom
OK   a plain action carries NO ai block (nothing auto-exposes it)   ->  ai=undefined
OK   action.ai.exposed default is false (opt-IN)                    ->  ai.exposed=false
OK   ai.exposed:true without ai.description is REFUSED at ActionSchema
       ->  "ai.description is required (at least 40 chars) when ai.exposed is true."
OK   confirmText alone does NOT make an action destructive          ->  actionLooksDestructive(confirmText) = false
OK   mode:'delete' IS a destructive signal                          ->  true
OK   variant:'danger' IS a destructive signal                       ->  true
OK   ai.requiresConfirmation overrides the heuristic                ->  false
OK   MCP tool-binding approval accepts 'always'; default is 'never'

Both enum member-set assertions are read by parsing each candidate, not by introspecting a Zod internal — a first pass that reached into .def.entries returned undefined through the lazySchema proxy and would have read as a finding. That was an instrument fault, corrected, not a result.

Probe 3 — the capability F5b adds, proven by execution (open edition, in-repo)

pnpm --filter @objectstack/mcp exec vitest run --maxWorkers=2 src/skill-prompts.test.ts
  Test Files  1 passed (1)
       Tests  12 passed (12)

including keeps the skill name, label and description, drops the tool-binding half — it is cloud-runtime-only, does not project an inactive skill, and does not project a skill without instructions — the four facts the new callout states.

---

Gates

Families derived from the real diff by node scripts/pm/dispatch-gates.mjs (no path argument — the script takes its own change set from the merge base; exit 0; it names the tree it answered for, objectstack-ai/objectstack at 6e6d02bd1). 14 families, harvested with --commands so neither spelling is dropped. Exit code captured with a redirect before any pipe.

Gate	Exit	Verdict line
node scripts/check-ci-filter-parity.mjs	0	green
node scripts/check-cross-package-test-inputs.mjs	0	green
node scripts/check-shard-attestation.mjs	0	green
node scripts/check-skills-token-ratchet.mjs	0	"38 authored bundle file(s) within their ceilings; 11 generator-owned file(s) measured, not ratcheted"
node scripts/check-test-completeness.mjs	3	NOT MEASURED — the gate itself prints "PREREQUISITE NOT MET … the local reading for this gate is NOT MEASURED. ⛔ It is not a red". It grades a saved turbo run test log that only CI produces
pnpm --filter @objectstack/lint run check:doc-formula-expressions	0	green
pnpm check:agent-test-spelling	0	green
pnpm check:corpus-claim-drift	0	green
pnpm check:cross-package-test-inputs	0	green
pnpm check:doc-authoring	0	green
pnpm check:pm-governed-merges	0	green
pnpm check:role-word	0	green
pnpm check:skill-compatibility	0	green
pnpm check:skill-frame-sync	0	green

Run beyond the derived list, because the diff edits prose inside an os:check fence (the agent example) and the flight-③ calibration makes the population rule binding:

Gate	Exit	Verdict line
pnpm --filter @objectstack/spec run check:skill-examples	0	"✅ 260 prose examples type-check across 3 surface(s) — every marked block parsed, so tsc ran the SEMANTIC pass on all of them"
node scripts/check-nul-bytes.mjs	0	"OK (scanned 7601 text file(s) … no raw ASCII control bytes)"

check:skill-examples population control (the rule from #13658 comment 5476328676): the population is non-empty for this surface — the gate reports skills + docs (@objectstack/spec): 227 block(s) of 260 total, and this package contributes 4 os:check fences to it. Not an empty-population reading. The first invocation exited 1 with "packages/client-react/dist holds no .d.ts declarations — the package is not built"; that is a refused verdict, not a finding, and it was cleared by building the workspace (turbo run build --concurrency=2, 70/70 successful) and re-running.

Ratchet, before and after

before:  skills/objectstack-ai/SKILL.md          6806 / 6806   (headroom 0)
         skills/objectstack-ai/evals/README.md    315 /  315   (headroom 0)
after:   skills/objectstack-ai/SKILL.md          6791 / 6806   (headroom 15)
         skills/objectstack-ai/evals/README.md    315 /  315   (headroom 0)

references/_index.md (796 tokens) is generator-owned — measured, not ratcheted, and not hand-edited; check:skill-frame-sync is green, so it is in sync with gen:skill-refs. Package delta −15 tokens / −10 lines; net line budget satisfied (≤ 0). Ceilings are the maintainer's and none was moved — the same posture the eight preceding flights landed with.

Repo-wide ESLint: a declared narrowing, with the measurement

pnpm lint (eslint . --no-inline-config) is CI's run. Locally the check was narrowed to the changed files, and the narrowing is measured, not assumed — all three legs:

1. The population is read from eslint's own config resolution, not from a guess about which files count: npx eslint --no-inline-config --format json on both changed paths returns, for each, File ignored because no matching configuration was supplied.
2. File count from the JSON output: 2 results, 0 errors, 0 lint warnings (2 ignore notices).
3. Invariance for untouched files: this repo's flat config supplies no matcher for .md at all, so a Markdown-only diff cannot move any judgment on any file eslint does lint. No type-aware linting is reachable for a file with no matching config.

skip-changeset applies and the label is attached at creation: the diff is two skills/** Markdown files and publishes nothing from any package — the same shape as the eight preceding sweep PRs, none of which carried a changeset.

---

NOT MEASURABLE — recorded, not skipped

The in-product AI runtime is out of this repo. git ls-files | grep -ic service-ai returns 0; packages/services/ holds 16 members and none of them is service-ai; every agent / skill liveness-ledger consumer is attributed cloud: packages/service-ai/…. So ~59 claims about that runtime are recorded NOT MEASURABLE rather than false — the flight-④ ai:routes boundary. Grouped:

• The ask / build runtime behaviour: surface binding, the resolution chain through app.defaultAgent, "no per-turn intent classifier", the decline-and-redirect, the data_chat / metadata_assistant alias resolution, and the identifier ASK_AGENT_NAME (zero code references in this tree; its only in-repo mention is an unchecked follow-up item in ADR-0063, so absence here is not evidence about cloud).
• resolveActiveSkills raising a fast load error on an incompatible binding, and the tool-set union with no global fall-through. Corroborated one level down — packages/lint/src/validate-ai-surface-affinity.ts sets ERROR severity because resolveActiveSkills() throws — but the runtime half is not readable here.
• Which built-in skill owns which tool. The tool names are verified against PLATFORM_TOOLS_BY_PACKAGE; the skill-to-tool assignment lives in cloud. Worth a note for a later flight: search_knowledge is in that registry and is referenced by this document's Knowledge section, yet no row of the built-in-skills table claims it. Not correctable from here without inventing an owner.
• The visualize_data callout (auto-registered only when an IAnalyticsService is wired).
• The Ops callout: AI_DAILY_USER_MESSAGES, the ai_usage_daily object, GET /api/v1/ai/status. All three have zero references anywhere in this tree — but so does ai_pending_actions, which the open repo's own IAIService contract does describe, so absence from the object-name registries is not evidence either way.
• The HITL section end to end: registerActionsAsTools, AIServicePlugin options, enableActionApproval, the pending-actions REST endpoints, actionSkipReason / actionRequiresApproval. The IAIService half is verified here — proposePendingAction / approvePendingAction / rejectPendingAction / listPendingActions all exist and are all optional, exactly as stated.

Verified-and-unchanged, worth naming because they are the claims most likely to be assumed stale: the whole KnowledgeSource table and its three source kinds (complete against the schema's member set — 9 rows covering all 10 members, label and description sharing one), both model-provider enums, the trigger-condition operator set, every structured-output default, defineStack's AI collections (agents / tools / skills, with no knowledge-source collection), the agent metadata type's allowRuntimeCreate: false, allowOrgOverride: false, action.requiredPermissions (ADR-0066), IKnowledgeService.registerSource / registerAdapter with the memory and ragflow adapter ids, the type:'api' body-assembly order, and the frontmatter's Requires @objectstack/spec 17.x (this tree is 17.2.0 on Zod ^4.4.3).

Spin-off

#13865 — ActionAiSchema.requiresConfirmation's JSDoc in packages/spec/src/ui/action.zod.ts still names confirmText as a destructive signal, contradicting actionLooksDestructive and ruling #7828. This is the spec-side twin of F8: fixing only the skill would leave the same false fact published one surface over. Filed unassigned, single-site drift (the two other in-repo carriers already state the corrected rule).

---

needs:contract-review is attached to this PR and to card #13841 in the same stroke as creation (clause ② CONTENT limb, 批 #12); this seat does not self-clear it.

Generated by Claude Code

---

Generated by Claude Code

[huangyiirene]

Contract review — PASS at head 6e6d02bd1

Applicability: clause-② CONTENT limb (批 #12). Both carriers attached same-stroke at creation, read back both; cleared in this stroke.

Reviewer qualification, machine-read this sitting: get_session → external_metadata.last_served_model = claude-fable-5 = CONTRACT_REVIEW_TIER.

Review: report + diff read; anchors independently re-measured on origin/main this hour, all verbatim —

• packages/spec/src/ui/action.zod.ts:729: exposed: z.boolean().default(false) with :1518's exact refine message ("ai.description is required (≥40 chars) when ai.exposed is true.") — the taught "every Action is auto-exposed" was false in the dangerous direction (an author trusting it ships nothing to AI and doesn't know why);
• packages/spec/src/ai/agent.zod.ts:236: "agent.tools was removed in @objectstack/spec 17 — use skills" — the six retired-key landing sites were teaching parse errors as live authoring surface;
• packages/mcp/src/skill-prompts.ts + its 12-case test exist — the hidden shipped capability (skill instructions projected onto the MCP prompts primitive) is real and now documented;
• packages/runtime/src/action-execution.ts:885–888: actionLooksDestructive = ai.requiresConfirmation override, else mode==='delete' || variant==='danger' — no confirmText leg, matching ruling actionLooksDestructive keys destructiveness off confirmText — the exact signal #7278/#7309 are migrating away, and 6 of 14 actions flip #7828; and ActionAiSchema.requiresConfirmation docblock still names confirmText as a destructive signal — the #7828 ruling removed that leg #13865's spec-JSDoc-twin premise holds (the twin rule applied exactly as bound).

Instrument discipline worth the record: the probe-1 Zod-internal introspection faults were disclosed, diagnosed as instrument error, and re-run as parse-based probes — none laundered into findings. The releases-page check (agent.tools "tombstoned and stripped" is true of the LOAD path via retiredFromLoadPath) correctly produced no card and no releases edit. ~59 NOT MEASURABLE recorded under the flight-④ cloud-boundary rule, listed not skipped.

Budget: net −10 lines / −15 tokens, no ceiling touched; generator-owned index untouched, frame-sync green. needs:contract-review cleared on BOTH carriers. Governed .md: PR remains DRAFT for the maintainer's merge (os-zhuang / hotlong requested).

---

Generated by Claude Code