Skip to content

docs(skills): objectstack-automation factual sweep — 11 false behavioral facts corrected - #13808

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-13793-skills-sweep-automation
Aug 31, 2026
Merged

docs(skills): objectstack-automation factual sweep — 11 false behavioral facts corrected#13808
os-zhuang merged 1 commit into
mainfrom
claude/issue-13793-skills-sweep-automation

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes #13793

Part of #13658 — flight 6 of the published-skills factual sweep (skills/objectstack-automation/**, 4 files, 1,163 lines).

Governed surface: this PR stays DRAFT for human merge, and needs:contract-review is attached to the PR and to card #13793 in the same stroke (clause 2 CONTENT limb, 批 #12). No self-clear.

Method

Every claim below was settled against the implementing codepackages/spec automation/data schemas, plugin-approvals, service-automation, trigger-api / trigger-schedule, @objectstack/lint — never against another document. Behaviour-bearing claims were settled by an executed probe (schema parse, CEL evaluation, schedule normalization) run against the freshly built @objectstack/spec and @objectstack/formula dists. Probe files lived in a scratch dir inside the worktree and were deleted before the gate derivation; nothing probe-related is in this diff.

Order followed the calibration comments: cross-file contradiction scan first, then parse-the-surface probes in both directions, then batched executable-oracle rows, with evals/** at full weight.

Corrections — landing site, before, after

11 distinct false facts across 12 landing sites.

# Landing site Before After Oracle
1 SKILL.md L134 a bare next: on a node "is ignored" "is refused" FlowNodeSchema is strictObject — executed parse returns Unrecognized key(s) on this flow node: next
2 SKILL.md L189-191 a top-level schedule key "is silently stripped and the flow never binds" "is a named parse error, not a silent strip" executed FlowSchema.parse returns the named error plus the guidance.schedule prescription (protocol-17 strict-unknown-keys, ADR-0078)
3 SKILL.md L479 org_membership_level = "the better-auth org-membership tier — value is one of owner/admin/member, and nothing else" "the org-membership tier — value is one of owner/admin/delegated_admin/member" ORG_MEMBERSHIP_LEVELS derives from BUILTIN_MEMBERSHIP_ROLES = 4 tiers (ADR-0108 / ADR-0105 D8); expandMembershipTierUsers filters sys_member on the tier verbatim, so delegated_admin routes. The spec source explicitly retires the "better-auth's closed set" attribution and the stale three-value copy
4 SKILL.md L484 queue listed as a working approver type ("A data-ownership queue") marked declared-but-never-resolved, do not author resolveApproverSpec has no queue branch — the slot warns and routes to nobody (#3508); the type is in NON_AUTHORABLE_APPROVER_TYPES and published as xEnumDeprecated so designers stop offering it
5 SKILL.md L604 minApprovals "Default 1" omitted means ALL resolvable approvers under quorum, 1 per group isApprovalSatisfied: quorum computes Math.min(Math.max(1, config.minApprovals ?? n), n) — the omitted default is n, not 1; only per_group is ?? 1. Its own doc comment reads "quorum — default = all"
6 SKILL.md L608 typed decisionOutputs declaration enumerated as { key, label?, type, multiple? } required? added to the enumeration DecisionOutputDefSchema declares required, and it is the one key of the four the runtime enforces (an approve carrying no value for a required key is rejected). The same file already teaches required: true in prose and in the os:check example, so the enumeration contradicted its own file
7 SKILL.md L660 without the queue capability every inbound POST "returns 503 queue_unavailable" "returns 503 SERVICE_UNAVAILABLE" phantom identifier: a repo-wide grep for queue_unavailable returns exactly one hit — this skill line itself. api-trigger.ts returns SERVICE_UNAVAILABLE (no queue registered) or ENQUEUE_FAILED
8 SKILL.md L692-698 Trigger Types table enumerates 6 tokens record-before-write / record-after-write row added triggerTypeToHookEvents accepts record-(before|after)-(create|insert|update|delete|write); write is the create-OR-update union (#3427) that binds BOTH hooks. @objectstack/lint prescribes it by name: "For 'created or updated' use record-after-write (one flow, both events)". The table hid a shipped capability
9 SKILL.md L721 OLD / NEW "were removed in M9.5 and now evaluate to null" "and now fault the predicate" executed CEL: OLD.status == null, OLD == null, NEW == null each abort with Unknown variable: OLD / : NEW. Strict CEL faults on an unbound root; it never yields null, so the old text taught a silent-false where the platform aborts loudly
10 SKILL.md L846 "the build does not catch these (conditions are checked, value expressions are not)" "(conditions are checked, call-position names are not)" same fact as 11
11 SKILL.md L964 "no validator implements it [the single-brace flow-template dialect]" "no validator checks its call names" packages/lint/src/validate-flow-template-paths.ts implements three build rules over that dialect (flow-template-unknown-field — an error inside a CRUD filter — plus -lookup-traversal and -field-unprovisioned), and flow-double-brace-interpolation / flow-bare-dollar-reference cover its other two misuses. What is genuinely unchecked is only the call-position name, which is exactly what the sentence needed to say
12 evals/README.md L11 the eval covers "the signal wait node" "the approval_revise window" cross-file contradiction found by the first-pass scan, then settled against the implementation: ADR-0044 D3's original wait prescription was reversed by its 2026-07-28 amendment (#3823). APPROVAL_REVISE_NODE_TYPE is registered by plugin-approvals with resumeAuthority: 'service', a revise edge into anything else is an authoring-time error, and sendBack refuses the metadata. SKILL.md and the eval file itself both already said approval_revise — only the index row still carried the retired prescription

Budget offset (in-list, not a correction)

The token ratchet is shrink-only and its own message requires new text to be paid for by genuine deletion in the same file. One deletion pays for the additions above:

Landing site Before After
SKILL.md L619-625, "Branching, side-effects & rejection" a 7-line restatement of the send-back model (revise label, approval_revise window, "not a plain wait", back-edge, DAG-cycle exclusion, maxRevisions default) a 3-line summary pointing at the full "Send-back for revision (ADR-0044)" section ~200 lines above

No fact is lost at package level: every deleted sentence is stated in that section and in the maxRevisions node-config row. It also removes the second copy of precisely the fact whose third copy — the eval index row — is item 12 above.

Inventory and density

Claim carriers, counted mechanically over the four files: 78 table/enumeration rows · 42 bullets · 49 numbered rules · 14 code fences, plus roughly 22 callout sentences. Excluding non-behavioural scaffolding (eval format list, contributing notes, planned-structure tree) the behavioural inventory is about 205 claims.

  • 11 FALSE distinct facts / 12 landing sites — about 5.4% / 5.9%, at the top of the program's 1.5-6% working range.
  • About 46 NOT MEASURABLE, by class and hand-counted: cross-package runtime needing a booted kernel (about 28 — approval suspend/resume, record lock, status mirror, the triggers-capability gating of trigger arming, the run-summary acted metric); CLI end-to-end verdicts (about 9 — what os validate / os build / os migrate meta --from 16 actually print on a scaffolded project); Studio/designer rendering (about 6 — which picker each xRef kind renders); cross-repo showcase-app pointers (about 3). Recorded, never silently skipped.
  • Everything else VERIFIED, with an executed probe wherever the claim is behaviour-bearing.

Non-vacuity control (claims proven true by execution)

  • normalizeSchedule('0 9 * * *') returns { type: 'cron', expression: '0 9 * * *' }, while the same call on the cron tagged template's envelope — { dialect: 'cron', source: '0 9 * * *' } — returns null, i.e. "no recognizable schedule descriptor, not bound". Both halves of L191-193 ("a bare cron string also works"; "do NOT use the cron tagged template, its envelope is not a recognized shape") confirmed in one run, in both directions.
  • FlowNodeAction.options.length === 20, matching L64 and the four node tables exactly.
  • TimeRelativeTriggerSchema refuses both offsetDays+withinDays together and neither of them, with the message "Provide exactly one of ..." — L771 confirmed in both error directions.
  • previous.status != 'escalated' && record.status == 'escalated' evaluates true against the trigger scope — L717 / L725-727.
  • The os:check-marked defineFlow example (L534-579) parses clean under FlowSchema, decision outputs and all.
  • Approval node config defaults observed post-parse: behavior: 'first_response', lockRecord: true, onEmptyApprovers: 'admin_rescue', maxRevisions: 3 — L604-611 confirmed row by row.

Also confirmed VERIFIED and left untouched, since each was a live falsehood candidate: the readonly-strip block (L166-176, matched against validate-readonly-flow-writes.ts error/warning split), the FSM introspection endpoint's overloaded next: null (L280-283, matched against the REST handler), the resume 403 for a service-owned pause (L455-459, PERMISSION_DENIED to 403 in the dispatcher's own mapping), all six named lint rule ids, maxRecords default 1000 and the daily 08:00 UTC sweep cadence, all four requires: capability tokens, and the node_modules/@objectstack/spec/src/** pointers (the package really does publish src/**/*.zod.ts, so these are not the dist-only phantom paths flight 1 found elsewhere).

references/_index.md is generator-owned and was not hand-edited; its sync is asserted by its owning gate below, not by inspection.

Verification

Gate families derived from the REAL diff in this fresh worktree, not recalled:
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack — 14 families, no STALE warning, and the run confirms "no path-derived mandate" for this surface.

Union re-run at final head e197523 (byte-identical tree to the earlier 1382e479 run):

node scripts/check-ci-filter-parity.mjs                              exit=0
node scripts/check-cross-package-test-inputs.mjs                     exit=0
node scripts/check-shard-attestation.mjs                             exit=0
node scripts/check-skills-token-ratchet.mjs                          exit=0
node scripts/check-test-completeness.mjs                             exit=3   NOT MEASURED
pnpm --filter @objectstack/lint run check:doc-formula-expressions    exit=0
pnpm check:agent-test-spelling                                       exit=0
pnpm check:corpus-claim-drift                                        exit=0
pnpm check:cross-package-test-inputs                                 exit=0
pnpm check:doc-authoring                                             exit=0
pnpm check:pm-governed-merges                                        exit=0
pnpm check:role-word                                                 exit=0
pnpm check:skill-compatibility                                       exit=0
pnpm check:skill-frame-sync                                          exit=0
pnpm --filter @objectstack/spec run check:skill-refs                 exit=0
pnpm --filter @objectstack/spec run check:skill-examples             exit=0
node scripts/check-nul-bytes.mjs                                     exit=0

check-test-completeness exit 3 is that gate's own PREREQUISITE-NOT-MET code — it grades a saved turbo run test log, this family names it with no argument, and it exits before parsing a single line. Recorded as NOT MEASURED, per the script's own instruction; it is not a red.

Two gates were red on their first run for a build prerequisite, not a finding, and both went green once the prerequisite was met: check:doc-formula-expressions needed @objectstack/lint built, and check:skill-examples needed @objectstack/client-react built. Both builds ran through the shared verify lock.

Gate readings quoted above are each gate's own verdict line with the exit code captured before any pipe.

Sync and population gates

  • check:skill-refs green: "9 generated files in sync with packages/spec" — references/_index.md needs no regeneration and was never hand-edited.
  • check:skill-examples green: "260 prose examples type-check across 3 surface(s)". Population for this package: exactly 1 os:check-marked fence (the defineFlow dynamic-approval example in SKILL.md); the other three files carry zero markers. This diff touches no marked fence — verified with git diff -U0 | grep os:check, which returns nothing — so the package's contribution to that population is unchanged and green.

Budget

before after delta
SKILL.md tokens 12,642 12,618 -24 (ceiling 12,643 untouched; headroom 1 to 25)
evals/README.md tokens 414 414 0 (ceiling 414, headroom 0, still exactly at it)
package authored tokens 13,062 13,038 -24
package lines (4 files) 1,163 1,159 -4

Net line budget across the package: -4, inside the "at most 0" constraint. No ratchet ceiling was raised or touched — the additions are paid for by the deletion listed above.

Changeset

None, and skip-changeset is applied at creation. Verified rather than assumed: the diff is pure skills/** and publishes nothing from any package, skip-changeset is a real mechanism in this repo (the changeset-check job in .github/workflows/pr-automation.yml exempts on it), and the three merged predecessor sweeps carry no changeset — for example 597020a (#13777) shipped one file, skills/objectstack-ui/SKILL.md, alone.

Deviations and notes

  • No separate claim comment on the card. The PM's claim comment (skills-sweep ⑥: objectstack-automation (1,163 lines, 4 files) — behavioral-claim verification, content-class execution-first #13793, 5478620644) already carries this seat's session id session_01EnE7G31tqbxN1rqpQmzurT and this branch, so a second comment would add no anti-duplication signal. Recorded here and in the report, as flight 5 did.
  • REST channel unavailable to this seat. A repo-scoped REST probe returned 403 "GitHub access is not enabled for this session", so card and program reads went through the zero-quota public-issue payload channel and one MCP issue_read get_comments (the payload channel serves only the first 15 timeline items and Published-skills factual sweep: verify every behavioral claim in skills/** against the implementation — program anchor #13658 has 33). Label writes therefore go through MCP with a read-back rather than the additive REST endpoint.
  • Serial constraint re-checked mid-flight: 13 open PRs at push time, none touching skills/objectstack-automation/**.
  • One claim was examined and deliberately left alone: http_request "survives only as a deprecation-window alias" (L96). The alias itself was removed — AutomationEngine no longer registers those executors — but the ADR-0087 conversion flow-node-http-callout-rename (toMajor 11) still rewrites the token to http inside canonicalizeStoredFlow, which registerFlow calls, so an authored http_request does still reach the http node. "Alias" versus "conversion" is a mechanism nuance rather than a behavioural falsehood, and confirming which one an author meets end to end needs the boot harness this flight does not have. Recorded as NOT MEASURABLE rather than rewritten on a guess.

Generated by Claude Code


Generated by Claude Code

…ral facts corrected

Flight 6 of the published-skills factual sweep. Every corrected claim was
settled against the implementing code (spec schemas, plugin-approvals,
service-automation, trigger-api / trigger-schedule, @objectstack/lint), never
against another document, with executed parse / CEL / schedule-normalization
probes where the claim is behaviour-bearing.

Net: -24 tokens, -4 lines across the package; the two ratcheted files that
moved stay under their existing ceilings (no ceiling touched).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT

Copy link
Copy Markdown
Collaborator Author

Contract review — PASS at head e197523c1

Applicability: clause-② CONTENT limb (批 #12) — falsifiable operator/contract semantic claims in published skills/**. Both carriers attached same-stroke at PR creation and read back twice on independent paths; the double pre-write read that caught the labeler's concurrent documentation add is the RMW discipline earning its keep.

Reviewer qualification, machine-read this sitting: get_sessionexternal_metadata.last_served_model = claude-fable-5 = CONTRACT_REVIEW_TIER. Dispatching-seat review permitted per the 2026-08-21 relaxation.

Review: report + diff read; anchors independently re-measured on origin/main this hour —

  • packages/plugins/plugin-approvals/src/approval-service.ts:2295/4857: quorum need defaults ?? n / ?? total (ALL resolvable approvers); only per_group is ?? 1 — the skill's "Default 1" was the single most dangerous falsehood in the package, and the correction matches the runtime exactly;
  • packages/spec/src/automation/approval.zod.ts:717–718: the spec-side .describe('… Default 1') really does publish the same false default — the out-of-scope filing spec: ApprovalNodeConfigSchema.minApprovals describes "Default 1", but the quorum runtime defaults to ALL resolvable approvers #13809 has a true premise and is correctly routed for central triage (spec territory, options honestly split between text-fix and a semantic .default(1) needing a maintainer ruling);
  • ORG_MEMBERSHIP_LEVELS contains delegated_admin with the ADR-0105 D8 pin (approval.test.ts:91–92) — the 3-tier claim under-enumerated;
  • record-before-write / record-after-write are live create-OR-update union triggers (trigger-record-change README table + integration tests) — the 6-row table was hiding shipped capability;
  • The schedule/bare-next: protocol-17 refusals were settled by executed parse probes quoting the exact "Unrecognized key(s)" prescriptions — consistent with the strict-unknown-keys regime verified in flights ④/⑤.

The mid-flight check:role-word red (the fix's first draft introducing a reserved-word occurrence, caught and removed) is the ratchet working as designed, recorded not hidden. Budget: net −4 lines / −24 tokens, ceilings untouched; the budget-offset deletion (7-line restatement → 3-line pointer) removes a redundant copy of exactly the fact whose eval-index copy this PR corrects — package-level fact count preserved.

needs:contract-review cleared on BOTH carriers in this stroke. Governed .md content: PR remains DRAFT for the maintainer's merge (os-zhuang / hotlong requested); the review chain never enqueues a governed surface.


Generated by Claude Code

@os-zhuang
os-zhuang marked this pull request as ready for review August 31, 2026 13:46
@os-zhuang
os-zhuang added this pull request to the merge queue Aug 31, 2026
Merged via the queue into main with commit 16c3601 Aug 31, 2026
40 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-13793-skills-sweep-automation branch August 31, 2026 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

skills-sweep ⑥: objectstack-automation (1,163 lines, 4 files) — behavioral-claim verification, content-class execution-first

3 participants