From 3ee781639faa1540ed2b029b2cc3c3171c2dc0c4 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Tue, 10 Nov 2026 21:05:31 -0500 Subject: [PATCH] docs(agent-context): make the audit skill check that a gate blocks and that adopted artifacts are used The metaobjects-audit skill asked whether a verify gate was wired, never whether it blocked, and it did not join unused generated artifacts to the hand-written pages that replaced them. Add the checks and the remedy for each gap: - Axis F: does the gate block (exit code propagates, green now with a ratchet for a tolerated count, every deploy path passes it); score BLOCKING / RUNS BUT DOES NOT BLOCK / ABSENT; no CI is NO whatever untracked hooks exist, with a per-port snippet. - Axis F2: consumed / emitted per generator kind, and a generated-and-bypassed finding. - Prompts: unlinked payload fields, linked / (linked + unlinked copies). - Requirements: ledger staleness from git, --require-implementers, unwired implementers. - Owned-copy check in the Java, Kotlin, C# and Python references, one total. - Phase 1b shape-coverage census; scorecard 'used as described' line; re-runs mark earlier findings fixed / open / deferred; JSON top-level fields documented. Conformance fixture copies of the skill are updated to match. --- CHANGELOG.md | 16 ++++ .../skills/metaobjects-audit/SKILL.md | 82 ++++++++++++++++--- .../metaobjects-audit/references/csharp.md | 18 +++- .../metaobjects-audit/references/java.md | 27 +++++- .../metaobjects-audit/references/kotlin.md | 27 +++++- .../references/prompt-anti-patterns.md | 20 ++++- .../metaobjects-audit/references/python.md | 19 +++++ .../references/requirements.md | 37 ++++++++- .../references/surface-review-axes.md | 70 +++++++++++++++- .../references/typescript.md | 5 ++ .../.claude/skills/metaobjects-audit/SKILL.md | 82 ++++++++++++++++--- .../metaobjects-audit/references/java.md | 27 +++++- .../metaobjects-audit/references/kotlin.md | 27 +++++- .../references/prompt-anti-patterns.md | 20 ++++- .../references/surface-review-axes.md | 70 +++++++++++++++- .../.claude/skills/metaobjects-audit/SKILL.md | 82 ++++++++++++++++--- .../metaobjects-audit/references/java.md | 27 +++++- .../references/prompt-anti-patterns.md | 20 ++++- .../references/surface-review-axes.md | 70 +++++++++++++++- .../.claude/skills/metaobjects-audit/SKILL.md | 82 ++++++++++++++++--- .../references/prompt-anti-patterns.md | 20 ++++- .../metaobjects-audit/references/python.md | 19 +++++ .../references/surface-review-axes.md | 70 +++++++++++++++- .../.claude/skills/metaobjects-audit/SKILL.md | 82 ++++++++++++++++--- .../references/prompt-anti-patterns.md | 20 ++++- .../references/surface-review-axes.md | 70 +++++++++++++++- .../references/typescript.md | 5 ++ .../.claude/skills/metaobjects-audit/SKILL.md | 82 ++++++++++++++++--- .../references/prompt-anti-patterns.md | 20 ++++- .../references/requirements.md | 37 ++++++++- .../references/surface-review-axes.md | 70 +++++++++++++++- .../references/typescript.md | 5 ++ 32 files changed, 1227 insertions(+), 101 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7e9e8f648..f5292f24a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -298,6 +298,22 @@ loading. A port that reports `1.2` does not know that vocabulary; upgrade every the shipped per-port matrix (consume, check, publish, runtime load, transports), the deferred list names what is still not built, and the verify, codegen, authoring, audit and capability-sheet skills teach the Java, Kotlin and C# publish paths, the `deps` checks and the `maven` and `nuget` transports. +- **The `metaobjects-audit` skill checks that a project uses MetaObjects as documented, not only + that it installed it.** Axis F now asks whether the verify gate blocks: it looks for swallowed + exit codes, runs the project's own verify and records the error count (recommending a ratchet + for a tolerated count), lists every deploy path that skips it, and scores the gate BLOCKING, + RUNS BUT DOES NOT BLOCK or ABSENT; a project with no CI answers NO whatever untracked hooks it + has. Axis F2 reports `consumed / emitted` per generator kind and raises one + `generated-and-bypassed` finding where a hand-built page stands beside an unused generated one. + The prompt reference counts payload fields linked to entity fields (`extends: Entity.field`) + against unlinked copies. The requirements reference adds three currency checks: ledger + staleness from git history, `--require-implementers`, and live claims whose generated + implementer nothing uses. The unmodified-owned-copy check now covers Java, Kotlin, C# and + Python (the Java and Kotlin references no longer say those ports cannot eject), reported as one + line total. Phase 1b adds a shape-coverage census of every boundary shape. The scorecard gains a + "used as described" line, a re-run marks each earlier finding fixed, open or deferred against + the previous `.metaobjects/adoption-audit.json`, and the report documents the JSON's top-level + fields. ### Changed diff --git a/agent-context/skills/metaobjects-audit/SKILL.md b/agent-context/skills/metaobjects-audit/SKILL.md index 525029b69..5225faad9 100644 --- a/agent-context/skills/metaobjects-audit/SKILL.md +++ b/agent-context/skills/metaobjects-audit/SKILL.md @@ -74,7 +74,13 @@ Actual cutovers run through the existing skills mapped per finding tier (§ Brid `queriesFile` / `routesFile` / `barrel` on a 1.0.x install** fails to load, because 1.0 removed those four and 1.1 restored them: there it is an upgrade blocker, fixed by upgrading to 1.1 or by ejecting. Importing anything from that subpath is otherwise - correct and is never a finding by itself. + correct and is never a finding by itself. **Every port owns generators the same way**: + run the owned-copy check in each port reference the project uses (TypeScript + `meta eject --list --format text`, Python `metaobjects gen --list`, Java / Kotlin + `mvn metaobjects:eject -Dlist`, C# `dotnet meta gen --list`), and report ONE total across + all of them: **unmodified owned-generator lines / all owned-generator lines**, where + unmodified counts only copies proven unmodified. Report the proven copies together as + one finding, `owned-generator-stock-copies`, listing each copy and the port's undo command. - [ ] **Cross-language version consistency (silent-drift check).** If the project uses MetaObjects in more than one language (e.g. a TS web client + a Java/Python/C# backend), enumerate EVERY MetaObjects package across ALL ecosystems (npm `@metaobjectsdev/*`, Maven `com.metaobjects:*`, PyPI `metaobjects`, NuGet `MetaObjects.*`) and record each version. **The version-number LINES differ by ecosystem (npm/PyPI/NuGet `0.x`/`1.x` vs Maven `7.x`/`8.x`), so you CANNOT eyeball drift** — a `0.12` next to a `7.7` looks fine but can be badly out of sync. Compare the **`metamodelVersion`** each port reports (the shared spec version on the registry manifest): a mismatch is real cross-language drift and a **finding** — the ports disagree on vocabulary/wire behavior. Also flag any port not on the latest release for its ecosystem. (This is a known real-world failure mode: newest backend, stale client, invisible because the numbers differ.) - [ ] Classify: **Greenfield** (none/minimal) · **Partial** · **Deep** → choose path below. @@ -103,6 +109,20 @@ code grows. The gap between "modeled + query helpers" and "has view + route + UI" is the headline lopsidedness. +**Shape coverage census.** The coverage matrix lists modelled entities only, so a shape +nobody modelled never appears in it. Count the other side too. List every shape that crosses +a boundary: database tables and views, request and response bodies, queue and socket +messages, prompt inputs, LLM replies, and file formats the project reads or writes. For each +shape, record whether it is modelled, and for each layer that states it (DDL or migration, +ORM model, validator, DTO, client type, prompt payload, reply parser) whether that statement +is **generated**, **derived** from generated code (a type built from a generated type, a +`z.infer` of a generated schema), or **hand-declared**. Report +`generated or derived statements / all statements` and list the unmodelled shapes. A table +present in the database or the migrations with no entity or projection behind it is a +finding (`unmodelled-table`). Like the leverage ratio this is a measurement, never a target: +an unmodelled shape is a finding only where the cost test in § Owned-codegen says modelling it +pays, or it is a table. + **Surface review — 8 axes (independently runnable).** Work the full `references/capability-checklist.md` on every axis. Check calibration guards (`references/calibration.md`) before raising a finding. **Verify, don't assume** — read the @@ -113,7 +133,7 @@ Axes (the full hunt for each is in `references/surface-review-axes.md` — open - **A.** Codegen candidates — API / server routes · **B.** Codegen candidates — web / client - **C.** Drift hotspot — validators, mappers, runtime models · **D.** Prompt pillar - **E.** Owned generators and scaffold-and-own · **E2.** Capability ledger (`requirement.*`, opt-in) -- **F.** Drift-gate adoption · **F2.** Over-generation (codegen waste) +- **F.** Drift-gate adoption — does it block? · **F2.** Over-generation and generated-but-bypassed (codegen waste) - **G.** Runtime-contract anti-patterns · **G2.** `own*()` accessor discipline (a correctness defect, not advisory) - **H.** Authoring correctness / ADR conformance · **H2.** Wrong native type (`field.string` + `@dbColumnType: uuid`) - **I.** Vocabulary hygiene / modernization (advisory, never a failing finding) @@ -199,14 +219,36 @@ one-consumer need, read it codegen-locally. 1. **Headline MATURITY TIER** — Greenfield → Partial → Deep → Exemplary; worst-of with prerequisite gating (a missing pillar can't be averaged away); rendered with **the single - next unmet check** ("you're Partial; the next rung needs `verify` in CI"). + next unmet check** ("you're Partial; the next rung needs a blocking `verify` in CI"). 2. **Per-pillar breakdown (never rolled into one number)** — `pillar | tier | top gap` over codegen / runtime / drift-gate / prompts. This is the core deliverable. -3. **Binary CI drift gate** — prominent and separate: **"Is `meta verify` drift detection - wired into CI?"** It is binary because the risk is binary. - -Coarse bands only (none / some / most / all). Worst-of within a pillar. On re-run, grade -the delta. Lead with gaps, not the grade. +3. **Binary CI drift gate** — prominent and separate: **"Does `meta verify` drift detection + block every route to production?"** It is binary because the risk is binary. YES only + when axis F scores the gate **BLOCKING**; RUNS BUT DOES NOT BLOCK and ABSENT both answer + NO, with the reason (exit code swallowed, tolerated error count, a deploy path that + skips it, no CI). Local hooks with no tracked installer do not count. +4. **Used as described** — one line, separate from the tier: does the project use what it + adopted the way MetaObjects documents it? Built from four measurements, each shown with + its numbers: generated artifacts consumed rather than bypassed (axis F2, `consumed / + emitted` and the `generated-and-bypassed` count); the gate blocks (axis F score); prompt + payload fields linked to entity fields (axis D, `linked / (linked + unlinked copies)`); + and boundary shapes generated or derived (Phase 1b shape coverage census). Render it as + `Used as described: — generated forms 5/12 consumed · gate RUNS BUT DOES NOT + BLOCK · payload fields 6/10 linked · shapes 50% generated or derived`, banded by its worst + part. + +Coarse bands only (none / some / most / all). Worst-of within a pillar. Lead with gaps, not +the grade. + +**Re-run: compare against the previous audit.** If `.metaobjects/adoption-audit.json` +already exists, read it before starting and keep it as the baseline. Match each earlier +finding to this run's by `id` plus location, and give every earlier finding a `status`: +**fixed** (the location no longer shows it; say what changed), **open** (still present, +carrying its original `first_seen`), or **deferred** (the project recorded a decision not to +fix it now, in a commit, an ADR, a tracked issue or a code comment, quoted as +`deferred_reason`). A finding that is still present with no recorded decision is open, never +deferred. New findings are `new`. Report the counts per status at the top of the report, +and show each scorecard line beside its previous value. **Vocabulary hygiene (axis I) is advisory** — it surfaces as modernization opportunities in the roadmap, scored as such, and **never gates a tier or fails the @@ -218,8 +260,10 @@ audit.** Dated-but-working vocabulary is a quality nudge, not a defect. **Two artifacts:** `.metaobjects/adoption-audit.json` + rendered Markdown. -**Markdown sections (lead with Scorecard):** 0. Scorecard (tier + pillar table + CI gate) · -1. Triage + census · 2. Coverage matrix · 3. Per-surface classification tables · 4. Drift +**Markdown sections (lead with Scorecard):** 0. Scorecard (tier + pillar table + CI gate + +used as described; on a re-run, the fixed / open / deferred / new counts) · +1. Triage + census (including the owned-generator unmodified-lines total and the shape +coverage census) · 2. Coverage matrix · 3. Per-surface classification tables · 4. Drift findings (active bugs first) · 5. Owned-codegen + generator gaps · 6. Drift-gate adoption · 7. Runtime-contract + authoring-correctness (axes G+H) · 8. Semantic-constraint ratification · 9. Prompt-pillar assessment · 10. Prioritized roadmap: Tier 1 drift kill → Tier 2 existing @@ -243,6 +287,24 @@ Each roadmap item: LOC retired, prerequisite, **parity-gate before deleting hand | `next_command` | the exact command / skill that performs the cutover (see bridge below) | | `parity_gate` | the specific check proving behavior-equivalence | | `tier` | 1–4 | +| `status` | `new` / `open` / `fixed` / `deferred` against the previous audit (`new` on a first run) | +| `first_seen` | date of the audit that first raised it (carried forward on re-runs) | +| `deferred_reason` | for `deferred` only: the recorded decision, quoted, and where it is recorded | + +Fixed findings stay in the file with `status: fixed` for one run, so the next reader sees +what closed. + +**Top-level fields in `.metaobjects/adoption-audit.json`**, beside `findings[]`: + +| Field | Content | +|---|---| +| `audited_at`, `commit` | when the audit ran and the commit it read | +| `previous` | `audited_at` and `commit` of the baseline audit, or `null` on a first run | +| `tier`, `pillars[]` | headline tier and `{ pillar, tier, top_gap }` per pillar | +| `ci_gate` | `{ score: "BLOCKING" \| "RUNS_BUT_DOES_NOT_BLOCK" \| "ABSENT", exit_code, error_count, swallowed_at[], deploy_paths_skipping[] }` | +| `used_as_described` | `{ band, generated_consumed, generated_emitted, bypassed_count, gate_score, payload_fields_linked, payload_fields_unlinked, shape_statements_generated_or_derived, shape_statements_total, unmodelled_shapes[] }` | +| `owned_generators` | `{ unmodified_lines, total_lines, copies[] }`, one total across every port | +| `status_counts` | `{ new, open, fixed, deferred }` | Within each tier, sort by impact ÷ effort (quick wins first). Tier 1 leads. diff --git a/agent-context/skills/metaobjects-audit/references/csharp.md b/agent-context/skills/metaobjects-audit/references/csharp.md index 3579d5371..66b150442 100644 --- a/agent-context/skills/metaobjects-audit/references/csharp.md +++ b/agent-context/skills/metaobjects-audit/references/csharp.md @@ -80,7 +80,23 @@ copies the helper runtime the generated routes call (`FilterParser`, `EfCoreFilt `ValueObjectValidator`, `ConstraintErrors`, `Iso8601TimestampConverter`) into `codegen/runtime/` under the namespace `Codegen.Runtime`. That folder is owned code, not a hand-rolled filter parser: do not flag it. `dotnet meta gen --list` marks each owned copy -`identical` or `DIFFERS`. A generator the project WRITES is an `IGenerator` listed in +`identical` or `DIFFERS`. + +**Audit the owned copies first.** A copy exists to carry a change; one that carries none is +duplication that falls behind the reference. In `dotnet meta gen --list` an owned row reads +`[owned — identical]` or `[owned — DIFFERS: N behind, M of your own]` (`0 behind, 0 of your +own, reordered` for a copy with the same lines in another order), and `routes` adds a clause +for each `codegen/runtime/` file that differs. Only `identical`, with every runtime clause +identical too, proves a copy unmodified; that row is a **finding**: point the +`codegen/Generators.cs` entry back at the stock generator and remove the copy +(`dotnet meta eject --undo` removes every identical copy and prints the edit). `DIFFERS` +with `0 of your own` rules out added lines, not removed ones: confirm from `git log -p` on +the copy that nothing changed after the eject, else report it as **possibly customized: +check its history**. A copy with lines of its own, or an edited runtime file, is a +customization; check only how far behind it is. Count lines for the Phase 0 total: lines in +copies proven unmodified, over all lines under `codegen/generators/` and `codegen/runtime/`. + +A generator the project WRITES is an `IGenerator` listed in `codegen/Generators.cs` the same way. With no `codegen/Generators.cs`, `dotnet meta gen` exits 2 and generates nothing. diff --git a/agent-context/skills/metaobjects-audit/references/java.md b/agent-context/skills/metaobjects-audit/references/java.md index 9b5d4778b..7a20c3849 100644 --- a/agent-context/skills/metaobjects-audit/references/java.md +++ b/agent-context/skills/metaobjects-audit/references/java.md @@ -72,11 +72,30 @@ plugin `` — that is the source of truth for which generators ru ## Owned generators -Java does not scaffold-and-own generators — they are provided by -`metaobjects-codegen-spring` and wired by FQ class name in `pom.xml`. There is no -analog to the TS `codegen/generators/*.ts` pattern here. +Stock generators come from `metaobjects-codegen-spring` and are wired by fully qualified +class name in `pom.xml`. `mvn metaobjects:eject -Dnames= -Dport=java` copies one into +a separate `codegen/` Maven module under the project's own package, and `pom.xml` then names +the copy. -Java does, however, have **both** authoring paths, so a project needing a shape the +**Audit the owned copies first.** A copy exists to carry a change; one that carries none is +duplication that falls behind the reference. + +```bash +mvn metaobjects:eject -Dlist # the catalog, each owned copy marked +``` + +Each owned row is marked `[owned — identical]` or `[owned — DIFFERS: N behind, M of your +own]`, and a generator copied with companion classes (emitters, Java `entity` and +`extractor`) adds `; differing: `. Only `identical` proves a copy unmodified; that +row is a **finding**: point the `pom.xml` entry back at the stock class and remove the copy +(`mvn metaobjects:eject -Dundo=true` removes every identical copy and prints the `pom.xml` +edit). `DIFFERS` with `0 of your own` rules out added lines, not removed ones: confirm from +`git log -p` on the copy that nothing changed after the eject, else report it as **possibly +customized: check its history**. A copy with lines of its own is a customization; check +only how far behind it is. Count lines for the Phase 0 total: lines in copies proven +unmodified, over all lines under `codegen/` that eject wrote or the project wrote. + +Java also has **both** authoring paths, so a project needing a shape the built-ins do not emit has two legitimate answers and neither is a finding on its own: a **programmatic** generator (implement `com.metaobjects.generator.Generator`, name the class in a `` element — the plugin loads it from the project classpath), or a diff --git a/agent-context/skills/metaobjects-audit/references/kotlin.md b/agent-context/skills/metaobjects-audit/references/kotlin.md index 586fdaa6e..58903ecdc 100644 --- a/agent-context/skills/metaobjects-audit/references/kotlin.md +++ b/agent-context/skills/metaobjects-audit/references/kotlin.md @@ -77,11 +77,30 @@ The active generator list is declared in `pom.xml` under the plugin ` -Dport=kotlin` copies one +into a separate `codegen/` Maven module under the project's own package, and `pom.xml` then +names the copy. -Kotlin does, however, have **both** authoring paths — it shares the JVM engine — so a +**Audit the owned copies first.** A copy exists to carry a change; one that carries none is +duplication that falls behind the reference. + +```bash +mvn metaobjects:eject -Dlist # the catalog, each owned copy marked +``` + +Each owned row is marked `[owned — identical]` or `[owned — DIFFERS: N behind, M of your +own]`, and a generator copied with companion classes adds `; differing: `. Only +`identical` proves a copy unmodified; that row is a **finding**: point the `pom.xml` entry +back at the stock class and remove the copy (`mvn metaobjects:eject -Dundo=true` removes +every identical copy and prints the `pom.xml` edit). `DIFFERS` with `0 of your own` rules +out added lines, not removed ones: confirm from `git log -p` on the copy that nothing +changed after the eject, else report it as **possibly customized: check its history**. A +copy with lines of its own is a customization; check only how far behind it is. Count lines +for the Phase 0 total: lines in copies proven unmodified, over all lines under `codegen/` +that eject wrote or the project wrote. + +Kotlin also has **both** authoring paths — it shares the JVM engine — so a project needing a shape the built-ins do not emit has two legitimate answers and neither is a finding on its own: a **programmatic** generator (implement `com.metaobjects.generator.Generator`, name the class in a `` element), or a diff --git a/agent-context/skills/metaobjects-audit/references/prompt-anti-patterns.md b/agent-context/skills/metaobjects-audit/references/prompt-anti-patterns.md index 92cfb4748..4f5678520 100644 --- a/agent-context/skills/metaobjects-audit/references/prompt-anti-patterns.md +++ b/agent-context/skills/metaobjects-audit/references/prompt-anti-patterns.md @@ -3,7 +3,7 @@ ## Contents - Census scope - Keeping the result in place -- Anti-pattern list +- Anti-pattern list (includes the unlinked-payload ratio) - Template-text duplication **The census covers EVERY module**, not only the application: test harnesses, simulators, @@ -26,6 +26,24 @@ format-built argument at the project's own LLM request call sites. **sourceless `object.projection`** carrying those origins (#210 — assembly origins on an `object.value` fail load with `ERR_SUBTYPE_RULE_VIOLATION`; `@payloadRef` accepts the sourceless projection). +- **Unlinked payload.** A declared payload shape is necessary, not sufficient: the check above + asks whether the payload IS a shape, this one asks where its fields COME FROM. For each object + a `@payloadRef` names, class every field: + - **LINKED** — a dotted `extends: Entity.field` (for example `"extends": "Customer.email"`), + or an origin that resolves to an entity field (`origin.passthrough` with + `@from: Customer.email`, or an `aggregate` / `first` origin whose `@of` names one). + - **CALLER-SUPPLIED** — no entity owns the value (the user's question, a run-time setting, a + date the caller computes). Say why for each; "the caller fills it" is not a reason when the + caller reads it off an entity. + - **UNLINKED COPY** — declared as a free-standing field, while the call site fills it from an + entity property (`payload.email = customer.email`, `email=customer.email`). + Report `linked / (linked + unlinked copies)` per payload and in total. Each unlinked copy is a + finding (`unlinked-payload-field`): renaming or re-typing the entity field will not fail + `verify --templates`, because the payload declares its own copy and the template still + matches that copy. Remedy: replace the field with `extends: Entity.field` (or a passthrough + origin on a sourceless projection). The `parity_gate` is a scratch-copy proof: rename the + entity field in a throwaway copy of the metadata and confirm the project's verify now fails, + where before the change it passed. - Silent-degradation hack (`try/except KeyError` or `?? ''` around formatting) — flag every instance. - Hand-rolled output parsing (regex / XML / ad-hoc JSON) vs a declared **responding `template.prompt`** (one carrying `@responseRef`) + generated `parse*` / `safeParse*` / diff --git a/agent-context/skills/metaobjects-audit/references/python.md b/agent-context/skills/metaobjects-audit/references/python.md index 92c1d4250..08974a711 100644 --- a/agent-context/skills/metaobjects-audit/references/python.md +++ b/agent-context/skills/metaobjects-audit/references/python.md @@ -80,6 +80,25 @@ the same way — a `module:symbol` entry in a target's `generators` in `metaobje reading the model through `metaobjects.codegen.model_walk`. (`--provider module:symbol` registers **metamodel vocabulary**, not a generator — do not read it as one.) +**Audit the owned copies first.** A copy exists to carry a change; one that carries none is +duplication that falls behind the reference. + +```bash +metaobjects gen --list # the catalog, each owned copy marked +``` + +An owned row ends `[owned — identical]` or `[owned — DIFFERS: N behind, M of your own]`, and +a generator that reads a `codegen/runtime/` module (`routes`: `filter_parser`, +`constraint_errors`) prints an indented `runtime []` line per companion. Only +`identical`, with every companion identical too, proves a copy unmodified; that row is a +**finding**: point the `metaobjects.config.yaml` entry back at the stock name and remove the +copy (`metaobjects eject --undo` removes every identical copy and prints the config edit). +`DIFFERS` with `0 of your own` rules out added lines, not removed ones: confirm from +`git log -p` on the copy that nothing changed after the eject, else report it as **possibly +customized: check its history**. A copy with lines of its own, or an edited companion, is a +customization; check only how far behind it is. Count lines for the Phase 0 total: lines in +copies proven unmodified, over all lines under `codegen/generators/` and `codegen/runtime/`. + **A finding of the form "the built-ins do not emit the shape this project needs", or a hand-written layer the model fully describes, resolves to a generator the project writes** (ADR-0034 Amendment 4), or to a declarative template (a target's `templateSpec: `, diff --git a/agent-context/skills/metaobjects-audit/references/requirements.md b/agent-context/skills/metaobjects-audit/references/requirements.md index 5c48284f2..2bcebe9ec 100644 --- a/agent-context/skills/metaobjects-audit/references/requirements.md +++ b/agent-context/skills/metaobjects-audit/references/requirements.md @@ -11,6 +11,7 @@ what a green run already proves. ## Contents - What verify has already proven (do not re-check by hand) +- Currency checks — run these before sampling - What only a human or an agent reading the code can catch - Scope — do NOT flag these as defects - Reporting @@ -54,12 +55,46 @@ Three limits, and each puts something back on your list: `title` is deliberately NOT flagged as an inert slot; it is chartered as the entry's label and is rendered (see item 6). Do not report a populated `title` as a defect. +## Currency checks — run these before sampling + +A green gate proves the ledger's links resolve, not that the ledger keeps up with the code. +These three checks measure that, and the first one chooses the sample for item 1 below. + +**(a) Ledger stale.** From git history, take the commits that add a feature (a `feat:` / +`feat(...)` subject under Conventional Commits; otherwise merged feature branches or PRs +labelled as features) over the last six months or the last 50 such commits, whichever is +fewer. Report two numbers: the share of those commits that also change requirement metadata +(any file holding `requirement.*` nodes), and the count of feature commits since the most +recent change to that metadata. A share **under half**, or **more than ten** feature commits +since the last ledger change, is the finding `ledger-stale`, stating both numbers. The +feature commits that did not touch the ledger are the sample for item 1: read what each one +built and check that a requirement says so with a true status. + +**(b) Nothing implements a live claim.** Run the project's verify with the strict switch: +`meta verify --require-implementers`, `metaobjects verify --require-implementers`, +`mvn metaobjects:verify -Dmeta.verify.requireImplementers=true` or +`dotnet meta verify --require-implementers` (all ports also read +`META_REQUIRE_IMPLEMENTERS=1`). Each `WARN_REQUIREMENT_NOTHING_IMPLEMENTS` on a `live` entry +is a finding (`live-claim-unimplemented`) naming the entry. State whether the project's own +gate passes the switch; if it does not and the ledger has no such entries, recommend adding +it so the ledger cannot fall behind again. + +**(c) Live claim, unwired implementer.** For each `@implementedBy` target of a `live` entry, +find the generated code that target produces and confirm hand-written code imports or calls +it. A live claim whose implementer is generated code nothing uses is the finding +`live-claim-unwired-implementer`: the claim resolves, so verify is satisfied, but the +capability is not reachable. Before raising it, rule out a runtime use: code that reaches the +entity through the runtime `ObjectManager` (or a metadata-driven admin or tool registry) +uses it without importing anything generated, so search for the entity's name in those calls +too. + ## What only a human or an agent reading the code can catch **1. Statuses that are false.** The highest-value finding in the whole dimension. A requirement marked `live` whose implementation was gutted; one marked `partial` that is now complete; one marked `planned` that was quietly built months ago. Sample the claims -and read the nodes. `status` is the only payload with controlled evidence behind it — the +and read the nodes, starting with the feature commits currency check (a) found outside the +ledger. `status` is the only payload with controlled evidence behind it — the resurrection protection rides entirely on it being true. **2. Non-violable statements.** Every requirement must state what breaking it looks like. diff --git a/agent-context/skills/metaobjects-audit/references/surface-review-axes.md b/agent-context/skills/metaobjects-audit/references/surface-review-axes.md index 98cb47063..4a5e29277 100644 --- a/agent-context/skills/metaobjects-audit/references/surface-review-axes.md +++ b/agent-context/skills/metaobjects-audit/references/surface-review-axes.md @@ -22,7 +22,9 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in grids/forms/filters vs `layout.dataGrid` / form generators / filter-allowlist. - [ ] **C. Drift hotspot — validators, mappers, runtime models.** Hand validators / DTO-mappers / dataclasses shadowing a generated shape. Diff field-by-field; the divergence is the bug. -- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`). +- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`), + including the **unlinked payload** count: for each `@payloadRef` object, how many of its + fields are linked to the entity field they copy. - [ ] **E. Owned generators & scaffold-and-own** (see "Owned-codegen & scaffold-and-own assessment" in `SKILL.md`). - [ ] **E2. Capability ledger — `requirement.*`** (see `references/requirements.md` and the Requirement axis of the capability checklist). The one axis whose hunt is hand-written @@ -34,6 +36,9 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in design, and a real estate reports most of itself unclaimed on day one). Score the prose it would replace, and prefer ONE concrete claim an author can check over a whole-estate migration. + Where a ledger exists, run the three **currency checks** in `references/requirements.md` + (ledger stale, `--require-implementers`, live claim with an unwired implementer) before + sampling statuses. **First establish the estate LOADS.** An older ledger fails the load outright, so `meta verify` never runs and everything in `references/requirements.md` is unavailable: `@violation` and `@verifiedBy` were retired in **0.24.0** (no deprecation shim — ADR-0023 @@ -53,6 +58,52 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in subverbs (`--codegen` / `--templates` / `--db`)? Committed-codegen freshness gate? Advisories heeded? Routine `--no-verify` bypass? Loader `ERR_*` / warnings addressed? Parse the stable `code` field, not message text (ADR-0009). + **No CI means NO, whatever local hooks exist.** A git hook is not part of the clone: it + counts only when a tracked setup step installs it (a `prepare` / `postinstall` script, a + `core.hooksPath` set by a committed setup script, a hook manager declared in the + project's dependencies). An untracked `.git/hooks/pre-push` on one machine is not a gate. + A project with no CI workflow and no tracked hook installer answers the Binary CI drift + gate **NO**, and that is a **Tier 1 finding** whose `next_command` is the snippet below + for the project's port, ready to paste. + + | Port | Verify commands for the snippet (one per subverb the project's artifacts need) | + |---|---| + | TypeScript | `npx meta verify --codegen --templates`; add `npx meta verify --db` against a provisioned database | + | Python | `metaobjects verify --codegen` and `metaobjects verify --templates` | + | Java / Kotlin | `mvn -B metaobjects:verify -Dmeta.verify.mode=codegen` and `-Dmeta.verify.mode=templates` | + | C# | `dotnet meta verify --codegen` and `dotnet meta verify --templates` | + + The snippet is one CI job that checks out the repo, installs the toolchain and runs those + commands with nothing after them, for example GitHub Actions steps + `- run: npx meta verify --codegen --templates`. Where the project has no CI host, it is a + tracked `pre-push` hook that runs the same commands plus the tracked setup line that + installs it. Schema drift (`--db`) runs through the Node `meta` CLI in every port. + **Does it block?** Wired is not blocking. A gate that runs and is ignored scores the same as + no gate, so answer all three before scoring this axis: + - **(a) The exit code propagates.** Search CI files, package scripts, Makefiles, shell + scripts and build files on every path that runs verify for anything that swallows its + failure: `|| true`, `|| echo`, `; exit 0`, `continue-on-error: true`, `set +e`, + `allow_failure: true`, `ignore_errors`, a step or job marked optional or non-required, + a Maven `testFailureIgnore` / `failOnError=false` style switch, an MSBuild + `ContinueOnError`, a `-Dmeta.lax=true` or other option that turns a verify failure into a + warning, and verify output piped into a command whose exit code is the one that counts + (`meta verify | tee log` without `pipefail`). Each one is a finding naming the file and + line. + - **(b) It is green now.** Run the project's own verify command, the exact one its gate + runs, and record the exit code and the error count. A non-zero exit the project tolerates + ("known errors", a baseline file, a count it compares against) is a finding that states + the count and recommends a **ratchet**: the gate stores today's count and fails when it + rises, so the number can only fall (see the CI ratchet gate in `metaobjects-verify`). A + tolerated count with no ratchet is a gate that will absorb every new error. + - **(c) Every route to production passes it.** List each deploy entry point: CI deploy + jobs, release workflows, deploy scripts, `Makefile` / package-script deploy targets, a + hosting platform's build command, a manual `deploy.sh`. Flag each one that ships without + verify having passed on the same commit (a deploy job that does not depend on the verify + job, a script that builds and pushes directly, a `skip-checks` / `--no-verify` input). + Score the gate **BLOCKING** (verify runs on every route to production and a failure stops + it), **RUNS BUT DOES NOT BLOCK** (it runs somewhere, but (a), (b) or (c) failed) or + **ABSENT**. Only **BLOCKING** passes the drift-gate pillar's tier and answers YES on the + Binary CI drift gate; report the other two as **NO** and name which of (a), (b), (c) failed. **Is any advisory half switched OFF?** A gate can be wired, green, and muted. The requirements authoring lint is advisory and mutable — `--no-requirement-lint` or `META_NO_REQUIREMENT_LINT=1` silences it while the gate above it still runs and can still @@ -96,9 +147,24 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in code imports: drop the generator rather than generate into the void. - **Wrong target / duplicate output** — the same logical artifact emitted to two places (a mis-set per-target `outDir`), one of which is orphaned. + - **Consumed / emitted, per generator kind.** For each generator kind (entity, routes, + query hooks, grid, form, and so on) report `consumed / emitted` files, where consumed is + confirmed by an import or reference from hand-written code, not by the file existing. + - **Generated and bypassed.** Where a kind is under half consumed and axis B found a + hand-written equivalent for the same entity (a hand-built admin page, form or grid + beside the unused generated one), raise ONE finding, `generated-and-bypassed`, that + names both files. This is not the same as dead output: the project needed the surface + and built it again by hand, so the generated one is a second, unused source of truth. + The app chooses per entity: **use the generated artifact** (customising the owned + generator until it fits, then deleting the hand-written page after a parity gate), or + **filter it out** with the generator's `filter` and record the hand-written page as + ACCEPTABLE with a stated reason. Emitted and unused is not an end state: the finding + closes only when each entity is on one side or the other. Recommend generating ONLY what is consumed — decide per generator: drop the ones whose whole output class nothing imports, and narrow the ones that over-emit with - their `filter`. A smaller, fully-consumed generated surface beats a large one with + their `filter`. Dropping is the remedy only when no hand-written equivalent exists; + where one does, the finding is `generated-and-bypassed` above. A smaller, + fully-consumed generated surface beats a large one with dead files (which also make the leverage ratio lie; discount them from the census). - [ ] **G. Runtime-contract anti-patterns.** Module-global `db` vs context-as-parameter (ADR-0008); wire-canonicalization in the query path vs native in-process return types diff --git a/agent-context/skills/metaobjects-audit/references/typescript.md b/agent-context/skills/metaobjects-audit/references/typescript.md index 3fc99245c..d9041026b 100644 --- a/agent-context/skills/metaobjects-audit/references/typescript.md +++ b/agent-context/skills/metaobjects-audit/references/typescript.md @@ -121,6 +121,11 @@ are the `--format text` labels. In `--format json` or `toon` the same row carrie - A copy with lines of its own is an intentional customization. Check only how far behind it is. +Count lines for the Phase 0 total: lines in copies proven unmodified, with the +`codegen/runtime/` files they reach, over all lines the project owns under `codegen/` +(`generators/`, `runtime/`, and the `templates/` modules an eject copies). +`meta eject --undo` removes every identical copy and prints the config edit. + **`entity`, `routes` and `routes-hono` own more than the generator file.** Ejecting one also copies the HTTP adapter its output imports into `codegen/runtime/`, and a route fix is often made there with the generator file left untouched. Read the state of those files diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md index 525029b69..5225faad9 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -74,7 +74,13 @@ Actual cutovers run through the existing skills mapped per finding tier (§ Brid `queriesFile` / `routesFile` / `barrel` on a 1.0.x install** fails to load, because 1.0 removed those four and 1.1 restored them: there it is an upgrade blocker, fixed by upgrading to 1.1 or by ejecting. Importing anything from that subpath is otherwise - correct and is never a finding by itself. + correct and is never a finding by itself. **Every port owns generators the same way**: + run the owned-copy check in each port reference the project uses (TypeScript + `meta eject --list --format text`, Python `metaobjects gen --list`, Java / Kotlin + `mvn metaobjects:eject -Dlist`, C# `dotnet meta gen --list`), and report ONE total across + all of them: **unmodified owned-generator lines / all owned-generator lines**, where + unmodified counts only copies proven unmodified. Report the proven copies together as + one finding, `owned-generator-stock-copies`, listing each copy and the port's undo command. - [ ] **Cross-language version consistency (silent-drift check).** If the project uses MetaObjects in more than one language (e.g. a TS web client + a Java/Python/C# backend), enumerate EVERY MetaObjects package across ALL ecosystems (npm `@metaobjectsdev/*`, Maven `com.metaobjects:*`, PyPI `metaobjects`, NuGet `MetaObjects.*`) and record each version. **The version-number LINES differ by ecosystem (npm/PyPI/NuGet `0.x`/`1.x` vs Maven `7.x`/`8.x`), so you CANNOT eyeball drift** — a `0.12` next to a `7.7` looks fine but can be badly out of sync. Compare the **`metamodelVersion`** each port reports (the shared spec version on the registry manifest): a mismatch is real cross-language drift and a **finding** — the ports disagree on vocabulary/wire behavior. Also flag any port not on the latest release for its ecosystem. (This is a known real-world failure mode: newest backend, stale client, invisible because the numbers differ.) - [ ] Classify: **Greenfield** (none/minimal) · **Partial** · **Deep** → choose path below. @@ -103,6 +109,20 @@ code grows. The gap between "modeled + query helpers" and "has view + route + UI" is the headline lopsidedness. +**Shape coverage census.** The coverage matrix lists modelled entities only, so a shape +nobody modelled never appears in it. Count the other side too. List every shape that crosses +a boundary: database tables and views, request and response bodies, queue and socket +messages, prompt inputs, LLM replies, and file formats the project reads or writes. For each +shape, record whether it is modelled, and for each layer that states it (DDL or migration, +ORM model, validator, DTO, client type, prompt payload, reply parser) whether that statement +is **generated**, **derived** from generated code (a type built from a generated type, a +`z.infer` of a generated schema), or **hand-declared**. Report +`generated or derived statements / all statements` and list the unmodelled shapes. A table +present in the database or the migrations with no entity or projection behind it is a +finding (`unmodelled-table`). Like the leverage ratio this is a measurement, never a target: +an unmodelled shape is a finding only where the cost test in § Owned-codegen says modelling it +pays, or it is a table. + **Surface review — 8 axes (independently runnable).** Work the full `references/capability-checklist.md` on every axis. Check calibration guards (`references/calibration.md`) before raising a finding. **Verify, don't assume** — read the @@ -113,7 +133,7 @@ Axes (the full hunt for each is in `references/surface-review-axes.md` — open - **A.** Codegen candidates — API / server routes · **B.** Codegen candidates — web / client - **C.** Drift hotspot — validators, mappers, runtime models · **D.** Prompt pillar - **E.** Owned generators and scaffold-and-own · **E2.** Capability ledger (`requirement.*`, opt-in) -- **F.** Drift-gate adoption · **F2.** Over-generation (codegen waste) +- **F.** Drift-gate adoption — does it block? · **F2.** Over-generation and generated-but-bypassed (codegen waste) - **G.** Runtime-contract anti-patterns · **G2.** `own*()` accessor discipline (a correctness defect, not advisory) - **H.** Authoring correctness / ADR conformance · **H2.** Wrong native type (`field.string` + `@dbColumnType: uuid`) - **I.** Vocabulary hygiene / modernization (advisory, never a failing finding) @@ -199,14 +219,36 @@ one-consumer need, read it codegen-locally. 1. **Headline MATURITY TIER** — Greenfield → Partial → Deep → Exemplary; worst-of with prerequisite gating (a missing pillar can't be averaged away); rendered with **the single - next unmet check** ("you're Partial; the next rung needs `verify` in CI"). + next unmet check** ("you're Partial; the next rung needs a blocking `verify` in CI"). 2. **Per-pillar breakdown (never rolled into one number)** — `pillar | tier | top gap` over codegen / runtime / drift-gate / prompts. This is the core deliverable. -3. **Binary CI drift gate** — prominent and separate: **"Is `meta verify` drift detection - wired into CI?"** It is binary because the risk is binary. - -Coarse bands only (none / some / most / all). Worst-of within a pillar. On re-run, grade -the delta. Lead with gaps, not the grade. +3. **Binary CI drift gate** — prominent and separate: **"Does `meta verify` drift detection + block every route to production?"** It is binary because the risk is binary. YES only + when axis F scores the gate **BLOCKING**; RUNS BUT DOES NOT BLOCK and ABSENT both answer + NO, with the reason (exit code swallowed, tolerated error count, a deploy path that + skips it, no CI). Local hooks with no tracked installer do not count. +4. **Used as described** — one line, separate from the tier: does the project use what it + adopted the way MetaObjects documents it? Built from four measurements, each shown with + its numbers: generated artifacts consumed rather than bypassed (axis F2, `consumed / + emitted` and the `generated-and-bypassed` count); the gate blocks (axis F score); prompt + payload fields linked to entity fields (axis D, `linked / (linked + unlinked copies)`); + and boundary shapes generated or derived (Phase 1b shape coverage census). Render it as + `Used as described: — generated forms 5/12 consumed · gate RUNS BUT DOES NOT + BLOCK · payload fields 6/10 linked · shapes 50% generated or derived`, banded by its worst + part. + +Coarse bands only (none / some / most / all). Worst-of within a pillar. Lead with gaps, not +the grade. + +**Re-run: compare against the previous audit.** If `.metaobjects/adoption-audit.json` +already exists, read it before starting and keep it as the baseline. Match each earlier +finding to this run's by `id` plus location, and give every earlier finding a `status`: +**fixed** (the location no longer shows it; say what changed), **open** (still present, +carrying its original `first_seen`), or **deferred** (the project recorded a decision not to +fix it now, in a commit, an ADR, a tracked issue or a code comment, quoted as +`deferred_reason`). A finding that is still present with no recorded decision is open, never +deferred. New findings are `new`. Report the counts per status at the top of the report, +and show each scorecard line beside its previous value. **Vocabulary hygiene (axis I) is advisory** — it surfaces as modernization opportunities in the roadmap, scored as such, and **never gates a tier or fails the @@ -218,8 +260,10 @@ audit.** Dated-but-working vocabulary is a quality nudge, not a defect. **Two artifacts:** `.metaobjects/adoption-audit.json` + rendered Markdown. -**Markdown sections (lead with Scorecard):** 0. Scorecard (tier + pillar table + CI gate) · -1. Triage + census · 2. Coverage matrix · 3. Per-surface classification tables · 4. Drift +**Markdown sections (lead with Scorecard):** 0. Scorecard (tier + pillar table + CI gate + +used as described; on a re-run, the fixed / open / deferred / new counts) · +1. Triage + census (including the owned-generator unmodified-lines total and the shape +coverage census) · 2. Coverage matrix · 3. Per-surface classification tables · 4. Drift findings (active bugs first) · 5. Owned-codegen + generator gaps · 6. Drift-gate adoption · 7. Runtime-contract + authoring-correctness (axes G+H) · 8. Semantic-constraint ratification · 9. Prompt-pillar assessment · 10. Prioritized roadmap: Tier 1 drift kill → Tier 2 existing @@ -243,6 +287,24 @@ Each roadmap item: LOC retired, prerequisite, **parity-gate before deleting hand | `next_command` | the exact command / skill that performs the cutover (see bridge below) | | `parity_gate` | the specific check proving behavior-equivalence | | `tier` | 1–4 | +| `status` | `new` / `open` / `fixed` / `deferred` against the previous audit (`new` on a first run) | +| `first_seen` | date of the audit that first raised it (carried forward on re-runs) | +| `deferred_reason` | for `deferred` only: the recorded decision, quoted, and where it is recorded | + +Fixed findings stay in the file with `status: fixed` for one run, so the next reader sees +what closed. + +**Top-level fields in `.metaobjects/adoption-audit.json`**, beside `findings[]`: + +| Field | Content | +|---|---| +| `audited_at`, `commit` | when the audit ran and the commit it read | +| `previous` | `audited_at` and `commit` of the baseline audit, or `null` on a first run | +| `tier`, `pillars[]` | headline tier and `{ pillar, tier, top_gap }` per pillar | +| `ci_gate` | `{ score: "BLOCKING" \| "RUNS_BUT_DOES_NOT_BLOCK" \| "ABSENT", exit_code, error_count, swallowed_at[], deploy_paths_skipping[] }` | +| `used_as_described` | `{ band, generated_consumed, generated_emitted, bypassed_count, gate_score, payload_fields_linked, payload_fields_unlinked, shape_statements_generated_or_derived, shape_statements_total, unmodelled_shapes[] }` | +| `owned_generators` | `{ unmodified_lines, total_lines, copies[] }`, one total across every port | +| `status_counts` | `{ new, open, fixed, deferred }` | Within each tier, sort by impact ÷ effort (quick wins first). Tier 1 leads. diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/java.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/java.md index 9b5d4778b..7a20c3849 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/java.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/java.md @@ -72,11 +72,30 @@ plugin `` — that is the source of truth for which generators ru ## Owned generators -Java does not scaffold-and-own generators — they are provided by -`metaobjects-codegen-spring` and wired by FQ class name in `pom.xml`. There is no -analog to the TS `codegen/generators/*.ts` pattern here. +Stock generators come from `metaobjects-codegen-spring` and are wired by fully qualified +class name in `pom.xml`. `mvn metaobjects:eject -Dnames= -Dport=java` copies one into +a separate `codegen/` Maven module under the project's own package, and `pom.xml` then names +the copy. -Java does, however, have **both** authoring paths, so a project needing a shape the +**Audit the owned copies first.** A copy exists to carry a change; one that carries none is +duplication that falls behind the reference. + +```bash +mvn metaobjects:eject -Dlist # the catalog, each owned copy marked +``` + +Each owned row is marked `[owned — identical]` or `[owned — DIFFERS: N behind, M of your +own]`, and a generator copied with companion classes (emitters, Java `entity` and +`extractor`) adds `; differing: `. Only `identical` proves a copy unmodified; that +row is a **finding**: point the `pom.xml` entry back at the stock class and remove the copy +(`mvn metaobjects:eject -Dundo=true` removes every identical copy and prints the `pom.xml` +edit). `DIFFERS` with `0 of your own` rules out added lines, not removed ones: confirm from +`git log -p` on the copy that nothing changed after the eject, else report it as **possibly +customized: check its history**. A copy with lines of its own is a customization; check +only how far behind it is. Count lines for the Phase 0 total: lines in copies proven +unmodified, over all lines under `codegen/` that eject wrote or the project wrote. + +Java also has **both** authoring paths, so a project needing a shape the built-ins do not emit has two legitimate answers and neither is a finding on its own: a **programmatic** generator (implement `com.metaobjects.generator.Generator`, name the class in a `` element — the plugin loads it from the project classpath), or a diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/kotlin.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/kotlin.md index 586fdaa6e..58903ecdc 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/kotlin.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/kotlin.md @@ -77,11 +77,30 @@ The active generator list is declared in `pom.xml` under the plugin ` -Dport=kotlin` copies one +into a separate `codegen/` Maven module under the project's own package, and `pom.xml` then +names the copy. -Kotlin does, however, have **both** authoring paths — it shares the JVM engine — so a +**Audit the owned copies first.** A copy exists to carry a change; one that carries none is +duplication that falls behind the reference. + +```bash +mvn metaobjects:eject -Dlist # the catalog, each owned copy marked +``` + +Each owned row is marked `[owned — identical]` or `[owned — DIFFERS: N behind, M of your +own]`, and a generator copied with companion classes adds `; differing: `. Only +`identical` proves a copy unmodified; that row is a **finding**: point the `pom.xml` entry +back at the stock class and remove the copy (`mvn metaobjects:eject -Dundo=true` removes +every identical copy and prints the `pom.xml` edit). `DIFFERS` with `0 of your own` rules +out added lines, not removed ones: confirm from `git log -p` on the copy that nothing +changed after the eject, else report it as **possibly customized: check its history**. A +copy with lines of its own is a customization; check only how far behind it is. Count lines +for the Phase 0 total: lines in copies proven unmodified, over all lines under `codegen/` +that eject wrote or the project wrote. + +Kotlin also has **both** authoring paths — it shares the JVM engine — so a project needing a shape the built-ins do not emit has two legitimate answers and neither is a finding on its own: a **programmatic** generator (implement `com.metaobjects.generator.Generator`, name the class in a `` element), or a diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md index 92cfb4748..4f5678520 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md @@ -3,7 +3,7 @@ ## Contents - Census scope - Keeping the result in place -- Anti-pattern list +- Anti-pattern list (includes the unlinked-payload ratio) - Template-text duplication **The census covers EVERY module**, not only the application: test harnesses, simulators, @@ -26,6 +26,24 @@ format-built argument at the project's own LLM request call sites. **sourceless `object.projection`** carrying those origins (#210 — assembly origins on an `object.value` fail load with `ERR_SUBTYPE_RULE_VIOLATION`; `@payloadRef` accepts the sourceless projection). +- **Unlinked payload.** A declared payload shape is necessary, not sufficient: the check above + asks whether the payload IS a shape, this one asks where its fields COME FROM. For each object + a `@payloadRef` names, class every field: + - **LINKED** — a dotted `extends: Entity.field` (for example `"extends": "Customer.email"`), + or an origin that resolves to an entity field (`origin.passthrough` with + `@from: Customer.email`, or an `aggregate` / `first` origin whose `@of` names one). + - **CALLER-SUPPLIED** — no entity owns the value (the user's question, a run-time setting, a + date the caller computes). Say why for each; "the caller fills it" is not a reason when the + caller reads it off an entity. + - **UNLINKED COPY** — declared as a free-standing field, while the call site fills it from an + entity property (`payload.email = customer.email`, `email=customer.email`). + Report `linked / (linked + unlinked copies)` per payload and in total. Each unlinked copy is a + finding (`unlinked-payload-field`): renaming or re-typing the entity field will not fail + `verify --templates`, because the payload declares its own copy and the template still + matches that copy. Remedy: replace the field with `extends: Entity.field` (or a passthrough + origin on a sourceless projection). The `parity_gate` is a scratch-copy proof: rename the + entity field in a throwaway copy of the metadata and confirm the project's verify now fails, + where before the change it passed. - Silent-degradation hack (`try/except KeyError` or `?? ''` around formatting) — flag every instance. - Hand-rolled output parsing (regex / XML / ad-hoc JSON) vs a declared **responding `template.prompt`** (one carrying `@responseRef`) + generated `parse*` / `safeParse*` / diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md index 98cb47063..4a5e29277 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md @@ -22,7 +22,9 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in grids/forms/filters vs `layout.dataGrid` / form generators / filter-allowlist. - [ ] **C. Drift hotspot — validators, mappers, runtime models.** Hand validators / DTO-mappers / dataclasses shadowing a generated shape. Diff field-by-field; the divergence is the bug. -- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`). +- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`), + including the **unlinked payload** count: for each `@payloadRef` object, how many of its + fields are linked to the entity field they copy. - [ ] **E. Owned generators & scaffold-and-own** (see "Owned-codegen & scaffold-and-own assessment" in `SKILL.md`). - [ ] **E2. Capability ledger — `requirement.*`** (see `references/requirements.md` and the Requirement axis of the capability checklist). The one axis whose hunt is hand-written @@ -34,6 +36,9 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in design, and a real estate reports most of itself unclaimed on day one). Score the prose it would replace, and prefer ONE concrete claim an author can check over a whole-estate migration. + Where a ledger exists, run the three **currency checks** in `references/requirements.md` + (ledger stale, `--require-implementers`, live claim with an unwired implementer) before + sampling statuses. **First establish the estate LOADS.** An older ledger fails the load outright, so `meta verify` never runs and everything in `references/requirements.md` is unavailable: `@violation` and `@verifiedBy` were retired in **0.24.0** (no deprecation shim — ADR-0023 @@ -53,6 +58,52 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in subverbs (`--codegen` / `--templates` / `--db`)? Committed-codegen freshness gate? Advisories heeded? Routine `--no-verify` bypass? Loader `ERR_*` / warnings addressed? Parse the stable `code` field, not message text (ADR-0009). + **No CI means NO, whatever local hooks exist.** A git hook is not part of the clone: it + counts only when a tracked setup step installs it (a `prepare` / `postinstall` script, a + `core.hooksPath` set by a committed setup script, a hook manager declared in the + project's dependencies). An untracked `.git/hooks/pre-push` on one machine is not a gate. + A project with no CI workflow and no tracked hook installer answers the Binary CI drift + gate **NO**, and that is a **Tier 1 finding** whose `next_command` is the snippet below + for the project's port, ready to paste. + + | Port | Verify commands for the snippet (one per subverb the project's artifacts need) | + |---|---| + | TypeScript | `npx meta verify --codegen --templates`; add `npx meta verify --db` against a provisioned database | + | Python | `metaobjects verify --codegen` and `metaobjects verify --templates` | + | Java / Kotlin | `mvn -B metaobjects:verify -Dmeta.verify.mode=codegen` and `-Dmeta.verify.mode=templates` | + | C# | `dotnet meta verify --codegen` and `dotnet meta verify --templates` | + + The snippet is one CI job that checks out the repo, installs the toolchain and runs those + commands with nothing after them, for example GitHub Actions steps + `- run: npx meta verify --codegen --templates`. Where the project has no CI host, it is a + tracked `pre-push` hook that runs the same commands plus the tracked setup line that + installs it. Schema drift (`--db`) runs through the Node `meta` CLI in every port. + **Does it block?** Wired is not blocking. A gate that runs and is ignored scores the same as + no gate, so answer all three before scoring this axis: + - **(a) The exit code propagates.** Search CI files, package scripts, Makefiles, shell + scripts and build files on every path that runs verify for anything that swallows its + failure: `|| true`, `|| echo`, `; exit 0`, `continue-on-error: true`, `set +e`, + `allow_failure: true`, `ignore_errors`, a step or job marked optional or non-required, + a Maven `testFailureIgnore` / `failOnError=false` style switch, an MSBuild + `ContinueOnError`, a `-Dmeta.lax=true` or other option that turns a verify failure into a + warning, and verify output piped into a command whose exit code is the one that counts + (`meta verify | tee log` without `pipefail`). Each one is a finding naming the file and + line. + - **(b) It is green now.** Run the project's own verify command, the exact one its gate + runs, and record the exit code and the error count. A non-zero exit the project tolerates + ("known errors", a baseline file, a count it compares against) is a finding that states + the count and recommends a **ratchet**: the gate stores today's count and fails when it + rises, so the number can only fall (see the CI ratchet gate in `metaobjects-verify`). A + tolerated count with no ratchet is a gate that will absorb every new error. + - **(c) Every route to production passes it.** List each deploy entry point: CI deploy + jobs, release workflows, deploy scripts, `Makefile` / package-script deploy targets, a + hosting platform's build command, a manual `deploy.sh`. Flag each one that ships without + verify having passed on the same commit (a deploy job that does not depend on the verify + job, a script that builds and pushes directly, a `skip-checks` / `--no-verify` input). + Score the gate **BLOCKING** (verify runs on every route to production and a failure stops + it), **RUNS BUT DOES NOT BLOCK** (it runs somewhere, but (a), (b) or (c) failed) or + **ABSENT**. Only **BLOCKING** passes the drift-gate pillar's tier and answers YES on the + Binary CI drift gate; report the other two as **NO** and name which of (a), (b), (c) failed. **Is any advisory half switched OFF?** A gate can be wired, green, and muted. The requirements authoring lint is advisory and mutable — `--no-requirement-lint` or `META_NO_REQUIREMENT_LINT=1` silences it while the gate above it still runs and can still @@ -96,9 +147,24 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in code imports: drop the generator rather than generate into the void. - **Wrong target / duplicate output** — the same logical artifact emitted to two places (a mis-set per-target `outDir`), one of which is orphaned. + - **Consumed / emitted, per generator kind.** For each generator kind (entity, routes, + query hooks, grid, form, and so on) report `consumed / emitted` files, where consumed is + confirmed by an import or reference from hand-written code, not by the file existing. + - **Generated and bypassed.** Where a kind is under half consumed and axis B found a + hand-written equivalent for the same entity (a hand-built admin page, form or grid + beside the unused generated one), raise ONE finding, `generated-and-bypassed`, that + names both files. This is not the same as dead output: the project needed the surface + and built it again by hand, so the generated one is a second, unused source of truth. + The app chooses per entity: **use the generated artifact** (customising the owned + generator until it fits, then deleting the hand-written page after a parity gate), or + **filter it out** with the generator's `filter` and record the hand-written page as + ACCEPTABLE with a stated reason. Emitted and unused is not an end state: the finding + closes only when each entity is on one side or the other. Recommend generating ONLY what is consumed — decide per generator: drop the ones whose whole output class nothing imports, and narrow the ones that over-emit with - their `filter`. A smaller, fully-consumed generated surface beats a large one with + their `filter`. Dropping is the remedy only when no hand-written equivalent exists; + where one does, the finding is `generated-and-bypassed` above. A smaller, + fully-consumed generated surface beats a large one with dead files (which also make the leverage ratio lie; discount them from the census). - [ ] **G. Runtime-contract anti-patterns.** Module-global `db` vs context-as-parameter (ADR-0008); wire-canonicalization in the query path vs native in-process return types diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/SKILL.md index 525029b69..5225faad9 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -74,7 +74,13 @@ Actual cutovers run through the existing skills mapped per finding tier (§ Brid `queriesFile` / `routesFile` / `barrel` on a 1.0.x install** fails to load, because 1.0 removed those four and 1.1 restored them: there it is an upgrade blocker, fixed by upgrading to 1.1 or by ejecting. Importing anything from that subpath is otherwise - correct and is never a finding by itself. + correct and is never a finding by itself. **Every port owns generators the same way**: + run the owned-copy check in each port reference the project uses (TypeScript + `meta eject --list --format text`, Python `metaobjects gen --list`, Java / Kotlin + `mvn metaobjects:eject -Dlist`, C# `dotnet meta gen --list`), and report ONE total across + all of them: **unmodified owned-generator lines / all owned-generator lines**, where + unmodified counts only copies proven unmodified. Report the proven copies together as + one finding, `owned-generator-stock-copies`, listing each copy and the port's undo command. - [ ] **Cross-language version consistency (silent-drift check).** If the project uses MetaObjects in more than one language (e.g. a TS web client + a Java/Python/C# backend), enumerate EVERY MetaObjects package across ALL ecosystems (npm `@metaobjectsdev/*`, Maven `com.metaobjects:*`, PyPI `metaobjects`, NuGet `MetaObjects.*`) and record each version. **The version-number LINES differ by ecosystem (npm/PyPI/NuGet `0.x`/`1.x` vs Maven `7.x`/`8.x`), so you CANNOT eyeball drift** — a `0.12` next to a `7.7` looks fine but can be badly out of sync. Compare the **`metamodelVersion`** each port reports (the shared spec version on the registry manifest): a mismatch is real cross-language drift and a **finding** — the ports disagree on vocabulary/wire behavior. Also flag any port not on the latest release for its ecosystem. (This is a known real-world failure mode: newest backend, stale client, invisible because the numbers differ.) - [ ] Classify: **Greenfield** (none/minimal) · **Partial** · **Deep** → choose path below. @@ -103,6 +109,20 @@ code grows. The gap between "modeled + query helpers" and "has view + route + UI" is the headline lopsidedness. +**Shape coverage census.** The coverage matrix lists modelled entities only, so a shape +nobody modelled never appears in it. Count the other side too. List every shape that crosses +a boundary: database tables and views, request and response bodies, queue and socket +messages, prompt inputs, LLM replies, and file formats the project reads or writes. For each +shape, record whether it is modelled, and for each layer that states it (DDL or migration, +ORM model, validator, DTO, client type, prompt payload, reply parser) whether that statement +is **generated**, **derived** from generated code (a type built from a generated type, a +`z.infer` of a generated schema), or **hand-declared**. Report +`generated or derived statements / all statements` and list the unmodelled shapes. A table +present in the database or the migrations with no entity or projection behind it is a +finding (`unmodelled-table`). Like the leverage ratio this is a measurement, never a target: +an unmodelled shape is a finding only where the cost test in § Owned-codegen says modelling it +pays, or it is a table. + **Surface review — 8 axes (independently runnable).** Work the full `references/capability-checklist.md` on every axis. Check calibration guards (`references/calibration.md`) before raising a finding. **Verify, don't assume** — read the @@ -113,7 +133,7 @@ Axes (the full hunt for each is in `references/surface-review-axes.md` — open - **A.** Codegen candidates — API / server routes · **B.** Codegen candidates — web / client - **C.** Drift hotspot — validators, mappers, runtime models · **D.** Prompt pillar - **E.** Owned generators and scaffold-and-own · **E2.** Capability ledger (`requirement.*`, opt-in) -- **F.** Drift-gate adoption · **F2.** Over-generation (codegen waste) +- **F.** Drift-gate adoption — does it block? · **F2.** Over-generation and generated-but-bypassed (codegen waste) - **G.** Runtime-contract anti-patterns · **G2.** `own*()` accessor discipline (a correctness defect, not advisory) - **H.** Authoring correctness / ADR conformance · **H2.** Wrong native type (`field.string` + `@dbColumnType: uuid`) - **I.** Vocabulary hygiene / modernization (advisory, never a failing finding) @@ -199,14 +219,36 @@ one-consumer need, read it codegen-locally. 1. **Headline MATURITY TIER** — Greenfield → Partial → Deep → Exemplary; worst-of with prerequisite gating (a missing pillar can't be averaged away); rendered with **the single - next unmet check** ("you're Partial; the next rung needs `verify` in CI"). + next unmet check** ("you're Partial; the next rung needs a blocking `verify` in CI"). 2. **Per-pillar breakdown (never rolled into one number)** — `pillar | tier | top gap` over codegen / runtime / drift-gate / prompts. This is the core deliverable. -3. **Binary CI drift gate** — prominent and separate: **"Is `meta verify` drift detection - wired into CI?"** It is binary because the risk is binary. - -Coarse bands only (none / some / most / all). Worst-of within a pillar. On re-run, grade -the delta. Lead with gaps, not the grade. +3. **Binary CI drift gate** — prominent and separate: **"Does `meta verify` drift detection + block every route to production?"** It is binary because the risk is binary. YES only + when axis F scores the gate **BLOCKING**; RUNS BUT DOES NOT BLOCK and ABSENT both answer + NO, with the reason (exit code swallowed, tolerated error count, a deploy path that + skips it, no CI). Local hooks with no tracked installer do not count. +4. **Used as described** — one line, separate from the tier: does the project use what it + adopted the way MetaObjects documents it? Built from four measurements, each shown with + its numbers: generated artifacts consumed rather than bypassed (axis F2, `consumed / + emitted` and the `generated-and-bypassed` count); the gate blocks (axis F score); prompt + payload fields linked to entity fields (axis D, `linked / (linked + unlinked copies)`); + and boundary shapes generated or derived (Phase 1b shape coverage census). Render it as + `Used as described: — generated forms 5/12 consumed · gate RUNS BUT DOES NOT + BLOCK · payload fields 6/10 linked · shapes 50% generated or derived`, banded by its worst + part. + +Coarse bands only (none / some / most / all). Worst-of within a pillar. Lead with gaps, not +the grade. + +**Re-run: compare against the previous audit.** If `.metaobjects/adoption-audit.json` +already exists, read it before starting and keep it as the baseline. Match each earlier +finding to this run's by `id` plus location, and give every earlier finding a `status`: +**fixed** (the location no longer shows it; say what changed), **open** (still present, +carrying its original `first_seen`), or **deferred** (the project recorded a decision not to +fix it now, in a commit, an ADR, a tracked issue or a code comment, quoted as +`deferred_reason`). A finding that is still present with no recorded decision is open, never +deferred. New findings are `new`. Report the counts per status at the top of the report, +and show each scorecard line beside its previous value. **Vocabulary hygiene (axis I) is advisory** — it surfaces as modernization opportunities in the roadmap, scored as such, and **never gates a tier or fails the @@ -218,8 +260,10 @@ audit.** Dated-but-working vocabulary is a quality nudge, not a defect. **Two artifacts:** `.metaobjects/adoption-audit.json` + rendered Markdown. -**Markdown sections (lead with Scorecard):** 0. Scorecard (tier + pillar table + CI gate) · -1. Triage + census · 2. Coverage matrix · 3. Per-surface classification tables · 4. Drift +**Markdown sections (lead with Scorecard):** 0. Scorecard (tier + pillar table + CI gate + +used as described; on a re-run, the fixed / open / deferred / new counts) · +1. Triage + census (including the owned-generator unmodified-lines total and the shape +coverage census) · 2. Coverage matrix · 3. Per-surface classification tables · 4. Drift findings (active bugs first) · 5. Owned-codegen + generator gaps · 6. Drift-gate adoption · 7. Runtime-contract + authoring-correctness (axes G+H) · 8. Semantic-constraint ratification · 9. Prompt-pillar assessment · 10. Prioritized roadmap: Tier 1 drift kill → Tier 2 existing @@ -243,6 +287,24 @@ Each roadmap item: LOC retired, prerequisite, **parity-gate before deleting hand | `next_command` | the exact command / skill that performs the cutover (see bridge below) | | `parity_gate` | the specific check proving behavior-equivalence | | `tier` | 1–4 | +| `status` | `new` / `open` / `fixed` / `deferred` against the previous audit (`new` on a first run) | +| `first_seen` | date of the audit that first raised it (carried forward on re-runs) | +| `deferred_reason` | for `deferred` only: the recorded decision, quoted, and where it is recorded | + +Fixed findings stay in the file with `status: fixed` for one run, so the next reader sees +what closed. + +**Top-level fields in `.metaobjects/adoption-audit.json`**, beside `findings[]`: + +| Field | Content | +|---|---| +| `audited_at`, `commit` | when the audit ran and the commit it read | +| `previous` | `audited_at` and `commit` of the baseline audit, or `null` on a first run | +| `tier`, `pillars[]` | headline tier and `{ pillar, tier, top_gap }` per pillar | +| `ci_gate` | `{ score: "BLOCKING" \| "RUNS_BUT_DOES_NOT_BLOCK" \| "ABSENT", exit_code, error_count, swallowed_at[], deploy_paths_skipping[] }` | +| `used_as_described` | `{ band, generated_consumed, generated_emitted, bypassed_count, gate_score, payload_fields_linked, payload_fields_unlinked, shape_statements_generated_or_derived, shape_statements_total, unmodelled_shapes[] }` | +| `owned_generators` | `{ unmodified_lines, total_lines, copies[] }`, one total across every port | +| `status_counts` | `{ new, open, fixed, deferred }` | Within each tier, sort by impact ÷ effort (quick wins first). Tier 1 leads. diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/java.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/java.md index 9b5d4778b..7a20c3849 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/java.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/java.md @@ -72,11 +72,30 @@ plugin `` — that is the source of truth for which generators ru ## Owned generators -Java does not scaffold-and-own generators — they are provided by -`metaobjects-codegen-spring` and wired by FQ class name in `pom.xml`. There is no -analog to the TS `codegen/generators/*.ts` pattern here. +Stock generators come from `metaobjects-codegen-spring` and are wired by fully qualified +class name in `pom.xml`. `mvn metaobjects:eject -Dnames= -Dport=java` copies one into +a separate `codegen/` Maven module under the project's own package, and `pom.xml` then names +the copy. -Java does, however, have **both** authoring paths, so a project needing a shape the +**Audit the owned copies first.** A copy exists to carry a change; one that carries none is +duplication that falls behind the reference. + +```bash +mvn metaobjects:eject -Dlist # the catalog, each owned copy marked +``` + +Each owned row is marked `[owned — identical]` or `[owned — DIFFERS: N behind, M of your +own]`, and a generator copied with companion classes (emitters, Java `entity` and +`extractor`) adds `; differing: `. Only `identical` proves a copy unmodified; that +row is a **finding**: point the `pom.xml` entry back at the stock class and remove the copy +(`mvn metaobjects:eject -Dundo=true` removes every identical copy and prints the `pom.xml` +edit). `DIFFERS` with `0 of your own` rules out added lines, not removed ones: confirm from +`git log -p` on the copy that nothing changed after the eject, else report it as **possibly +customized: check its history**. A copy with lines of its own is a customization; check +only how far behind it is. Count lines for the Phase 0 total: lines in copies proven +unmodified, over all lines under `codegen/` that eject wrote or the project wrote. + +Java also has **both** authoring paths, so a project needing a shape the built-ins do not emit has two legitimate answers and neither is a finding on its own: a **programmatic** generator (implement `com.metaobjects.generator.Generator`, name the class in a `` element — the plugin loads it from the project classpath), or a diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md index 92cfb4748..4f5678520 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md @@ -3,7 +3,7 @@ ## Contents - Census scope - Keeping the result in place -- Anti-pattern list +- Anti-pattern list (includes the unlinked-payload ratio) - Template-text duplication **The census covers EVERY module**, not only the application: test harnesses, simulators, @@ -26,6 +26,24 @@ format-built argument at the project's own LLM request call sites. **sourceless `object.projection`** carrying those origins (#210 — assembly origins on an `object.value` fail load with `ERR_SUBTYPE_RULE_VIOLATION`; `@payloadRef` accepts the sourceless projection). +- **Unlinked payload.** A declared payload shape is necessary, not sufficient: the check above + asks whether the payload IS a shape, this one asks where its fields COME FROM. For each object + a `@payloadRef` names, class every field: + - **LINKED** — a dotted `extends: Entity.field` (for example `"extends": "Customer.email"`), + or an origin that resolves to an entity field (`origin.passthrough` with + `@from: Customer.email`, or an `aggregate` / `first` origin whose `@of` names one). + - **CALLER-SUPPLIED** — no entity owns the value (the user's question, a run-time setting, a + date the caller computes). Say why for each; "the caller fills it" is not a reason when the + caller reads it off an entity. + - **UNLINKED COPY** — declared as a free-standing field, while the call site fills it from an + entity property (`payload.email = customer.email`, `email=customer.email`). + Report `linked / (linked + unlinked copies)` per payload and in total. Each unlinked copy is a + finding (`unlinked-payload-field`): renaming or re-typing the entity field will not fail + `verify --templates`, because the payload declares its own copy and the template still + matches that copy. Remedy: replace the field with `extends: Entity.field` (or a passthrough + origin on a sourceless projection). The `parity_gate` is a scratch-copy proof: rename the + entity field in a throwaway copy of the metadata and confirm the project's verify now fails, + where before the change it passed. - Silent-degradation hack (`try/except KeyError` or `?? ''` around formatting) — flag every instance. - Hand-rolled output parsing (regex / XML / ad-hoc JSON) vs a declared **responding `template.prompt`** (one carrying `@responseRef`) + generated `parse*` / `safeParse*` / diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md index 98cb47063..4a5e29277 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md @@ -22,7 +22,9 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in grids/forms/filters vs `layout.dataGrid` / form generators / filter-allowlist. - [ ] **C. Drift hotspot — validators, mappers, runtime models.** Hand validators / DTO-mappers / dataclasses shadowing a generated shape. Diff field-by-field; the divergence is the bug. -- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`). +- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`), + including the **unlinked payload** count: for each `@payloadRef` object, how many of its + fields are linked to the entity field they copy. - [ ] **E. Owned generators & scaffold-and-own** (see "Owned-codegen & scaffold-and-own assessment" in `SKILL.md`). - [ ] **E2. Capability ledger — `requirement.*`** (see `references/requirements.md` and the Requirement axis of the capability checklist). The one axis whose hunt is hand-written @@ -34,6 +36,9 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in design, and a real estate reports most of itself unclaimed on day one). Score the prose it would replace, and prefer ONE concrete claim an author can check over a whole-estate migration. + Where a ledger exists, run the three **currency checks** in `references/requirements.md` + (ledger stale, `--require-implementers`, live claim with an unwired implementer) before + sampling statuses. **First establish the estate LOADS.** An older ledger fails the load outright, so `meta verify` never runs and everything in `references/requirements.md` is unavailable: `@violation` and `@verifiedBy` were retired in **0.24.0** (no deprecation shim — ADR-0023 @@ -53,6 +58,52 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in subverbs (`--codegen` / `--templates` / `--db`)? Committed-codegen freshness gate? Advisories heeded? Routine `--no-verify` bypass? Loader `ERR_*` / warnings addressed? Parse the stable `code` field, not message text (ADR-0009). + **No CI means NO, whatever local hooks exist.** A git hook is not part of the clone: it + counts only when a tracked setup step installs it (a `prepare` / `postinstall` script, a + `core.hooksPath` set by a committed setup script, a hook manager declared in the + project's dependencies). An untracked `.git/hooks/pre-push` on one machine is not a gate. + A project with no CI workflow and no tracked hook installer answers the Binary CI drift + gate **NO**, and that is a **Tier 1 finding** whose `next_command` is the snippet below + for the project's port, ready to paste. + + | Port | Verify commands for the snippet (one per subverb the project's artifacts need) | + |---|---| + | TypeScript | `npx meta verify --codegen --templates`; add `npx meta verify --db` against a provisioned database | + | Python | `metaobjects verify --codegen` and `metaobjects verify --templates` | + | Java / Kotlin | `mvn -B metaobjects:verify -Dmeta.verify.mode=codegen` and `-Dmeta.verify.mode=templates` | + | C# | `dotnet meta verify --codegen` and `dotnet meta verify --templates` | + + The snippet is one CI job that checks out the repo, installs the toolchain and runs those + commands with nothing after them, for example GitHub Actions steps + `- run: npx meta verify --codegen --templates`. Where the project has no CI host, it is a + tracked `pre-push` hook that runs the same commands plus the tracked setup line that + installs it. Schema drift (`--db`) runs through the Node `meta` CLI in every port. + **Does it block?** Wired is not blocking. A gate that runs and is ignored scores the same as + no gate, so answer all three before scoring this axis: + - **(a) The exit code propagates.** Search CI files, package scripts, Makefiles, shell + scripts and build files on every path that runs verify for anything that swallows its + failure: `|| true`, `|| echo`, `; exit 0`, `continue-on-error: true`, `set +e`, + `allow_failure: true`, `ignore_errors`, a step or job marked optional or non-required, + a Maven `testFailureIgnore` / `failOnError=false` style switch, an MSBuild + `ContinueOnError`, a `-Dmeta.lax=true` or other option that turns a verify failure into a + warning, and verify output piped into a command whose exit code is the one that counts + (`meta verify | tee log` without `pipefail`). Each one is a finding naming the file and + line. + - **(b) It is green now.** Run the project's own verify command, the exact one its gate + runs, and record the exit code and the error count. A non-zero exit the project tolerates + ("known errors", a baseline file, a count it compares against) is a finding that states + the count and recommends a **ratchet**: the gate stores today's count and fails when it + rises, so the number can only fall (see the CI ratchet gate in `metaobjects-verify`). A + tolerated count with no ratchet is a gate that will absorb every new error. + - **(c) Every route to production passes it.** List each deploy entry point: CI deploy + jobs, release workflows, deploy scripts, `Makefile` / package-script deploy targets, a + hosting platform's build command, a manual `deploy.sh`. Flag each one that ships without + verify having passed on the same commit (a deploy job that does not depend on the verify + job, a script that builds and pushes directly, a `skip-checks` / `--no-verify` input). + Score the gate **BLOCKING** (verify runs on every route to production and a failure stops + it), **RUNS BUT DOES NOT BLOCK** (it runs somewhere, but (a), (b) or (c) failed) or + **ABSENT**. Only **BLOCKING** passes the drift-gate pillar's tier and answers YES on the + Binary CI drift gate; report the other two as **NO** and name which of (a), (b), (c) failed. **Is any advisory half switched OFF?** A gate can be wired, green, and muted. The requirements authoring lint is advisory and mutable — `--no-requirement-lint` or `META_NO_REQUIREMENT_LINT=1` silences it while the gate above it still runs and can still @@ -96,9 +147,24 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in code imports: drop the generator rather than generate into the void. - **Wrong target / duplicate output** — the same logical artifact emitted to two places (a mis-set per-target `outDir`), one of which is orphaned. + - **Consumed / emitted, per generator kind.** For each generator kind (entity, routes, + query hooks, grid, form, and so on) report `consumed / emitted` files, where consumed is + confirmed by an import or reference from hand-written code, not by the file existing. + - **Generated and bypassed.** Where a kind is under half consumed and axis B found a + hand-written equivalent for the same entity (a hand-built admin page, form or grid + beside the unused generated one), raise ONE finding, `generated-and-bypassed`, that + names both files. This is not the same as dead output: the project needed the surface + and built it again by hand, so the generated one is a second, unused source of truth. + The app chooses per entity: **use the generated artifact** (customising the owned + generator until it fits, then deleting the hand-written page after a parity gate), or + **filter it out** with the generator's `filter` and record the hand-written page as + ACCEPTABLE with a stated reason. Emitted and unused is not an end state: the finding + closes only when each entity is on one side or the other. Recommend generating ONLY what is consumed — decide per generator: drop the ones whose whole output class nothing imports, and narrow the ones that over-emit with - their `filter`. A smaller, fully-consumed generated surface beats a large one with + their `filter`. Dropping is the remedy only when no hand-written equivalent exists; + where one does, the finding is `generated-and-bypassed` above. A smaller, + fully-consumed generated surface beats a large one with dead files (which also make the leverage ratio lie; discount them from the census). - [ ] **G. Runtime-contract anti-patterns.** Module-global `db` vs context-as-parameter (ADR-0008); wire-canonicalization in the query path vs native in-process return types diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/SKILL.md index 525029b69..5225faad9 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -74,7 +74,13 @@ Actual cutovers run through the existing skills mapped per finding tier (§ Brid `queriesFile` / `routesFile` / `barrel` on a 1.0.x install** fails to load, because 1.0 removed those four and 1.1 restored them: there it is an upgrade blocker, fixed by upgrading to 1.1 or by ejecting. Importing anything from that subpath is otherwise - correct and is never a finding by itself. + correct and is never a finding by itself. **Every port owns generators the same way**: + run the owned-copy check in each port reference the project uses (TypeScript + `meta eject --list --format text`, Python `metaobjects gen --list`, Java / Kotlin + `mvn metaobjects:eject -Dlist`, C# `dotnet meta gen --list`), and report ONE total across + all of them: **unmodified owned-generator lines / all owned-generator lines**, where + unmodified counts only copies proven unmodified. Report the proven copies together as + one finding, `owned-generator-stock-copies`, listing each copy and the port's undo command. - [ ] **Cross-language version consistency (silent-drift check).** If the project uses MetaObjects in more than one language (e.g. a TS web client + a Java/Python/C# backend), enumerate EVERY MetaObjects package across ALL ecosystems (npm `@metaobjectsdev/*`, Maven `com.metaobjects:*`, PyPI `metaobjects`, NuGet `MetaObjects.*`) and record each version. **The version-number LINES differ by ecosystem (npm/PyPI/NuGet `0.x`/`1.x` vs Maven `7.x`/`8.x`), so you CANNOT eyeball drift** — a `0.12` next to a `7.7` looks fine but can be badly out of sync. Compare the **`metamodelVersion`** each port reports (the shared spec version on the registry manifest): a mismatch is real cross-language drift and a **finding** — the ports disagree on vocabulary/wire behavior. Also flag any port not on the latest release for its ecosystem. (This is a known real-world failure mode: newest backend, stale client, invisible because the numbers differ.) - [ ] Classify: **Greenfield** (none/minimal) · **Partial** · **Deep** → choose path below. @@ -103,6 +109,20 @@ code grows. The gap between "modeled + query helpers" and "has view + route + UI" is the headline lopsidedness. +**Shape coverage census.** The coverage matrix lists modelled entities only, so a shape +nobody modelled never appears in it. Count the other side too. List every shape that crosses +a boundary: database tables and views, request and response bodies, queue and socket +messages, prompt inputs, LLM replies, and file formats the project reads or writes. For each +shape, record whether it is modelled, and for each layer that states it (DDL or migration, +ORM model, validator, DTO, client type, prompt payload, reply parser) whether that statement +is **generated**, **derived** from generated code (a type built from a generated type, a +`z.infer` of a generated schema), or **hand-declared**. Report +`generated or derived statements / all statements` and list the unmodelled shapes. A table +present in the database or the migrations with no entity or projection behind it is a +finding (`unmodelled-table`). Like the leverage ratio this is a measurement, never a target: +an unmodelled shape is a finding only where the cost test in § Owned-codegen says modelling it +pays, or it is a table. + **Surface review — 8 axes (independently runnable).** Work the full `references/capability-checklist.md` on every axis. Check calibration guards (`references/calibration.md`) before raising a finding. **Verify, don't assume** — read the @@ -113,7 +133,7 @@ Axes (the full hunt for each is in `references/surface-review-axes.md` — open - **A.** Codegen candidates — API / server routes · **B.** Codegen candidates — web / client - **C.** Drift hotspot — validators, mappers, runtime models · **D.** Prompt pillar - **E.** Owned generators and scaffold-and-own · **E2.** Capability ledger (`requirement.*`, opt-in) -- **F.** Drift-gate adoption · **F2.** Over-generation (codegen waste) +- **F.** Drift-gate adoption — does it block? · **F2.** Over-generation and generated-but-bypassed (codegen waste) - **G.** Runtime-contract anti-patterns · **G2.** `own*()` accessor discipline (a correctness defect, not advisory) - **H.** Authoring correctness / ADR conformance · **H2.** Wrong native type (`field.string` + `@dbColumnType: uuid`) - **I.** Vocabulary hygiene / modernization (advisory, never a failing finding) @@ -199,14 +219,36 @@ one-consumer need, read it codegen-locally. 1. **Headline MATURITY TIER** — Greenfield → Partial → Deep → Exemplary; worst-of with prerequisite gating (a missing pillar can't be averaged away); rendered with **the single - next unmet check** ("you're Partial; the next rung needs `verify` in CI"). + next unmet check** ("you're Partial; the next rung needs a blocking `verify` in CI"). 2. **Per-pillar breakdown (never rolled into one number)** — `pillar | tier | top gap` over codegen / runtime / drift-gate / prompts. This is the core deliverable. -3. **Binary CI drift gate** — prominent and separate: **"Is `meta verify` drift detection - wired into CI?"** It is binary because the risk is binary. - -Coarse bands only (none / some / most / all). Worst-of within a pillar. On re-run, grade -the delta. Lead with gaps, not the grade. +3. **Binary CI drift gate** — prominent and separate: **"Does `meta verify` drift detection + block every route to production?"** It is binary because the risk is binary. YES only + when axis F scores the gate **BLOCKING**; RUNS BUT DOES NOT BLOCK and ABSENT both answer + NO, with the reason (exit code swallowed, tolerated error count, a deploy path that + skips it, no CI). Local hooks with no tracked installer do not count. +4. **Used as described** — one line, separate from the tier: does the project use what it + adopted the way MetaObjects documents it? Built from four measurements, each shown with + its numbers: generated artifacts consumed rather than bypassed (axis F2, `consumed / + emitted` and the `generated-and-bypassed` count); the gate blocks (axis F score); prompt + payload fields linked to entity fields (axis D, `linked / (linked + unlinked copies)`); + and boundary shapes generated or derived (Phase 1b shape coverage census). Render it as + `Used as described: — generated forms 5/12 consumed · gate RUNS BUT DOES NOT + BLOCK · payload fields 6/10 linked · shapes 50% generated or derived`, banded by its worst + part. + +Coarse bands only (none / some / most / all). Worst-of within a pillar. Lead with gaps, not +the grade. + +**Re-run: compare against the previous audit.** If `.metaobjects/adoption-audit.json` +already exists, read it before starting and keep it as the baseline. Match each earlier +finding to this run's by `id` plus location, and give every earlier finding a `status`: +**fixed** (the location no longer shows it; say what changed), **open** (still present, +carrying its original `first_seen`), or **deferred** (the project recorded a decision not to +fix it now, in a commit, an ADR, a tracked issue or a code comment, quoted as +`deferred_reason`). A finding that is still present with no recorded decision is open, never +deferred. New findings are `new`. Report the counts per status at the top of the report, +and show each scorecard line beside its previous value. **Vocabulary hygiene (axis I) is advisory** — it surfaces as modernization opportunities in the roadmap, scored as such, and **never gates a tier or fails the @@ -218,8 +260,10 @@ audit.** Dated-but-working vocabulary is a quality nudge, not a defect. **Two artifacts:** `.metaobjects/adoption-audit.json` + rendered Markdown. -**Markdown sections (lead with Scorecard):** 0. Scorecard (tier + pillar table + CI gate) · -1. Triage + census · 2. Coverage matrix · 3. Per-surface classification tables · 4. Drift +**Markdown sections (lead with Scorecard):** 0. Scorecard (tier + pillar table + CI gate + +used as described; on a re-run, the fixed / open / deferred / new counts) · +1. Triage + census (including the owned-generator unmodified-lines total and the shape +coverage census) · 2. Coverage matrix · 3. Per-surface classification tables · 4. Drift findings (active bugs first) · 5. Owned-codegen + generator gaps · 6. Drift-gate adoption · 7. Runtime-contract + authoring-correctness (axes G+H) · 8. Semantic-constraint ratification · 9. Prompt-pillar assessment · 10. Prioritized roadmap: Tier 1 drift kill → Tier 2 existing @@ -243,6 +287,24 @@ Each roadmap item: LOC retired, prerequisite, **parity-gate before deleting hand | `next_command` | the exact command / skill that performs the cutover (see bridge below) | | `parity_gate` | the specific check proving behavior-equivalence | | `tier` | 1–4 | +| `status` | `new` / `open` / `fixed` / `deferred` against the previous audit (`new` on a first run) | +| `first_seen` | date of the audit that first raised it (carried forward on re-runs) | +| `deferred_reason` | for `deferred` only: the recorded decision, quoted, and where it is recorded | + +Fixed findings stay in the file with `status: fixed` for one run, so the next reader sees +what closed. + +**Top-level fields in `.metaobjects/adoption-audit.json`**, beside `findings[]`: + +| Field | Content | +|---|---| +| `audited_at`, `commit` | when the audit ran and the commit it read | +| `previous` | `audited_at` and `commit` of the baseline audit, or `null` on a first run | +| `tier`, `pillars[]` | headline tier and `{ pillar, tier, top_gap }` per pillar | +| `ci_gate` | `{ score: "BLOCKING" \| "RUNS_BUT_DOES_NOT_BLOCK" \| "ABSENT", exit_code, error_count, swallowed_at[], deploy_paths_skipping[] }` | +| `used_as_described` | `{ band, generated_consumed, generated_emitted, bypassed_count, gate_score, payload_fields_linked, payload_fields_unlinked, shape_statements_generated_or_derived, shape_statements_total, unmodelled_shapes[] }` | +| `owned_generators` | `{ unmodified_lines, total_lines, copies[] }`, one total across every port | +| `status_counts` | `{ new, open, fixed, deferred }` | Within each tier, sort by impact ÷ effort (quick wins first). Tier 1 leads. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md index 92cfb4748..4f5678520 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md @@ -3,7 +3,7 @@ ## Contents - Census scope - Keeping the result in place -- Anti-pattern list +- Anti-pattern list (includes the unlinked-payload ratio) - Template-text duplication **The census covers EVERY module**, not only the application: test harnesses, simulators, @@ -26,6 +26,24 @@ format-built argument at the project's own LLM request call sites. **sourceless `object.projection`** carrying those origins (#210 — assembly origins on an `object.value` fail load with `ERR_SUBTYPE_RULE_VIOLATION`; `@payloadRef` accepts the sourceless projection). +- **Unlinked payload.** A declared payload shape is necessary, not sufficient: the check above + asks whether the payload IS a shape, this one asks where its fields COME FROM. For each object + a `@payloadRef` names, class every field: + - **LINKED** — a dotted `extends: Entity.field` (for example `"extends": "Customer.email"`), + or an origin that resolves to an entity field (`origin.passthrough` with + `@from: Customer.email`, or an `aggregate` / `first` origin whose `@of` names one). + - **CALLER-SUPPLIED** — no entity owns the value (the user's question, a run-time setting, a + date the caller computes). Say why for each; "the caller fills it" is not a reason when the + caller reads it off an entity. + - **UNLINKED COPY** — declared as a free-standing field, while the call site fills it from an + entity property (`payload.email = customer.email`, `email=customer.email`). + Report `linked / (linked + unlinked copies)` per payload and in total. Each unlinked copy is a + finding (`unlinked-payload-field`): renaming or re-typing the entity field will not fail + `verify --templates`, because the payload declares its own copy and the template still + matches that copy. Remedy: replace the field with `extends: Entity.field` (or a passthrough + origin on a sourceless projection). The `parity_gate` is a scratch-copy proof: rename the + entity field in a throwaway copy of the metadata and confirm the project's verify now fails, + where before the change it passed. - Silent-degradation hack (`try/except KeyError` or `?? ''` around formatting) — flag every instance. - Hand-rolled output parsing (regex / XML / ad-hoc JSON) vs a declared **responding `template.prompt`** (one carrying `@responseRef`) + generated `parse*` / `safeParse*` / diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/python.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/python.md index 92c1d4250..08974a711 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/python.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/python.md @@ -80,6 +80,25 @@ the same way — a `module:symbol` entry in a target's `generators` in `metaobje reading the model through `metaobjects.codegen.model_walk`. (`--provider module:symbol` registers **metamodel vocabulary**, not a generator — do not read it as one.) +**Audit the owned copies first.** A copy exists to carry a change; one that carries none is +duplication that falls behind the reference. + +```bash +metaobjects gen --list # the catalog, each owned copy marked +``` + +An owned row ends `[owned — identical]` or `[owned — DIFFERS: N behind, M of your own]`, and +a generator that reads a `codegen/runtime/` module (`routes`: `filter_parser`, +`constraint_errors`) prints an indented `runtime []` line per companion. Only +`identical`, with every companion identical too, proves a copy unmodified; that row is a +**finding**: point the `metaobjects.config.yaml` entry back at the stock name and remove the +copy (`metaobjects eject --undo` removes every identical copy and prints the config edit). +`DIFFERS` with `0 of your own` rules out added lines, not removed ones: confirm from +`git log -p` on the copy that nothing changed after the eject, else report it as **possibly +customized: check its history**. A copy with lines of its own, or an edited companion, is a +customization; check only how far behind it is. Count lines for the Phase 0 total: lines in +copies proven unmodified, over all lines under `codegen/generators/` and `codegen/runtime/`. + **A finding of the form "the built-ins do not emit the shape this project needs", or a hand-written layer the model fully describes, resolves to a generator the project writes** (ADR-0034 Amendment 4), or to a declarative template (a target's `templateSpec: `, diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md index 98cb47063..4a5e29277 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md @@ -22,7 +22,9 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in grids/forms/filters vs `layout.dataGrid` / form generators / filter-allowlist. - [ ] **C. Drift hotspot — validators, mappers, runtime models.** Hand validators / DTO-mappers / dataclasses shadowing a generated shape. Diff field-by-field; the divergence is the bug. -- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`). +- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`), + including the **unlinked payload** count: for each `@payloadRef` object, how many of its + fields are linked to the entity field they copy. - [ ] **E. Owned generators & scaffold-and-own** (see "Owned-codegen & scaffold-and-own assessment" in `SKILL.md`). - [ ] **E2. Capability ledger — `requirement.*`** (see `references/requirements.md` and the Requirement axis of the capability checklist). The one axis whose hunt is hand-written @@ -34,6 +36,9 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in design, and a real estate reports most of itself unclaimed on day one). Score the prose it would replace, and prefer ONE concrete claim an author can check over a whole-estate migration. + Where a ledger exists, run the three **currency checks** in `references/requirements.md` + (ledger stale, `--require-implementers`, live claim with an unwired implementer) before + sampling statuses. **First establish the estate LOADS.** An older ledger fails the load outright, so `meta verify` never runs and everything in `references/requirements.md` is unavailable: `@violation` and `@verifiedBy` were retired in **0.24.0** (no deprecation shim — ADR-0023 @@ -53,6 +58,52 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in subverbs (`--codegen` / `--templates` / `--db`)? Committed-codegen freshness gate? Advisories heeded? Routine `--no-verify` bypass? Loader `ERR_*` / warnings addressed? Parse the stable `code` field, not message text (ADR-0009). + **No CI means NO, whatever local hooks exist.** A git hook is not part of the clone: it + counts only when a tracked setup step installs it (a `prepare` / `postinstall` script, a + `core.hooksPath` set by a committed setup script, a hook manager declared in the + project's dependencies). An untracked `.git/hooks/pre-push` on one machine is not a gate. + A project with no CI workflow and no tracked hook installer answers the Binary CI drift + gate **NO**, and that is a **Tier 1 finding** whose `next_command` is the snippet below + for the project's port, ready to paste. + + | Port | Verify commands for the snippet (one per subverb the project's artifacts need) | + |---|---| + | TypeScript | `npx meta verify --codegen --templates`; add `npx meta verify --db` against a provisioned database | + | Python | `metaobjects verify --codegen` and `metaobjects verify --templates` | + | Java / Kotlin | `mvn -B metaobjects:verify -Dmeta.verify.mode=codegen` and `-Dmeta.verify.mode=templates` | + | C# | `dotnet meta verify --codegen` and `dotnet meta verify --templates` | + + The snippet is one CI job that checks out the repo, installs the toolchain and runs those + commands with nothing after them, for example GitHub Actions steps + `- run: npx meta verify --codegen --templates`. Where the project has no CI host, it is a + tracked `pre-push` hook that runs the same commands plus the tracked setup line that + installs it. Schema drift (`--db`) runs through the Node `meta` CLI in every port. + **Does it block?** Wired is not blocking. A gate that runs and is ignored scores the same as + no gate, so answer all three before scoring this axis: + - **(a) The exit code propagates.** Search CI files, package scripts, Makefiles, shell + scripts and build files on every path that runs verify for anything that swallows its + failure: `|| true`, `|| echo`, `; exit 0`, `continue-on-error: true`, `set +e`, + `allow_failure: true`, `ignore_errors`, a step or job marked optional or non-required, + a Maven `testFailureIgnore` / `failOnError=false` style switch, an MSBuild + `ContinueOnError`, a `-Dmeta.lax=true` or other option that turns a verify failure into a + warning, and verify output piped into a command whose exit code is the one that counts + (`meta verify | tee log` without `pipefail`). Each one is a finding naming the file and + line. + - **(b) It is green now.** Run the project's own verify command, the exact one its gate + runs, and record the exit code and the error count. A non-zero exit the project tolerates + ("known errors", a baseline file, a count it compares against) is a finding that states + the count and recommends a **ratchet**: the gate stores today's count and fails when it + rises, so the number can only fall (see the CI ratchet gate in `metaobjects-verify`). A + tolerated count with no ratchet is a gate that will absorb every new error. + - **(c) Every route to production passes it.** List each deploy entry point: CI deploy + jobs, release workflows, deploy scripts, `Makefile` / package-script deploy targets, a + hosting platform's build command, a manual `deploy.sh`. Flag each one that ships without + verify having passed on the same commit (a deploy job that does not depend on the verify + job, a script that builds and pushes directly, a `skip-checks` / `--no-verify` input). + Score the gate **BLOCKING** (verify runs on every route to production and a failure stops + it), **RUNS BUT DOES NOT BLOCK** (it runs somewhere, but (a), (b) or (c) failed) or + **ABSENT**. Only **BLOCKING** passes the drift-gate pillar's tier and answers YES on the + Binary CI drift gate; report the other two as **NO** and name which of (a), (b), (c) failed. **Is any advisory half switched OFF?** A gate can be wired, green, and muted. The requirements authoring lint is advisory and mutable — `--no-requirement-lint` or `META_NO_REQUIREMENT_LINT=1` silences it while the gate above it still runs and can still @@ -96,9 +147,24 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in code imports: drop the generator rather than generate into the void. - **Wrong target / duplicate output** — the same logical artifact emitted to two places (a mis-set per-target `outDir`), one of which is orphaned. + - **Consumed / emitted, per generator kind.** For each generator kind (entity, routes, + query hooks, grid, form, and so on) report `consumed / emitted` files, where consumed is + confirmed by an import or reference from hand-written code, not by the file existing. + - **Generated and bypassed.** Where a kind is under half consumed and axis B found a + hand-written equivalent for the same entity (a hand-built admin page, form or grid + beside the unused generated one), raise ONE finding, `generated-and-bypassed`, that + names both files. This is not the same as dead output: the project needed the surface + and built it again by hand, so the generated one is a second, unused source of truth. + The app chooses per entity: **use the generated artifact** (customising the owned + generator until it fits, then deleting the hand-written page after a parity gate), or + **filter it out** with the generator's `filter` and record the hand-written page as + ACCEPTABLE with a stated reason. Emitted and unused is not an end state: the finding + closes only when each entity is on one side or the other. Recommend generating ONLY what is consumed — decide per generator: drop the ones whose whole output class nothing imports, and narrow the ones that over-emit with - their `filter`. A smaller, fully-consumed generated surface beats a large one with + their `filter`. Dropping is the remedy only when no hand-written equivalent exists; + where one does, the finding is `generated-and-bypassed` above. A smaller, + fully-consumed generated surface beats a large one with dead files (which also make the leverage ratio lie; discount them from the census). - [ ] **G. Runtime-contract anti-patterns.** Module-global `db` vs context-as-parameter (ADR-0008); wire-canonicalization in the query path vs native in-process return types diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md index 525029b69..5225faad9 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -74,7 +74,13 @@ Actual cutovers run through the existing skills mapped per finding tier (§ Brid `queriesFile` / `routesFile` / `barrel` on a 1.0.x install** fails to load, because 1.0 removed those four and 1.1 restored them: there it is an upgrade blocker, fixed by upgrading to 1.1 or by ejecting. Importing anything from that subpath is otherwise - correct and is never a finding by itself. + correct and is never a finding by itself. **Every port owns generators the same way**: + run the owned-copy check in each port reference the project uses (TypeScript + `meta eject --list --format text`, Python `metaobjects gen --list`, Java / Kotlin + `mvn metaobjects:eject -Dlist`, C# `dotnet meta gen --list`), and report ONE total across + all of them: **unmodified owned-generator lines / all owned-generator lines**, where + unmodified counts only copies proven unmodified. Report the proven copies together as + one finding, `owned-generator-stock-copies`, listing each copy and the port's undo command. - [ ] **Cross-language version consistency (silent-drift check).** If the project uses MetaObjects in more than one language (e.g. a TS web client + a Java/Python/C# backend), enumerate EVERY MetaObjects package across ALL ecosystems (npm `@metaobjectsdev/*`, Maven `com.metaobjects:*`, PyPI `metaobjects`, NuGet `MetaObjects.*`) and record each version. **The version-number LINES differ by ecosystem (npm/PyPI/NuGet `0.x`/`1.x` vs Maven `7.x`/`8.x`), so you CANNOT eyeball drift** — a `0.12` next to a `7.7` looks fine but can be badly out of sync. Compare the **`metamodelVersion`** each port reports (the shared spec version on the registry manifest): a mismatch is real cross-language drift and a **finding** — the ports disagree on vocabulary/wire behavior. Also flag any port not on the latest release for its ecosystem. (This is a known real-world failure mode: newest backend, stale client, invisible because the numbers differ.) - [ ] Classify: **Greenfield** (none/minimal) · **Partial** · **Deep** → choose path below. @@ -103,6 +109,20 @@ code grows. The gap between "modeled + query helpers" and "has view + route + UI" is the headline lopsidedness. +**Shape coverage census.** The coverage matrix lists modelled entities only, so a shape +nobody modelled never appears in it. Count the other side too. List every shape that crosses +a boundary: database tables and views, request and response bodies, queue and socket +messages, prompt inputs, LLM replies, and file formats the project reads or writes. For each +shape, record whether it is modelled, and for each layer that states it (DDL or migration, +ORM model, validator, DTO, client type, prompt payload, reply parser) whether that statement +is **generated**, **derived** from generated code (a type built from a generated type, a +`z.infer` of a generated schema), or **hand-declared**. Report +`generated or derived statements / all statements` and list the unmodelled shapes. A table +present in the database or the migrations with no entity or projection behind it is a +finding (`unmodelled-table`). Like the leverage ratio this is a measurement, never a target: +an unmodelled shape is a finding only where the cost test in § Owned-codegen says modelling it +pays, or it is a table. + **Surface review — 8 axes (independently runnable).** Work the full `references/capability-checklist.md` on every axis. Check calibration guards (`references/calibration.md`) before raising a finding. **Verify, don't assume** — read the @@ -113,7 +133,7 @@ Axes (the full hunt for each is in `references/surface-review-axes.md` — open - **A.** Codegen candidates — API / server routes · **B.** Codegen candidates — web / client - **C.** Drift hotspot — validators, mappers, runtime models · **D.** Prompt pillar - **E.** Owned generators and scaffold-and-own · **E2.** Capability ledger (`requirement.*`, opt-in) -- **F.** Drift-gate adoption · **F2.** Over-generation (codegen waste) +- **F.** Drift-gate adoption — does it block? · **F2.** Over-generation and generated-but-bypassed (codegen waste) - **G.** Runtime-contract anti-patterns · **G2.** `own*()` accessor discipline (a correctness defect, not advisory) - **H.** Authoring correctness / ADR conformance · **H2.** Wrong native type (`field.string` + `@dbColumnType: uuid`) - **I.** Vocabulary hygiene / modernization (advisory, never a failing finding) @@ -199,14 +219,36 @@ one-consumer need, read it codegen-locally. 1. **Headline MATURITY TIER** — Greenfield → Partial → Deep → Exemplary; worst-of with prerequisite gating (a missing pillar can't be averaged away); rendered with **the single - next unmet check** ("you're Partial; the next rung needs `verify` in CI"). + next unmet check** ("you're Partial; the next rung needs a blocking `verify` in CI"). 2. **Per-pillar breakdown (never rolled into one number)** — `pillar | tier | top gap` over codegen / runtime / drift-gate / prompts. This is the core deliverable. -3. **Binary CI drift gate** — prominent and separate: **"Is `meta verify` drift detection - wired into CI?"** It is binary because the risk is binary. - -Coarse bands only (none / some / most / all). Worst-of within a pillar. On re-run, grade -the delta. Lead with gaps, not the grade. +3. **Binary CI drift gate** — prominent and separate: **"Does `meta verify` drift detection + block every route to production?"** It is binary because the risk is binary. YES only + when axis F scores the gate **BLOCKING**; RUNS BUT DOES NOT BLOCK and ABSENT both answer + NO, with the reason (exit code swallowed, tolerated error count, a deploy path that + skips it, no CI). Local hooks with no tracked installer do not count. +4. **Used as described** — one line, separate from the tier: does the project use what it + adopted the way MetaObjects documents it? Built from four measurements, each shown with + its numbers: generated artifacts consumed rather than bypassed (axis F2, `consumed / + emitted` and the `generated-and-bypassed` count); the gate blocks (axis F score); prompt + payload fields linked to entity fields (axis D, `linked / (linked + unlinked copies)`); + and boundary shapes generated or derived (Phase 1b shape coverage census). Render it as + `Used as described: — generated forms 5/12 consumed · gate RUNS BUT DOES NOT + BLOCK · payload fields 6/10 linked · shapes 50% generated or derived`, banded by its worst + part. + +Coarse bands only (none / some / most / all). Worst-of within a pillar. Lead with gaps, not +the grade. + +**Re-run: compare against the previous audit.** If `.metaobjects/adoption-audit.json` +already exists, read it before starting and keep it as the baseline. Match each earlier +finding to this run's by `id` plus location, and give every earlier finding a `status`: +**fixed** (the location no longer shows it; say what changed), **open** (still present, +carrying its original `first_seen`), or **deferred** (the project recorded a decision not to +fix it now, in a commit, an ADR, a tracked issue or a code comment, quoted as +`deferred_reason`). A finding that is still present with no recorded decision is open, never +deferred. New findings are `new`. Report the counts per status at the top of the report, +and show each scorecard line beside its previous value. **Vocabulary hygiene (axis I) is advisory** — it surfaces as modernization opportunities in the roadmap, scored as such, and **never gates a tier or fails the @@ -218,8 +260,10 @@ audit.** Dated-but-working vocabulary is a quality nudge, not a defect. **Two artifacts:** `.metaobjects/adoption-audit.json` + rendered Markdown. -**Markdown sections (lead with Scorecard):** 0. Scorecard (tier + pillar table + CI gate) · -1. Triage + census · 2. Coverage matrix · 3. Per-surface classification tables · 4. Drift +**Markdown sections (lead with Scorecard):** 0. Scorecard (tier + pillar table + CI gate + +used as described; on a re-run, the fixed / open / deferred / new counts) · +1. Triage + census (including the owned-generator unmodified-lines total and the shape +coverage census) · 2. Coverage matrix · 3. Per-surface classification tables · 4. Drift findings (active bugs first) · 5. Owned-codegen + generator gaps · 6. Drift-gate adoption · 7. Runtime-contract + authoring-correctness (axes G+H) · 8. Semantic-constraint ratification · 9. Prompt-pillar assessment · 10. Prioritized roadmap: Tier 1 drift kill → Tier 2 existing @@ -243,6 +287,24 @@ Each roadmap item: LOC retired, prerequisite, **parity-gate before deleting hand | `next_command` | the exact command / skill that performs the cutover (see bridge below) | | `parity_gate` | the specific check proving behavior-equivalence | | `tier` | 1–4 | +| `status` | `new` / `open` / `fixed` / `deferred` against the previous audit (`new` on a first run) | +| `first_seen` | date of the audit that first raised it (carried forward on re-runs) | +| `deferred_reason` | for `deferred` only: the recorded decision, quoted, and where it is recorded | + +Fixed findings stay in the file with `status: fixed` for one run, so the next reader sees +what closed. + +**Top-level fields in `.metaobjects/adoption-audit.json`**, beside `findings[]`: + +| Field | Content | +|---|---| +| `audited_at`, `commit` | when the audit ran and the commit it read | +| `previous` | `audited_at` and `commit` of the baseline audit, or `null` on a first run | +| `tier`, `pillars[]` | headline tier and `{ pillar, tier, top_gap }` per pillar | +| `ci_gate` | `{ score: "BLOCKING" \| "RUNS_BUT_DOES_NOT_BLOCK" \| "ABSENT", exit_code, error_count, swallowed_at[], deploy_paths_skipping[] }` | +| `used_as_described` | `{ band, generated_consumed, generated_emitted, bypassed_count, gate_score, payload_fields_linked, payload_fields_unlinked, shape_statements_generated_or_derived, shape_statements_total, unmodelled_shapes[] }` | +| `owned_generators` | `{ unmodified_lines, total_lines, copies[] }`, one total across every port | +| `status_counts` | `{ new, open, fixed, deferred }` | Within each tier, sort by impact ÷ effort (quick wins first). Tier 1 leads. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md index 92cfb4748..4f5678520 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md @@ -3,7 +3,7 @@ ## Contents - Census scope - Keeping the result in place -- Anti-pattern list +- Anti-pattern list (includes the unlinked-payload ratio) - Template-text duplication **The census covers EVERY module**, not only the application: test harnesses, simulators, @@ -26,6 +26,24 @@ format-built argument at the project's own LLM request call sites. **sourceless `object.projection`** carrying those origins (#210 — assembly origins on an `object.value` fail load with `ERR_SUBTYPE_RULE_VIOLATION`; `@payloadRef` accepts the sourceless projection). +- **Unlinked payload.** A declared payload shape is necessary, not sufficient: the check above + asks whether the payload IS a shape, this one asks where its fields COME FROM. For each object + a `@payloadRef` names, class every field: + - **LINKED** — a dotted `extends: Entity.field` (for example `"extends": "Customer.email"`), + or an origin that resolves to an entity field (`origin.passthrough` with + `@from: Customer.email`, or an `aggregate` / `first` origin whose `@of` names one). + - **CALLER-SUPPLIED** — no entity owns the value (the user's question, a run-time setting, a + date the caller computes). Say why for each; "the caller fills it" is not a reason when the + caller reads it off an entity. + - **UNLINKED COPY** — declared as a free-standing field, while the call site fills it from an + entity property (`payload.email = customer.email`, `email=customer.email`). + Report `linked / (linked + unlinked copies)` per payload and in total. Each unlinked copy is a + finding (`unlinked-payload-field`): renaming or re-typing the entity field will not fail + `verify --templates`, because the payload declares its own copy and the template still + matches that copy. Remedy: replace the field with `extends: Entity.field` (or a passthrough + origin on a sourceless projection). The `parity_gate` is a scratch-copy proof: rename the + entity field in a throwaway copy of the metadata and confirm the project's verify now fails, + where before the change it passed. - Silent-degradation hack (`try/except KeyError` or `?? ''` around formatting) — flag every instance. - Hand-rolled output parsing (regex / XML / ad-hoc JSON) vs a declared **responding `template.prompt`** (one carrying `@responseRef`) + generated `parse*` / `safeParse*` / diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md index 98cb47063..4a5e29277 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md @@ -22,7 +22,9 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in grids/forms/filters vs `layout.dataGrid` / form generators / filter-allowlist. - [ ] **C. Drift hotspot — validators, mappers, runtime models.** Hand validators / DTO-mappers / dataclasses shadowing a generated shape. Diff field-by-field; the divergence is the bug. -- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`). +- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`), + including the **unlinked payload** count: for each `@payloadRef` object, how many of its + fields are linked to the entity field they copy. - [ ] **E. Owned generators & scaffold-and-own** (see "Owned-codegen & scaffold-and-own assessment" in `SKILL.md`). - [ ] **E2. Capability ledger — `requirement.*`** (see `references/requirements.md` and the Requirement axis of the capability checklist). The one axis whose hunt is hand-written @@ -34,6 +36,9 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in design, and a real estate reports most of itself unclaimed on day one). Score the prose it would replace, and prefer ONE concrete claim an author can check over a whole-estate migration. + Where a ledger exists, run the three **currency checks** in `references/requirements.md` + (ledger stale, `--require-implementers`, live claim with an unwired implementer) before + sampling statuses. **First establish the estate LOADS.** An older ledger fails the load outright, so `meta verify` never runs and everything in `references/requirements.md` is unavailable: `@violation` and `@verifiedBy` were retired in **0.24.0** (no deprecation shim — ADR-0023 @@ -53,6 +58,52 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in subverbs (`--codegen` / `--templates` / `--db`)? Committed-codegen freshness gate? Advisories heeded? Routine `--no-verify` bypass? Loader `ERR_*` / warnings addressed? Parse the stable `code` field, not message text (ADR-0009). + **No CI means NO, whatever local hooks exist.** A git hook is not part of the clone: it + counts only when a tracked setup step installs it (a `prepare` / `postinstall` script, a + `core.hooksPath` set by a committed setup script, a hook manager declared in the + project's dependencies). An untracked `.git/hooks/pre-push` on one machine is not a gate. + A project with no CI workflow and no tracked hook installer answers the Binary CI drift + gate **NO**, and that is a **Tier 1 finding** whose `next_command` is the snippet below + for the project's port, ready to paste. + + | Port | Verify commands for the snippet (one per subverb the project's artifacts need) | + |---|---| + | TypeScript | `npx meta verify --codegen --templates`; add `npx meta verify --db` against a provisioned database | + | Python | `metaobjects verify --codegen` and `metaobjects verify --templates` | + | Java / Kotlin | `mvn -B metaobjects:verify -Dmeta.verify.mode=codegen` and `-Dmeta.verify.mode=templates` | + | C# | `dotnet meta verify --codegen` and `dotnet meta verify --templates` | + + The snippet is one CI job that checks out the repo, installs the toolchain and runs those + commands with nothing after them, for example GitHub Actions steps + `- run: npx meta verify --codegen --templates`. Where the project has no CI host, it is a + tracked `pre-push` hook that runs the same commands plus the tracked setup line that + installs it. Schema drift (`--db`) runs through the Node `meta` CLI in every port. + **Does it block?** Wired is not blocking. A gate that runs and is ignored scores the same as + no gate, so answer all three before scoring this axis: + - **(a) The exit code propagates.** Search CI files, package scripts, Makefiles, shell + scripts and build files on every path that runs verify for anything that swallows its + failure: `|| true`, `|| echo`, `; exit 0`, `continue-on-error: true`, `set +e`, + `allow_failure: true`, `ignore_errors`, a step or job marked optional or non-required, + a Maven `testFailureIgnore` / `failOnError=false` style switch, an MSBuild + `ContinueOnError`, a `-Dmeta.lax=true` or other option that turns a verify failure into a + warning, and verify output piped into a command whose exit code is the one that counts + (`meta verify | tee log` without `pipefail`). Each one is a finding naming the file and + line. + - **(b) It is green now.** Run the project's own verify command, the exact one its gate + runs, and record the exit code and the error count. A non-zero exit the project tolerates + ("known errors", a baseline file, a count it compares against) is a finding that states + the count and recommends a **ratchet**: the gate stores today's count and fails when it + rises, so the number can only fall (see the CI ratchet gate in `metaobjects-verify`). A + tolerated count with no ratchet is a gate that will absorb every new error. + - **(c) Every route to production passes it.** List each deploy entry point: CI deploy + jobs, release workflows, deploy scripts, `Makefile` / package-script deploy targets, a + hosting platform's build command, a manual `deploy.sh`. Flag each one that ships without + verify having passed on the same commit (a deploy job that does not depend on the verify + job, a script that builds and pushes directly, a `skip-checks` / `--no-verify` input). + Score the gate **BLOCKING** (verify runs on every route to production and a failure stops + it), **RUNS BUT DOES NOT BLOCK** (it runs somewhere, but (a), (b) or (c) failed) or + **ABSENT**. Only **BLOCKING** passes the drift-gate pillar's tier and answers YES on the + Binary CI drift gate; report the other two as **NO** and name which of (a), (b), (c) failed. **Is any advisory half switched OFF?** A gate can be wired, green, and muted. The requirements authoring lint is advisory and mutable — `--no-requirement-lint` or `META_NO_REQUIREMENT_LINT=1` silences it while the gate above it still runs and can still @@ -96,9 +147,24 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in code imports: drop the generator rather than generate into the void. - **Wrong target / duplicate output** — the same logical artifact emitted to two places (a mis-set per-target `outDir`), one of which is orphaned. + - **Consumed / emitted, per generator kind.** For each generator kind (entity, routes, + query hooks, grid, form, and so on) report `consumed / emitted` files, where consumed is + confirmed by an import or reference from hand-written code, not by the file existing. + - **Generated and bypassed.** Where a kind is under half consumed and axis B found a + hand-written equivalent for the same entity (a hand-built admin page, form or grid + beside the unused generated one), raise ONE finding, `generated-and-bypassed`, that + names both files. This is not the same as dead output: the project needed the surface + and built it again by hand, so the generated one is a second, unused source of truth. + The app chooses per entity: **use the generated artifact** (customising the owned + generator until it fits, then deleting the hand-written page after a parity gate), or + **filter it out** with the generator's `filter` and record the hand-written page as + ACCEPTABLE with a stated reason. Emitted and unused is not an end state: the finding + closes only when each entity is on one side or the other. Recommend generating ONLY what is consumed — decide per generator: drop the ones whose whole output class nothing imports, and narrow the ones that over-emit with - their `filter`. A smaller, fully-consumed generated surface beats a large one with + their `filter`. Dropping is the remedy only when no hand-written equivalent exists; + where one does, the finding is `generated-and-bypassed` above. A smaller, + fully-consumed generated surface beats a large one with dead files (which also make the leverage ratio lie; discount them from the census). - [ ] **G. Runtime-contract anti-patterns.** Module-global `db` vs context-as-parameter (ADR-0008); wire-canonicalization in the query path vs native in-process return types diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/typescript.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/typescript.md index 3fc99245c..d9041026b 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/typescript.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-audit/references/typescript.md @@ -121,6 +121,11 @@ are the `--format text` labels. In `--format json` or `toon` the same row carrie - A copy with lines of its own is an intentional customization. Check only how far behind it is. +Count lines for the Phase 0 total: lines in copies proven unmodified, with the +`codegen/runtime/` files they reach, over all lines the project owns under `codegen/` +(`generators/`, `runtime/`, and the `templates/` modules an eject copies). +`meta eject --undo` removes every identical copy and prints the config edit. + **`entity`, `routes` and `routes-hono` own more than the generator file.** Ejecting one also copies the HTTP adapter its output imports into `codegen/runtime/`, and a route fix is often made there with the generator file left untouched. Read the state of those files diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/SKILL.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/SKILL.md index 525029b69..5225faad9 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/SKILL.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/SKILL.md @@ -74,7 +74,13 @@ Actual cutovers run through the existing skills mapped per finding tier (§ Brid `queriesFile` / `routesFile` / `barrel` on a 1.0.x install** fails to load, because 1.0 removed those four and 1.1 restored them: there it is an upgrade blocker, fixed by upgrading to 1.1 or by ejecting. Importing anything from that subpath is otherwise - correct and is never a finding by itself. + correct and is never a finding by itself. **Every port owns generators the same way**: + run the owned-copy check in each port reference the project uses (TypeScript + `meta eject --list --format text`, Python `metaobjects gen --list`, Java / Kotlin + `mvn metaobjects:eject -Dlist`, C# `dotnet meta gen --list`), and report ONE total across + all of them: **unmodified owned-generator lines / all owned-generator lines**, where + unmodified counts only copies proven unmodified. Report the proven copies together as + one finding, `owned-generator-stock-copies`, listing each copy and the port's undo command. - [ ] **Cross-language version consistency (silent-drift check).** If the project uses MetaObjects in more than one language (e.g. a TS web client + a Java/Python/C# backend), enumerate EVERY MetaObjects package across ALL ecosystems (npm `@metaobjectsdev/*`, Maven `com.metaobjects:*`, PyPI `metaobjects`, NuGet `MetaObjects.*`) and record each version. **The version-number LINES differ by ecosystem (npm/PyPI/NuGet `0.x`/`1.x` vs Maven `7.x`/`8.x`), so you CANNOT eyeball drift** — a `0.12` next to a `7.7` looks fine but can be badly out of sync. Compare the **`metamodelVersion`** each port reports (the shared spec version on the registry manifest): a mismatch is real cross-language drift and a **finding** — the ports disagree on vocabulary/wire behavior. Also flag any port not on the latest release for its ecosystem. (This is a known real-world failure mode: newest backend, stale client, invisible because the numbers differ.) - [ ] Classify: **Greenfield** (none/minimal) · **Partial** · **Deep** → choose path below. @@ -103,6 +109,20 @@ code grows. The gap between "modeled + query helpers" and "has view + route + UI" is the headline lopsidedness. +**Shape coverage census.** The coverage matrix lists modelled entities only, so a shape +nobody modelled never appears in it. Count the other side too. List every shape that crosses +a boundary: database tables and views, request and response bodies, queue and socket +messages, prompt inputs, LLM replies, and file formats the project reads or writes. For each +shape, record whether it is modelled, and for each layer that states it (DDL or migration, +ORM model, validator, DTO, client type, prompt payload, reply parser) whether that statement +is **generated**, **derived** from generated code (a type built from a generated type, a +`z.infer` of a generated schema), or **hand-declared**. Report +`generated or derived statements / all statements` and list the unmodelled shapes. A table +present in the database or the migrations with no entity or projection behind it is a +finding (`unmodelled-table`). Like the leverage ratio this is a measurement, never a target: +an unmodelled shape is a finding only where the cost test in § Owned-codegen says modelling it +pays, or it is a table. + **Surface review — 8 axes (independently runnable).** Work the full `references/capability-checklist.md` on every axis. Check calibration guards (`references/calibration.md`) before raising a finding. **Verify, don't assume** — read the @@ -113,7 +133,7 @@ Axes (the full hunt for each is in `references/surface-review-axes.md` — open - **A.** Codegen candidates — API / server routes · **B.** Codegen candidates — web / client - **C.** Drift hotspot — validators, mappers, runtime models · **D.** Prompt pillar - **E.** Owned generators and scaffold-and-own · **E2.** Capability ledger (`requirement.*`, opt-in) -- **F.** Drift-gate adoption · **F2.** Over-generation (codegen waste) +- **F.** Drift-gate adoption — does it block? · **F2.** Over-generation and generated-but-bypassed (codegen waste) - **G.** Runtime-contract anti-patterns · **G2.** `own*()` accessor discipline (a correctness defect, not advisory) - **H.** Authoring correctness / ADR conformance · **H2.** Wrong native type (`field.string` + `@dbColumnType: uuid`) - **I.** Vocabulary hygiene / modernization (advisory, never a failing finding) @@ -199,14 +219,36 @@ one-consumer need, read it codegen-locally. 1. **Headline MATURITY TIER** — Greenfield → Partial → Deep → Exemplary; worst-of with prerequisite gating (a missing pillar can't be averaged away); rendered with **the single - next unmet check** ("you're Partial; the next rung needs `verify` in CI"). + next unmet check** ("you're Partial; the next rung needs a blocking `verify` in CI"). 2. **Per-pillar breakdown (never rolled into one number)** — `pillar | tier | top gap` over codegen / runtime / drift-gate / prompts. This is the core deliverable. -3. **Binary CI drift gate** — prominent and separate: **"Is `meta verify` drift detection - wired into CI?"** It is binary because the risk is binary. - -Coarse bands only (none / some / most / all). Worst-of within a pillar. On re-run, grade -the delta. Lead with gaps, not the grade. +3. **Binary CI drift gate** — prominent and separate: **"Does `meta verify` drift detection + block every route to production?"** It is binary because the risk is binary. YES only + when axis F scores the gate **BLOCKING**; RUNS BUT DOES NOT BLOCK and ABSENT both answer + NO, with the reason (exit code swallowed, tolerated error count, a deploy path that + skips it, no CI). Local hooks with no tracked installer do not count. +4. **Used as described** — one line, separate from the tier: does the project use what it + adopted the way MetaObjects documents it? Built from four measurements, each shown with + its numbers: generated artifacts consumed rather than bypassed (axis F2, `consumed / + emitted` and the `generated-and-bypassed` count); the gate blocks (axis F score); prompt + payload fields linked to entity fields (axis D, `linked / (linked + unlinked copies)`); + and boundary shapes generated or derived (Phase 1b shape coverage census). Render it as + `Used as described: — generated forms 5/12 consumed · gate RUNS BUT DOES NOT + BLOCK · payload fields 6/10 linked · shapes 50% generated or derived`, banded by its worst + part. + +Coarse bands only (none / some / most / all). Worst-of within a pillar. Lead with gaps, not +the grade. + +**Re-run: compare against the previous audit.** If `.metaobjects/adoption-audit.json` +already exists, read it before starting and keep it as the baseline. Match each earlier +finding to this run's by `id` plus location, and give every earlier finding a `status`: +**fixed** (the location no longer shows it; say what changed), **open** (still present, +carrying its original `first_seen`), or **deferred** (the project recorded a decision not to +fix it now, in a commit, an ADR, a tracked issue or a code comment, quoted as +`deferred_reason`). A finding that is still present with no recorded decision is open, never +deferred. New findings are `new`. Report the counts per status at the top of the report, +and show each scorecard line beside its previous value. **Vocabulary hygiene (axis I) is advisory** — it surfaces as modernization opportunities in the roadmap, scored as such, and **never gates a tier or fails the @@ -218,8 +260,10 @@ audit.** Dated-but-working vocabulary is a quality nudge, not a defect. **Two artifacts:** `.metaobjects/adoption-audit.json` + rendered Markdown. -**Markdown sections (lead with Scorecard):** 0. Scorecard (tier + pillar table + CI gate) · -1. Triage + census · 2. Coverage matrix · 3. Per-surface classification tables · 4. Drift +**Markdown sections (lead with Scorecard):** 0. Scorecard (tier + pillar table + CI gate + +used as described; on a re-run, the fixed / open / deferred / new counts) · +1. Triage + census (including the owned-generator unmodified-lines total and the shape +coverage census) · 2. Coverage matrix · 3. Per-surface classification tables · 4. Drift findings (active bugs first) · 5. Owned-codegen + generator gaps · 6. Drift-gate adoption · 7. Runtime-contract + authoring-correctness (axes G+H) · 8. Semantic-constraint ratification · 9. Prompt-pillar assessment · 10. Prioritized roadmap: Tier 1 drift kill → Tier 2 existing @@ -243,6 +287,24 @@ Each roadmap item: LOC retired, prerequisite, **parity-gate before deleting hand | `next_command` | the exact command / skill that performs the cutover (see bridge below) | | `parity_gate` | the specific check proving behavior-equivalence | | `tier` | 1–4 | +| `status` | `new` / `open` / `fixed` / `deferred` against the previous audit (`new` on a first run) | +| `first_seen` | date of the audit that first raised it (carried forward on re-runs) | +| `deferred_reason` | for `deferred` only: the recorded decision, quoted, and where it is recorded | + +Fixed findings stay in the file with `status: fixed` for one run, so the next reader sees +what closed. + +**Top-level fields in `.metaobjects/adoption-audit.json`**, beside `findings[]`: + +| Field | Content | +|---|---| +| `audited_at`, `commit` | when the audit ran and the commit it read | +| `previous` | `audited_at` and `commit` of the baseline audit, or `null` on a first run | +| `tier`, `pillars[]` | headline tier and `{ pillar, tier, top_gap }` per pillar | +| `ci_gate` | `{ score: "BLOCKING" \| "RUNS_BUT_DOES_NOT_BLOCK" \| "ABSENT", exit_code, error_count, swallowed_at[], deploy_paths_skipping[] }` | +| `used_as_described` | `{ band, generated_consumed, generated_emitted, bypassed_count, gate_score, payload_fields_linked, payload_fields_unlinked, shape_statements_generated_or_derived, shape_statements_total, unmodelled_shapes[] }` | +| `owned_generators` | `{ unmodified_lines, total_lines, copies[] }`, one total across every port | +| `status_counts` | `{ new, open, fixed, deferred }` | Within each tier, sort by impact ÷ effort (quick wins first). Tier 1 leads. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md index 92cfb4748..4f5678520 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/prompt-anti-patterns.md @@ -3,7 +3,7 @@ ## Contents - Census scope - Keeping the result in place -- Anti-pattern list +- Anti-pattern list (includes the unlinked-payload ratio) - Template-text duplication **The census covers EVERY module**, not only the application: test harnesses, simulators, @@ -26,6 +26,24 @@ format-built argument at the project's own LLM request call sites. **sourceless `object.projection`** carrying those origins (#210 — assembly origins on an `object.value` fail load with `ERR_SUBTYPE_RULE_VIOLATION`; `@payloadRef` accepts the sourceless projection). +- **Unlinked payload.** A declared payload shape is necessary, not sufficient: the check above + asks whether the payload IS a shape, this one asks where its fields COME FROM. For each object + a `@payloadRef` names, class every field: + - **LINKED** — a dotted `extends: Entity.field` (for example `"extends": "Customer.email"`), + or an origin that resolves to an entity field (`origin.passthrough` with + `@from: Customer.email`, or an `aggregate` / `first` origin whose `@of` names one). + - **CALLER-SUPPLIED** — no entity owns the value (the user's question, a run-time setting, a + date the caller computes). Say why for each; "the caller fills it" is not a reason when the + caller reads it off an entity. + - **UNLINKED COPY** — declared as a free-standing field, while the call site fills it from an + entity property (`payload.email = customer.email`, `email=customer.email`). + Report `linked / (linked + unlinked copies)` per payload and in total. Each unlinked copy is a + finding (`unlinked-payload-field`): renaming or re-typing the entity field will not fail + `verify --templates`, because the payload declares its own copy and the template still + matches that copy. Remedy: replace the field with `extends: Entity.field` (or a passthrough + origin on a sourceless projection). The `parity_gate` is a scratch-copy proof: rename the + entity field in a throwaway copy of the metadata and confirm the project's verify now fails, + where before the change it passed. - Silent-degradation hack (`try/except KeyError` or `?? ''` around formatting) — flag every instance. - Hand-rolled output parsing (regex / XML / ad-hoc JSON) vs a declared **responding `template.prompt`** (one carrying `@responseRef`) + generated `parse*` / `safeParse*` / diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/requirements.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/requirements.md index 5c48284f2..2bcebe9ec 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/requirements.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/requirements.md @@ -11,6 +11,7 @@ what a green run already proves. ## Contents - What verify has already proven (do not re-check by hand) +- Currency checks — run these before sampling - What only a human or an agent reading the code can catch - Scope — do NOT flag these as defects - Reporting @@ -54,12 +55,46 @@ Three limits, and each puts something back on your list: `title` is deliberately NOT flagged as an inert slot; it is chartered as the entry's label and is rendered (see item 6). Do not report a populated `title` as a defect. +## Currency checks — run these before sampling + +A green gate proves the ledger's links resolve, not that the ledger keeps up with the code. +These three checks measure that, and the first one chooses the sample for item 1 below. + +**(a) Ledger stale.** From git history, take the commits that add a feature (a `feat:` / +`feat(...)` subject under Conventional Commits; otherwise merged feature branches or PRs +labelled as features) over the last six months or the last 50 such commits, whichever is +fewer. Report two numbers: the share of those commits that also change requirement metadata +(any file holding `requirement.*` nodes), and the count of feature commits since the most +recent change to that metadata. A share **under half**, or **more than ten** feature commits +since the last ledger change, is the finding `ledger-stale`, stating both numbers. The +feature commits that did not touch the ledger are the sample for item 1: read what each one +built and check that a requirement says so with a true status. + +**(b) Nothing implements a live claim.** Run the project's verify with the strict switch: +`meta verify --require-implementers`, `metaobjects verify --require-implementers`, +`mvn metaobjects:verify -Dmeta.verify.requireImplementers=true` or +`dotnet meta verify --require-implementers` (all ports also read +`META_REQUIRE_IMPLEMENTERS=1`). Each `WARN_REQUIREMENT_NOTHING_IMPLEMENTS` on a `live` entry +is a finding (`live-claim-unimplemented`) naming the entry. State whether the project's own +gate passes the switch; if it does not and the ledger has no such entries, recommend adding +it so the ledger cannot fall behind again. + +**(c) Live claim, unwired implementer.** For each `@implementedBy` target of a `live` entry, +find the generated code that target produces and confirm hand-written code imports or calls +it. A live claim whose implementer is generated code nothing uses is the finding +`live-claim-unwired-implementer`: the claim resolves, so verify is satisfied, but the +capability is not reachable. Before raising it, rule out a runtime use: code that reaches the +entity through the runtime `ObjectManager` (or a metadata-driven admin or tool registry) +uses it without importing anything generated, so search for the entity's name in those calls +too. + ## What only a human or an agent reading the code can catch **1. Statuses that are false.** The highest-value finding in the whole dimension. A requirement marked `live` whose implementation was gutted; one marked `partial` that is now complete; one marked `planned` that was quietly built months ago. Sample the claims -and read the nodes. `status` is the only payload with controlled evidence behind it — the +and read the nodes, starting with the feature commits currency check (a) found outside the +ledger. `status` is the only payload with controlled evidence behind it — the resurrection protection rides entirely on it being true. **2. Non-violable statements.** Every requirement must state what breaking it looks like. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md index 98cb47063..4a5e29277 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/surface-review-axes.md @@ -22,7 +22,9 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in grids/forms/filters vs `layout.dataGrid` / form generators / filter-allowlist. - [ ] **C. Drift hotspot — validators, mappers, runtime models.** Hand validators / DTO-mappers / dataclasses shadowing a generated shape. Diff field-by-field; the divergence is the bug. -- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`). +- [ ] **D. Prompt pillar.** Every LLM prompt-construction site (see `prompt-anti-patterns.md`), + including the **unlinked payload** count: for each `@payloadRef` object, how many of its + fields are linked to the entity field they copy. - [ ] **E. Owned generators & scaffold-and-own** (see "Owned-codegen & scaffold-and-own assessment" in `SKILL.md`). - [ ] **E2. Capability ledger — `requirement.*`** (see `references/requirements.md` and the Requirement axis of the capability checklist). The one axis whose hunt is hand-written @@ -34,6 +36,9 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in design, and a real estate reports most of itself unclaimed on day one). Score the prose it would replace, and prefer ONE concrete claim an author can check over a whole-estate migration. + Where a ledger exists, run the three **currency checks** in `references/requirements.md` + (ledger stale, `--require-implementers`, live claim with an unwired implementer) before + sampling statuses. **First establish the estate LOADS.** An older ledger fails the load outright, so `meta verify` never runs and everything in `references/requirements.md` is unavailable: `@violation` and `@verifiedBy` were retired in **0.24.0** (no deprecation shim — ADR-0023 @@ -53,6 +58,52 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in subverbs (`--codegen` / `--templates` / `--db`)? Committed-codegen freshness gate? Advisories heeded? Routine `--no-verify` bypass? Loader `ERR_*` / warnings addressed? Parse the stable `code` field, not message text (ADR-0009). + **No CI means NO, whatever local hooks exist.** A git hook is not part of the clone: it + counts only when a tracked setup step installs it (a `prepare` / `postinstall` script, a + `core.hooksPath` set by a committed setup script, a hook manager declared in the + project's dependencies). An untracked `.git/hooks/pre-push` on one machine is not a gate. + A project with no CI workflow and no tracked hook installer answers the Binary CI drift + gate **NO**, and that is a **Tier 1 finding** whose `next_command` is the snippet below + for the project's port, ready to paste. + + | Port | Verify commands for the snippet (one per subverb the project's artifacts need) | + |---|---| + | TypeScript | `npx meta verify --codegen --templates`; add `npx meta verify --db` against a provisioned database | + | Python | `metaobjects verify --codegen` and `metaobjects verify --templates` | + | Java / Kotlin | `mvn -B metaobjects:verify -Dmeta.verify.mode=codegen` and `-Dmeta.verify.mode=templates` | + | C# | `dotnet meta verify --codegen` and `dotnet meta verify --templates` | + + The snippet is one CI job that checks out the repo, installs the toolchain and runs those + commands with nothing after them, for example GitHub Actions steps + `- run: npx meta verify --codegen --templates`. Where the project has no CI host, it is a + tracked `pre-push` hook that runs the same commands plus the tracked setup line that + installs it. Schema drift (`--db`) runs through the Node `meta` CLI in every port. + **Does it block?** Wired is not blocking. A gate that runs and is ignored scores the same as + no gate, so answer all three before scoring this axis: + - **(a) The exit code propagates.** Search CI files, package scripts, Makefiles, shell + scripts and build files on every path that runs verify for anything that swallows its + failure: `|| true`, `|| echo`, `; exit 0`, `continue-on-error: true`, `set +e`, + `allow_failure: true`, `ignore_errors`, a step or job marked optional or non-required, + a Maven `testFailureIgnore` / `failOnError=false` style switch, an MSBuild + `ContinueOnError`, a `-Dmeta.lax=true` or other option that turns a verify failure into a + warning, and verify output piped into a command whose exit code is the one that counts + (`meta verify | tee log` without `pipefail`). Each one is a finding naming the file and + line. + - **(b) It is green now.** Run the project's own verify command, the exact one its gate + runs, and record the exit code and the error count. A non-zero exit the project tolerates + ("known errors", a baseline file, a count it compares against) is a finding that states + the count and recommends a **ratchet**: the gate stores today's count and fails when it + rises, so the number can only fall (see the CI ratchet gate in `metaobjects-verify`). A + tolerated count with no ratchet is a gate that will absorb every new error. + - **(c) Every route to production passes it.** List each deploy entry point: CI deploy + jobs, release workflows, deploy scripts, `Makefile` / package-script deploy targets, a + hosting platform's build command, a manual `deploy.sh`. Flag each one that ships without + verify having passed on the same commit (a deploy job that does not depend on the verify + job, a script that builds and pushes directly, a `skip-checks` / `--no-verify` input). + Score the gate **BLOCKING** (verify runs on every route to production and a failure stops + it), **RUNS BUT DOES NOT BLOCK** (it runs somewhere, but (a), (b) or (c) failed) or + **ABSENT**. Only **BLOCKING** passes the drift-gate pillar's tier and answers YES on the + Binary CI drift gate; report the other two as **NO** and name which of (a), (b), (c) failed. **Is any advisory half switched OFF?** A gate can be wired, green, and muted. The requirements authoring lint is advisory and mutable — `--no-requirement-lint` or `META_NO_REQUIREMENT_LINT=1` silences it while the gate above it still runs and can still @@ -96,9 +147,24 @@ The full hunt for each axis of Phase 1b (Partial / Deep path). Each axis runs in code imports: drop the generator rather than generate into the void. - **Wrong target / duplicate output** — the same logical artifact emitted to two places (a mis-set per-target `outDir`), one of which is orphaned. + - **Consumed / emitted, per generator kind.** For each generator kind (entity, routes, + query hooks, grid, form, and so on) report `consumed / emitted` files, where consumed is + confirmed by an import or reference from hand-written code, not by the file existing. + - **Generated and bypassed.** Where a kind is under half consumed and axis B found a + hand-written equivalent for the same entity (a hand-built admin page, form or grid + beside the unused generated one), raise ONE finding, `generated-and-bypassed`, that + names both files. This is not the same as dead output: the project needed the surface + and built it again by hand, so the generated one is a second, unused source of truth. + The app chooses per entity: **use the generated artifact** (customising the owned + generator until it fits, then deleting the hand-written page after a parity gate), or + **filter it out** with the generator's `filter` and record the hand-written page as + ACCEPTABLE with a stated reason. Emitted and unused is not an end state: the finding + closes only when each entity is on one side or the other. Recommend generating ONLY what is consumed — decide per generator: drop the ones whose whole output class nothing imports, and narrow the ones that over-emit with - their `filter`. A smaller, fully-consumed generated surface beats a large one with + their `filter`. Dropping is the remedy only when no hand-written equivalent exists; + where one does, the finding is `generated-and-bypassed` above. A smaller, + fully-consumed generated surface beats a large one with dead files (which also make the leverage ratio lie; discount them from the census). - [ ] **G. Runtime-contract anti-patterns.** Module-global `db` vs context-as-parameter (ADR-0008); wire-canonicalization in the query path vs native in-process return types diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/typescript.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/typescript.md index 3fc99245c..d9041026b 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/typescript.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-audit/references/typescript.md @@ -121,6 +121,11 @@ are the `--format text` labels. In `--format json` or `toon` the same row carrie - A copy with lines of its own is an intentional customization. Check only how far behind it is. +Count lines for the Phase 0 total: lines in copies proven unmodified, with the +`codegen/runtime/` files they reach, over all lines the project owns under `codegen/` +(`generators/`, `runtime/`, and the `templates/` modules an eject copies). +`meta eject --undo` removes every identical copy and prints the config edit. + **`entity`, `routes` and `routes-hono` own more than the generator file.** Ejecting one also copies the HTTP adapter its output imports into `codegen/runtime/`, and a route fix is often made there with the generator file left untouched. Read the state of those files