Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .agent/repo=.this/role=any/boot.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
always:
briefs:
say:
- briefs/practices/behavior.verification/rule.require.acceptance-full-roundtrip.md
skills:
say:
- skills/use.testdb.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# tldr

## severity: blocker

acceptance tests must **fully round-trip** the deliverable — drive real input through the
real system and read the real result back — not merely assert on static output text.

a test that stops at "the tool produced this text" proves the artifact was *authored*, never
that it *works*. the round-trip proves it works: apply what was produced, exercise it, and
read the effect back through the same public surface a caller would use.

---
---
---

# deets

## .what

an acceptance test proves a promise to a real caller. to earn that proof it must complete the
whole loop the caller completes:

1. **produce** — invoke the contract (run the CLI, call the endpoint, build the artifact)
2. **apply** — put the produced artifact into the real system it targets (apply the generated
schema to a real postgres, deploy the built config, load the generated module)
3. **exercise** — drive real input through it (call the generated function, hit the endpoint,
run the generated code)
4. **read back** — observe the real effect through the same public surface a caller reads
(query the generated view, read the response body, inspect the returned value)
5. **assert + snapshot** — assert the round-tripped result equals what was put in, and snapshot
it so a reviewer sees the real caller experience in a diff

a test that performs only steps 1 and 5 on the *text* of the produced artifact (e.g. reads a
generated `.sql` file and greps it for a column type) is a **static-output assertion**, not an
acceptance test. it is welcome as an extra check, but it does not satisfy this rule on its own.

## .why

- **authored is not the same as works.** generated SQL can read perfectly and still fail to
apply (a reserved word, a bad type, a broken constraint). only a real apply + execute proves
it runs.
- **the caller lives at the far end of the loop.** the promise is "you can use this," not "this
text exists." the round-trip is the only test that stands where the caller stands.
- **drift hides in the gap.** a generator can drift such that its output text still matches an
old snapshot yet no longer executes. the round-trip catches what a text snapshot cannot.
- **the effect is the truth.** the returned row, the response body, the read-back value — that
is the deliverable. assert on the effect, not on the recipe that was meant to cause it.

## severity: blocker

an acceptance test that only asserts on static output text, with no real apply + execute +
read-back, gives false confidence: it turns green while the deliverable is broken for every
caller. that is the exact failure this gate exists to prevent, so an acceptance suite without a
full round-trip for each contract is a blocker.

## .where

- every `*.acceptance.test.ts` that covers a contract which produces an artifact meant to run
or be applied (codegen output, schema DDL, built config, generated client)
- the round-trip uses the real dependency (a real postgres via `rhx use.testdb`), never a mock
— consistent with `rule.forbid.acceptance.mocks`

## .when

- applies whenever the contract's output is *executable* or *applyable* (SQL, code, config)
- does **not** demand a round-trip for a contract whose output is purely terminal text with no
downstream system (e.g. a `--help` screen) — there, the stdout snapshot IS the full effect
- when a specific artifact cannot itself be applied (e.g. it transitively emits a reserved
identifier the tool does not quote), round-trip an equivalent self-contained artifact that
exercises the identical code path, and record why the original could not serve

## .how

- provision the real dependency in setup (`rhx use.testdb`, or the CI `start:testdb` step)
- apply the LITERAL produced artifact (read the generated file and run it), so the test proves
the on-disk output, not an in-memory re-derivation
- drive input through the produced surface, read the result back through the produced surface
- assert the read-back equals the input, then snapshot a stabilized view of it (strip volatile
db-generated keys) so the caller experience is legible in a PR diff
- cover the mutation lifecycle where one exists: write, re-write unchanged (no-op), change
(one effect) — so idempotency and change-detection are proven, not assumed

## .examples

### positive

```ts
// produce -> apply -> exercise -> read back -> assert + snapshot
execSync('./bin/run generate -c config.yml'); // produce
await db.query({ sql: readGenerated('tables/parcel.sql') }); // apply the literal output
await db.query({ sql: readGenerated('functions/upsert_parcel.sql') });
await db.query({ sql: readGenerated('views/view_parcel_current.sql') });
const id = await upsertParcel({ tags: ['a', 'b'], land_use: ['RESIDENTIAL'] }); // exercise
const row = await db.query({ sql: `select * from view_parcel_current where id = ${id}` }); // read back
expect(row.tags).toEqual(['a', 'b']); // assert the effect
expect(asStableRow(row)).toMatchSnapshot(); // snapshot the caller experience
```

### negative

```ts
// static-output assertion only: proves the text was authored, never that it runs
execSync('./bin/run generate -c config.yml');
const sql = readGenerated('functions/upsert_parcel.sql');
expect(sql).toContain('in_tags varchar[]'); // the recipe reads right...
expect(sql).toMatchSnapshot(); // ...but no apply or execute ever ran
```

## .see also

- `rule.forbid.acceptance.mocks` — the round-trip uses the real dependency, never a mock
- `rule.require.acceptance.blackbox` — the round-trip drives + reads through the public surface
- `rule.require.acceptance-journey-coverage` — the journeys a round-trip must cover
- `skills/use.testdb.sh` — provisions the real postgres the round-trip needs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
branch: beav/native-primitive-enum-array-columns
bound_by: init.behavior skill
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# ignore all peer-review files
*
!.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
branch: beav/native-primitive-enum-array-columns
bound_by: route.bind skill
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# ignore all except passage.jsonl and .bind flags
*
!.gitignore
!passage.jsonl
!.bind.*
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
{"stone":"1.vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"1.vision","status":"blocked","blocker":"review.self","reason":"review.self required: has-grounded-in-reality"}
{"stone":"1.vision","status":"promised","reason":"promised review.self: has-grounded-in-reality"}
{"stone":"1.vision","status":"promised","reason":"promised review.self: has-questioned-requirements"}
{"stone":"1.vision","status":"promised","reason":"promised review.self: has-questioned-assumptions"}
{"stone":"1.vision","status":"promised","reason":"promised review.self: has-questioned-questions"}
{"stone":"1.vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"1.vision","status":"blocked","blocker":"approval","reason":"wait for human approval"}
{"stone":"1.vision","status":"approved"}
{"stone":"1.vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"1.vision","status":"passed"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"blocked","blocker":"review.self","reason":"review.self required: has-pruned-yagni"}
{"stone":"5.1.execution.from_vision","status":"promised","reason":"promised review.self: has-pruned-yagni"}
{"stone":"5.1.execution.from_vision","status":"promised","reason":"promised review.self: has-pruned-backcompat"}
{"stone":"5.1.execution.from_vision","status":"promised","reason":"promised review.self: has-consistent-mechanisms"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"blocked","blocker":"review.self","reason":"review.self required: has-consistent-conventions"}
{"stone":"5.1.execution.from_vision","status":"promised","reason":"promised review.self: has-consistent-conventions"}
{"stone":"5.1.execution.from_vision","status":"promised","reason":"promised review.self: behavior-declaration-coverage"}
{"stone":"5.1.execution.from_vision","status":"promised","reason":"promised review.self: behavior-declaration-adherance"}
{"stone":"5.1.execution.from_vision","status":"promised","reason":"promised review.self: role-standards-adherance"}
{"stone":"5.1.execution.from_vision","status":"promised","reason":"promised review.self: role-standards-coverage"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"blocked"}
{"stone":"5.1.execution.from_vision","status":"malfunction"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"blocked","blocker":"review.peer","reason":"blockers exceed threshold (15 > 0)"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"blocked","blocker":"review.peer","reason":"blockers exceed threshold (1 > 0)"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"blocked","blocker":"review.peer","reason":"no review files found for hash 92f335a3"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"blocked","blocker":"review.peer","reason":"no review files found for hash 3662c60a"}
{"stone":"5.1.execution.from_vision","status":"blocked","blocker":"review.peer","reason":"no review files found for hash 3ac7771a"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"blocked","blocker":"review.peer","reason":"blockers exceed threshold (1 > 0)"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"exhausted","reason":"peer reviewer budget exhausted: enroll-impl-behavior-intent"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"blocked","blocker":"review.peer","reason":"nitpicks exceed threshold (4 > 3)"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"exhausted","reason":"peer reviewer budget exhausted: enroll-impl-behavior-intent"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"blocked","blocker":"review.peer","reason":"blockers exceed threshold (1 > 0)"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"blocked","blocker":"review.peer","reason":"nitpicks exceed threshold (4 > 3)"}
{"stone":"5.1.execution.from_vision","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.1.execution.from_vision","status":"passed"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"blocked","blocker":"review.self","reason":"review.self required: has-behavior-coverage"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-behavior-coverage"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-zero-test-skips"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-all-tests-passed"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-preserved-test-intentions"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-snap-changes-rationalized"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-critical-paths-frictionless"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-ergonomics-validated"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-fixed-all-gaps"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"blocked","blocker":"review.peer","reason":"blockers exceed threshold (2 > 0)"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"blocked","blocker":"review.peer.uncontemplated","reason":"peer review awaits contemplation: enroll-verif-snapshot-blemishes"}
{"stone":"5.3.verification","status":"blocked","blocker":"review.peer.uncontemplated","reason":"peer review awaits contemplation: enroll-verif-snapshot-blemishes"}
{"stone":"5.3.verification","status":"contemplated","reason":"contemplated review.peer: enroll-verif-snapshot-blemishes"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"malfunction"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"blocked","blocker":"review.peer","reason":"blockers exceed threshold (10 > 0)"}
{"stone":"5.3.verification","status":"contemplated","reason":"contemplated review.peer: enroll-verif-snapshot-coverage"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"passed"}
{"stone":"5.3.verification","status":"rewound"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"blocked","blocker":"review.self","reason":"review.self required: has-behavior-coverage"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-behavior-coverage"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-zero-test-skips"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-all-tests-passed"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-preserved-test-intentions"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-snap-changes-rationalized"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-critical-paths-frictionless"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-ergonomics-validated"}
{"stone":"5.3.verification","status":"promised","reason":"promised review.self: has-fixed-all-gaps"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"blocked","blocker":"review.peer","reason":"blockers exceed threshold (10 > 0)"}
{"stone":"5.3.verification","status":"contemplated","reason":"contemplated review.peer: enroll-verif-snapshot-coverage"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"passed"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"passed"}
{"stone":"5.3.verification","status":"passed"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"blocked","blocker":"review.peer","reason":"blockers exceed threshold (4 > 0)"}
{"stone":"5.3.verification","status":"contemplated","reason":"contemplated review.peer: mech-test-scope-purity"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"passed"}
{"stone":"5.3.verification","status":"arrived","reason":"entered guard reviews"}
{"stone":"5.3.verification","status":"passed"}
Loading
Loading