Skip to content

Commit 8d0ce29

Browse files
claudeclaude
committed
skills(automation): deduplicate the send-back eval (AUTO-H-01)
The eval restated SKILL.md's "Send-back for revision (ADR-0044)" section rather than testing beyond it: the canonical shape, the four lint findings, the wait-vs-approval_revise rationale and the showcase pointer all had a second copy here. 2,204 tokens -- 15% of the authored package -- went to a surface with one real authoring in the whole corpus, while `notify` (20 authorings) got 15. Delete the Expected-Output fence and the Common-Mistakes rows that restate the skill; keep Scenario + Validation Criteria, which is what a grader needs. The one row that did NOT restate SKILL.md (a "revise mode" re-suspend of the approval node) survives as criterion 7, so no graded fact is lost. 1,329 -> 550 tokens (ceiling 1,329). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1
1 parent a98b61b commit 8d0ce29

1 file changed

Lines changed: 8 additions & 52 deletions

File tree

‎skills/objectstack-automation/evals/approvals/test-revise-loop.md‎

Lines changed: 8 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,9 @@ revision* step emits the full ADR-0044 shape — a `revise` branch, an
55
`approval_revise` window node, and a resubmit edge typed `type: 'back'` — so the
66
flow **registers** and the loop actually works at run time.
77

8-
Skill rule referenced: `SKILL.md` → "Send-back for revision (ADR-0044)".
8+
Skill rule referenced: `SKILL.md` → "Send-back for revision (ADR-0044)", which
9+
carries the canonical shape, the three authoring pieces and the lint findings.
10+
This file states only what a grader scores, so the rule has one home.
911

1012
## Scenario
1113

@@ -15,55 +17,6 @@ Skill rule referenced: `SKILL.md` → "Send-back for revision (ADR-0044)".
1517
> revision**; after the submitter reworks and resubmits, it returns to the
1618
> manager for another round. Cap it at two send-backs.
1719
18-
## Expected Output
19-
20-
An approval node with **three** labelled out-edges, an `approval_revise` node
21-
for the revision window, and a **declared back-edge** closing the loop:
22-
23-
```typescript
24-
{
25-
name: 'budget_approval',
26-
label: 'Budget Approval', // `label` is REQUIRED on the flow and every node
27-
type: 'autolaunched',
28-
nodes: [
29-
{ id: 'start', type: 'start', label: 'On Budget Increase',
30-
config: { objectName: 'project', triggerType: 'record-after-update',
31-
condition: 'budget > 100000 && budget != previous.budget' } },
32-
{ id: 'manager_review', type: 'approval', label: 'Manager Review',
33-
config: { approvers: [{ type: 'position', value: 'manager' }], lockRecord: true,
34-
maxRevisions: 2 } }, // send-back budget
35-
// `approval_revise`, not `wait`: the window is service-owned, ended only by
36-
// the submitter's resubmit — so it takes no config.
37-
{ id: 'wait_revision', type: 'approval_revise', label: 'Awaiting Revision' },
38-
{ id: 'approved', type: 'end', label: 'Approved' },
39-
{ id: 'rejected', type: 'end', label: 'Rejected' },
40-
],
41-
edges: [
42-
{ id: 'e1', source: 'start', target: 'manager_review' },
43-
{ id: 'e2', source: 'manager_review', target: 'approved', label: 'approve' },
44-
{ id: 'e3', source: 'manager_review', target: 'rejected', label: 'reject' },
45-
{ id: 'e4', source: 'manager_review', target: 'wait_revision', label: 'revise' }, // send-back
46-
{ id: 'e5', source: 'wait_revision', target: 'manager_review', label: 'resubmit',
47-
type: 'back' }, // declared back-edge
48-
],
49-
}
50-
```
51-
52-
Mirrors the canonical `showcase_budget_approval` flow in the showcase app in
53-
the framework repo.
54-
55-
## Common Mistakes
56-
57-
| Mistake | Why it is wrong | Caught by |
58-
|---|---|---|
59-
| Missing `label` on the flow or on a node | `label` is required by `FlowSchema` — `FlowSchema.parse` / `registerFlow` rejects the definition before any graph validation runs | `registerFlow` (schema parse) |
60-
| Resubmit edge **without** `type: 'back'` | `registerFlow` validates the graph-minus-back-edges as a DAG, so it rejects the cycle as un-declared | `registerFlow`; lint `flow-approval-revise-unmarked-backedge` |
61-
| `revise` edge into a plain **`wait`** node (or any other type) | The window is a service-owned pause: a `wait` is `resumeAuthority: 'any'`, so a raw run-resume walks the back-edge with no submitter check and no audit row, and can destroy the run. `sendBack` refuses this metadata (amended ADR-0044) | lint `flow-approval-revise-target-not-service-owned` (**error**) |
62-
| `revise` edge to a window that **never loops back** | A valid DAG (registerFlow accepts it), but the submitter has nowhere to resubmit — the branch dead-ends | lint `flow-approval-revise-dead-end` |
63-
| `maxRevisions: 0` together with a `revise` edge | Send-back is disabled, so every revise auto-rejects and the branch never runs | lint `flow-approval-revise-disabled` |
64-
| Re-suspending the approval node in a "revise mode" (no window node, no edge) | Hides a state machine inside one node — invisible to the canvas/run log; not the ADR-0044 model. (The 2026-07-28 amendment made the window a dedicated node TYPE, which keeps it visible; it did not move the pause inside the approval node.) | design review |
65-
| Reusing `reject` for send-back | `reject` terminates; send-back is a *movement* that returns the record for rework (status `returned`, not `rejected`) | semantics |
66-
6720
## Validation Criteria
6821

6922
Score the generated flow:
@@ -74,8 +27,11 @@ Score the generated flow:
7427
4. **Revise window** — the `revise` edge targets an `approval_revise` node. *(required)*
7528
5. **Guard** — `maxRevisions >= 1` on the approval config (the default `3` is fine; `0` fails). *(required)*
7629
6. **No lint findings** — `lint-flow-patterns` emits none of the four `flow-approval-revise-*` findings. *(required)*
77-
7. **Approve / reject intact** — the approval still has `approve` and `reject` out-edges. *(preferred)*
30+
7. **Window is a node, not a mode** — the pause is the `approval_revise` node on
31+
the graph, not a "revise mode" re-suspend of the approval node itself (that
32+
hides a state machine inside one node, invisible to the canvas and run log). *(required)*
33+
8. **Approve / reject intact** — the approval still has `approve` and `reject` out-edges. *(preferred)*
7834

79-
Pass = criteria 1–6 all hold. The canonical failure this eval guards against is a
35+
Pass = criteria 1–7 all hold. The canonical failure this eval guards against is a
8036
run that builds the loop but omits the back-edge (criterion 3) — accepted by a
8137
naive author, rejected by `registerFlow`.

0 commit comments

Comments
 (0)