Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .changes/app-control/feat-9584-lesson-routing-6a8a546e.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
bump: minor
---

### Added

- `/app-control:map` create and refresh place a lesson learned while driving by its scope: a lesson about one feature goes in that feature's `## Gotchas`, one that holds for every feature goes in a new optional `## Gotchas` section of the control skill's hub, and any other lesson is written nowhere in the control skill and reported for `/corrections:codify`. In refresh a placed lesson is doc drift and ships in the upkeep pull request.
- `/app-control:map refresh <feature>` narrows the source wave and live pass to one feature of the control skill that holds it; a clean narrowed pass records no state.
7 changes: 7 additions & 0 deletions .changes/verification/feat-9584-lesson-routing-c1ade89d.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
bump: minor
---

### Added

- `/verification:confirm` lists the lessons a run learned in its report, each with its scope and route, and its `## Next` names `/app-control:map refresh <feature>` for a lesson about a feature or drive step and `/corrections:codify` for any other. It never edits a control skill itself.
19 changes: 19 additions & 0 deletions plugins/app-control/evals/map-lesson-every-feature/case.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
schema_version: "1.1"
name: map-lesson-every-feature
description: Refresh places a lesson that holds for every feature in the hub's Gotchas once, not in each feature file
tags: [map, refresh, lessons, invented]
expected_outcome: "The reply puts the cookie-banner lesson once in a `## Gotchas` section of the hub, .claude/skills/control-clockwork/SKILL.md, rather than copying it into each of the four feature files, treats it as doc drift so the outcome is changed, ships it on control-skill-upkeep/control-clockwork in one pull request, does not hand it to /corrections:codify, and writes nothing."
execution:
prompt: "/app-control:map refresh .claude/skills/control-clockwork. Do not read the repository, drive the app or write any file. Treat the source wave and the live pass as already run. A /verification:confirm report handed in one lesson: on the first page load of a new browser session the app shows a cookie banner across the bottom of the page, and a click on anything under it lands on the banner, so a drive has to dismiss it first. The live pass saw it at the start of all four features (weekly-timesheet-export, start-a-timer, invoice-email, team-report) and each drive worked once the banner was dismissed. Ready passed and nothing else differs. Name the file and section the lesson goes to, the outcome, and how it ships, then stop."
allowed_tools: [Read, Glob, Grep]
max_turns: 6
timeout_seconds: 300
graders:
- name: names-hub-gotchas
type: regex
pattern: 'SKILL\.md[\s\S]{0,120}Gotchas|Gotchas[\s\S]{0,120}SKILL\.md|\bhub''?s?\s+`?(## )?Gotchas'
flags: i
arm: both
- name: hub-gotchas-once
type: llm
criteria: "PASS only if the reply (1) puts the lesson in a `## Gotchas` section of the hub, .claude/skills/control-clockwork/SKILL.md, (2) does not copy it into the Gotchas of each feature file, (3) says it ships as a commit on control-skill-upkeep/control-clockwork through one pull request, with the outcome changed, and (4) does not route it to /corrections:codify or any instruction file outside the control skill. FAIL if it writes the lesson into each of the four feature files, writes it outside .claude/skills/control-clockwork/, calls the outcome clean, or hands it off instead of placing it."
Comment thread
kyle-sexton marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"pass": [
{
"answer": "It holds for every feature, so it goes once in the `## Gotchas` section of the hub, .claude/skills/control-clockwork/SKILL.md, and in no feature file. It is doc drift: the outcome is changed and it ships as a commit on control-skill-upkeep/control-clockwork through one pull request.",
"why": "hub Gotchas once, changed outcome, upkeep branch and one pull request"
}
],
"fail": [
{
"answer": "Add the banner note to the hub's `## Gotchas` in SKILL.md and to the `## Gotchas` of each of the four feature READMEs, so every drive sees it. Outcome changed; it ships on control-skill-upkeep/control-clockwork in one pull request.",
"why": "copies it into each feature file as well"
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"pass": [
{
"answer": "It holds for every feature, so it goes once in the hub's ## Gotchas in .claude/skills/control-clockwork/SKILL.md. Doc drift; the outcome is changed and it ships on control-skill-upkeep/control-clockwork.",
"why": "names the hub's Gotchas"
}
],
"fail": [
{
"answer": "I would add the dismiss-the-banner step to the Drive of each of the four feature READMEs.",
"why": "copies it into every feature file"
},
{
"answer": "Add the dismiss-the-banner note to each of the four feature READMEs' Gotchas and open a PR on GitHub.",
"why": "copies it into every feature; GitHub must not read as the hub"
}
]
}
18 changes: 18 additions & 0 deletions plugins/app-control/evals/map-lesson-not-a-feature/case.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
schema_version: "1.1"
name: map-lesson-not-a-feature
description: Refresh writes nothing into the control skill for a lesson that is not about driving the app and leaves it for /corrections:codify
tags: [map, refresh, lessons, invented]
expected_outcome: "The reply says the dist/ staging lesson is not about a feature or a drive step, writes it into no file of .claude/skills/control-clockwork/ (neither a feature's Gotchas nor the hub's), reports it for /corrections:codify to place, says the outcome is clean because nothing else differs, names no instruction-placement or write-for-agents skill, and writes nothing."
execution:
prompt: "/app-control:map refresh .claude/skills/control-clockwork. Do not read the repository, drive the app or write any file. Treat the source wave and the live pass as already run; every feature drove as written and Ready passed. A /verification:confirm report handed in one lesson: agents in this repository keep staging the generated dist/ folder in their commits, and it should never be committed. Say where the lesson goes, what you would write and where, and the outcome, then stop."
allowed_tools: [Read, Glob, Grep]
max_turns: 6
timeout_seconds: 300
graders:
- name: names-codify
type: regex
pattern: '/corrections:codify'
arm: both
- name: hands-off-writes-nothing
type: llm
criteria: "PASS only if the reply (1) says the lesson is not about a feature or a drive step of the app, (2) writes or plans to write it into no file under .claude/skills/control-clockwork/, including the hub's and every feature's Gotchas, (3) leaves it for /corrections:codify to place, and (4) calls the outcome clean, with no upkeep commit or pull request for this lesson. FAIL if it puts the lesson in any Gotchas section or other control skill file, writes it into an instruction file itself, names any destination other than /corrections:codify, or opens a pull request for it."
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"pass": [
{
"answer": "This lesson is about how agents work in this repository, not about a feature or a drive step, so I write nothing into .claude/skills/control-clockwork/. I report it for /corrections:codify, which places it. Every feature drove as written, so the outcome is clean: no upkeep commit and no pull request.",
"why": "not a feature lesson, nothing written, left for codify, clean"
}
],
"fail": [
{
"answer": "Not a feature lesson, so I report it for /corrections:codify, and I also add 'never stage dist/' to the hub's `## Gotchas` so drives see it. That edit ships in the upkeep pull request on control-skill-upkeep/control-clockwork.",
"why": "names codify but still writes the lesson into the control skill and opens a pull request"
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"pass": [
{
"answer": "Not a feature or drive-step lesson, so nothing goes into the control skill. I report it for /corrections:codify, which places it. Nothing else differs: the outcome is clean.",
"why": "hands the lesson to codify"
}
],
"fail": [
{
"answer": "I would add 'never stage dist/' to the hub's Gotchas so drives see it, and ship it in the upkeep pull request.",
"why": "writes a non-feature lesson into the control skill"
}
]
}
18 changes: 18 additions & 0 deletions plugins/app-control/evals/map-lesson-one-feature/case.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
schema_version: "1.1"
name: map-lesson-one-feature
description: Refresh places a lesson that holds for one feature in that feature's Gotchas and ships it in the upkeep pull request
tags: [map, refresh, lessons, invented]
expected_outcome: "The reply puts the wait-for-Export lesson in the `## Gotchas` section of .claude/skills/control-clockwork/features/weekly-timesheet-export/README.md, not in the hub or another feature, treats it as doc drift so the outcome is changed, ships it on control-skill-upkeep/control-clockwork in one pull request, does not hand it to /corrections:codify, and writes nothing."
execution:
prompt: "/app-control:map refresh weekly-timesheet-export. Do not read the repository, drive the app or write any file. Treat the source wave and the live pass as already run on .claude/skills/control-clockwork, the only control skill. A /verification:confirm report handed in one lesson: on the weekly timesheet page the Export button stays disabled for about two seconds after the week loads, and a click before it enables does nothing, so the drive has to wait for the button to enable. The live pass confirmed it: the export worked once the step waited. No other feature has that button. Ready passed and nothing else differs. Name the file and section the lesson goes to, the outcome, and how it ships, then stop."
allowed_tools: [Read, Glob, Grep]
max_turns: 6
timeout_seconds: 300
graders:
- name: names-feature-file
type: regex
pattern: 'features/weekly-timesheet-export/README\.md[\s\S]{0,120}Gotchas|Gotchas[\s\S]{0,120}features/weekly-timesheet-export/README\.md'
arm: both
- name: feature-gotchas-in-pull-request
type: llm
criteria: "PASS only if the reply (1) puts the lesson in the `## Gotchas` section of features/weekly-timesheet-export/README.md, (2) does not also put it in the hub (SKILL.md) or any other feature file, (3) says it ships as a commit on control-skill-upkeep/control-clockwork through one pull request, with the outcome changed, and (4) does not route it to /corrections:codify or any instruction file. FAIL if it places the lesson in the hub's Gotchas, writes it into a file outside .claude/skills/control-clockwork/, calls the outcome clean, or hands it off instead of placing it."
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"pass": [
{
"answer": "The lesson goes in the `## Gotchas` section of .claude/skills/control-clockwork/features/weekly-timesheet-export/README.md: wait for the Export button to enable before clicking it. Nothing goes in the hub or another feature. It is doc drift, so the outcome is changed, and it ships as a commit on control-skill-upkeep/control-clockwork through one pull request.",
"why": "one feature's Gotchas, changed outcome, upkeep branch and one pull request"
}
],
"fail": [
{
"answer": "The lesson goes in the `## Gotchas` of .claude/skills/control-clockwork/features/weekly-timesheet-export/README.md and also in the hub's `## Gotchas` in SKILL.md so every drive sees it. Outcome changed; it ships on control-skill-upkeep/control-clockwork in one pull request.",
"why": "also copies a one-feature lesson into the hub"
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"pass": [
{
"answer": "The lesson goes in the Gotchas of .claude/skills/control-clockwork/features/weekly-timesheet-export/README.md: wait for Export to enable before clicking. It is doc drift, so the outcome is changed and it ships on control-skill-upkeep/control-clockwork.",
"why": "names the one feature's file"
}
],
"fail": [
{
"answer": "The lesson goes in the hub's Gotchas in .claude/skills/control-clockwork/SKILL.md so every drive sees it.",
"why": "places a one-feature lesson in the hub"
},
{
"answer": "Open features/weekly-timesheet-export/README.md and add a wait-for-Export step to its Drive.",
"why": "names the right file but no Gotchas section"
}
]
}
3 changes: 2 additions & 1 deletion plugins/app-control/reference/control-skill.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,15 @@ Skill tool; it reports one that carries more as a finding and runs none of it.
## Hub

The skill's `SKILL.md` (`.claude/skills/control-<app>/` in a project, `skills/control-<app>/` in a
plugin) holds four sections and an index:
plugin) holds four sections, an optional `## Gotchas`, and an index:

| Section | Holds |
|---|---|
| `## Launch` | A pointer to the launch recipe or the one command that starts the app |
| `## Ready` | One read-only command that shows a running instance is fit to drive: it answers, it is the expected build when the app exposes one, it uses the expected data. Also names the telemetry the app exposes |
| `## Capture` | For each kind (`trace`, `cpu`, `heap`, `threads`, `screenshot`, `aria`, `video`, `logs`) the tool that records it, or nothing when the project has none |
| `## Cleanup` | How to stop what a run started and nothing else |
| `## Gotchas` (optional) | What has made a run lie or fail in every feature; a one-feature lesson goes in that feature's own `## Gotchas` |
| App (above the sections) | The app's name and the path it is built from. `/app-control:map` writes it for the person and for later drive work; drive does not read it |
| Driving tool (above the sections) | The tool that drives the app, such as a browser driver or the project's CLI. Written and read as the App row is |
| `Attended runs only.` (above the sections) | Present when that tool drives the person's own signed-in browser. A marker for the person and for later drive work, as the App row is |
Expand Down
Loading
Loading