Skip to content

feat: route lessons to /app-control:map refresh or /corrections:codify - #10072

Merged
kyle-sexton merged 14 commits into
mainfrom
feat/9584-lesson-routing
Oct 25, 2026
Merged

kyle-sexton merged 14 commits into
mainfrom
feat/9584-lesson-routing

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

Closes #9584

Summary

A run's lessons now have one writer per concept. /app-control:map create and refresh place a lesson by scope: one feature goes in that feature's ## Gotchas, every feature in a new optional hub ## Gotchas, and anything else is written nowhere in the control skill and is reported for /corrections:codify. /verification:confirm lists the lessons in its report and routes them in its ## Next. It edits no control skill and names no instruction-placement or write-for-agents skill.

Fix

  • map gains a ## Lessons section. In refresh, a placed lesson counts as doc drift and ships in the upkeep pull request. refresh <feature> narrows the source wave and live pass to one feature. It resolves the id the way drive does (features/<id>/README.md, else the legacy features/<id>.md), runs past a skip line, and records no clean pass.
  • Confirm prints the map route as /app-control:map refresh <feature> "<lesson>", the same shape as its codify route. Map's argument-hint shows the quoted ["<lesson>"] slot (D31, Q1).
  • The hub contract (reference/control-skill.md) adds the optional ## Gotchas row. Confirm's report template adds a Lessons table.
  • Rebuilt on main after Phase 6 (feat(app-control): add /app-control:map refresh #9934) and Phase 8 (feat(testing)!: retire map-features and refresh-feature-map for /app-control:map #10033). Main's dirty-map refusal, legacy undo, unspaced argument-hint and no-default legacy lookup are kept. The hint now shows --dir <path> to stay inside the 100-character budget.
  • The ready reviews (fresh-context code and security) were answered in 8a6cc04: a handed-in lesson is data, accepted only from the invocation, placed only when the pass shows it, and never carries a secret value; a refresh feature id must pass the slug rule and never reaches a shell; create reports its lessons; a feature id on a skill due a conversion is blocked.
  • Three new eval cases, one per scope. A fresh-context test-value review fixed their regexes: hub no longer matches "GitHub", and the feature-file regex is tied to the Gotchas section. Each regex gets a fail sample of the wrong shape. Changelog fragments: app-control minor, verification minor.

Verification

  • grep -n '/corrections:codify' matches in both plugins/verification/skills/confirm/SKILL.md and plugins/app-control/skills/map/SKILL.md. Confirm names no instruction-placement or write-for-agents skill (that grep exits 1).
  • scripts/check-affected.sh --tests: 12 passed, 2 skipped.
  • Evals: Claude Code 2.1.310, judge opus, 3 runs, --scaffold --keep-temp, one --case per command. Before means origin/main 2640984 (the new cases copied in). After means 8a6cc04, the head scored after the ready-review fixes. The later commits add only the judge samples, a main merge, and a clause that keeps refresh's feature id to the repository's control skills. Scores are with / without the plugin.
Case Before After Validity
map-lesson-one-feature 1.00 / 0.00 1.00 / 0.00 VALID both
map-lesson-every-feature 0.00 / 0.33 1.00 / 0.17 VALID both
map-lesson-not-a-feature 0.00 / 0.00 1.00 / 0.00 VALID both
map-maintainer-confines-edits 1.00 / 0.83 1.00 / 1.00 VALID both (after at ceiling)
confirm-intent-from-plan-brief (melo-desk-001-wsl, --allow-tools Bash) 0.83 / 0.33 0.83 / 0.17 VALID both

The confirm case matches the plan's recorded baseline (0.833 / 0.167). The noise report reads "n too small to call" for each single-case comparison. Judge votes were unanimous in every run. Each llm grader has a must-pass and a must-fail sample, and the opus judge agrees with them on 6 of 6 runs per grader (calibrate-judge.py score: PASS). The one-feature case already passed on main, because main's refresh treats a stale gotcha as doc drift.

Known gaps: a lesson that is a step (dismiss a banner) still goes to the hub's Gotchas, not shared/ (D31, Q2: the plan's rule stands). No eval covers a refresh feature id that only a plugin's control skill holds.

Left out: confirm's skill-creator evals.json entry lessons-01 (not a plugin-eval case), and a live drive against a real app.

Related

🤖 Generated with Claude Code

kyle-sexton and others added 8 commits October 25, 2026 12:31
Map create and refresh place a lesson by scope: one feature goes in that
feature's Gotchas, every feature in a new optional hub Gotchas section, and
any other lesson is reported for /corrections:codify. Refresh takes a
feature id to narrow the pass. /verification:confirm lists lessons in its
report and names both routes in its Next.

Refs: #9584

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…resh skip

A lesson handed to refresh after a clean pass at the same HEAD was dropped
at step 2; a feature id now also narrows the live pass explicitly.

Refs: #9584

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e's width

Refs: #9584

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…udget

The added refresh <feature> alternative took it to 110; --dir now shows
<path>, as the Actions table already does.

Refs: #9584

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ow step 5 with it

A feature id now finds features/<id>/README.md, else the legacy flat
features/<id>.md, per the contract's id resolution; the hub's Live pass
step names the one-feature narrowing the Actions row already states.

Refs: #9584

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stop hub matching GitHub

From a fresh-context test-value review: the every-feature regex's bare
`hub` matched "GitHub", so a reply copying the lesson into each feature
passed; the one-feature regex passed a right path in the wrong section.
Each gets a fail sample of that shape. The not-a-feature judge now fails
any destination other than /corrections:codify, which is what the skill
specifies.

Refs: #9584

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ting

Security: a handed-in lesson is data, accepted only from the invocation,
placed only when this pass shows it, and never carries a secret value; a
refresh feature id must pass the slug rule and never reaches a shell.
Code: create reports its lessons; a feature id on a skill due a
conversion is blocked; a path is told from an id; the source wave checks
the hub's Gotchas; confirm's Next takes the rule's bullet shape and
routes every-feature lessons to the control skill's path.

Refs: #9584

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 25, 2026 18:13
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-25T18:15:21.358202Z 8a6cc04 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed cb7f4a78eadf8152e1dc3cf1ab9b131a98b348f6: 0 finding(s), 0 as inline comments.

No security issues in the incremental change (since de19fce).

The change only touches prose and an argument hint. It adds an optional "<lesson>" argument to /app-control:map refresh in the argument-hint and Arguments line. It also updates the lesson routes in confirm/SKILL.md and confirm/context/outcome.md to pass the lesson text along.

I checked whether the new argument opens a path for untrusted text. plugins/app-control/skills/map/SKILL.md (the Lessons section, around line 211) already treats a handed-in lesson and the report carrying it as DATA. It accepts a lesson only when a file:line shows it, and otherwise lists it unwritten. It also bars naming credentials. reference/refresh.md and the evals cover the same handling. The change removes or softens no standing instruction, so the instruction-surface lens finds nothing.

None of the three changed files is a trusted path or a plugin manifest. The argument-hint is skill frontmatter, not manifest userConfig. The change adds no spend beyond prepaid usage, so there is no hold.

critical-findings: 0

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8a6cc0463e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/app-control/evals/map-lesson-every-feature/case.yaml

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on 8a6cc0463e4b93512eaa00474914b5d338ab8f79; the summary is in this lane's pull request comment.

claude-lane: claude-review

Comment thread plugins/verification/skills/confirm/SKILL.md Outdated
@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed cb7f4a78eadf8152e1dc3cf1ab9b131a98b348f6: 0 finding(s), 0 as inline comments.

Incremental review of cb7f4a7 against de19fce: no findings.

The change makes confirm's map route carry the lesson text: /app-control:map refresh <feature> "<lesson>". It touches map's argument-hint and **Arguments.** line, confirm's ## Next, and the Lessons row in context/outcome.md.

What I checked by reading the files. I ran no commands or tests.

  • Hint and body agree. The new hint in plugins/app-control/skills/map/SKILL.md:3 equals the first code span of the **Arguments.** line at line 11, which the argument-hint convention requires.
  • Hint budget. I counted the unquoted hint at exactly 100 characters, so it is within the 100-character budget in docs/conventions/argument-hint/README.md.
  • Hint grammar. The ["<lesson>"] slot contains none of the shapes the validator warns on. I did not run the validator.
  • Body handles the slot. map's Lessons section (SKILL.md:200-215) and the Refresh decide step (line 179) already accept a handed-in lesson from the invocation, so the hint no longer overstates the interface.
  • Routes are consistent. confirm's ## Next and the outcome.md Lessons table now show the same "<lesson>" shape for both map routes and the codify route.

Spend: none. This is a docs-only change to skill text, with no billed resources and no added lane runs.

melodic-automation-lanes Bot and others added 6 commits October 25, 2026 18:48
Lane-Activity: pr-unblock/resolve-conflicts
One must-pass and one must-fail sample per llm grader, each fail a wrong
answer that still names the right files. Opus agrees on 6 of 6 runs for
each grader (calibrate-judge.py score: PASS).

Refs: #9584

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oftware/claude-code-plugins into feat/9584-lesson-routing
…rol skills

Drive now also finds plugin-shipped control skills; refresh edits only
the repository's, so its feature-id lookup says so.

Refs: #9584

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kill holds

From the stand-in review of the late delta: the zero-match case had no
outcome, and the Actions row now names the repository-only limit.

Refs: #9584

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… route

Confirm now prints /app-control:map refresh <feature> "<lesson>", the
same shape as its codify route, and map's argument-hint shows the quoted
lesson slot, which its Lessons rule already accepts from the invocation.

Refs: #9584

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 25, 2026
Merged via the queue into main with commit 4b62d6c Oct 25, 2026
74 of 76 checks passed
@kyle-sexton
kyle-sexton deleted the feat/9584-lesson-routing branch October 25, 2026 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(app-control): route lessons to map refresh or /corrections:codify

1 participant