Skip to content

feat(user-interface): honor an explicit accelerating-exit request and fix the write-css rule graders - #9899

Merged
kyle-sexton merged 5 commits into
mainfrom
test/user-interface-css-rule-graders-7129
Oct 25, 2026
Merged

kyle-sexton merged 5 commits into
mainfrom
test/user-interface-css-rule-graders-7129

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Closes #7129

Summary

Kept-trace reruns of the two write-css rule cases that scored 0/5 in every arm found two grader defects. In css-rule-logical, no-physical failed 5/5 runs whose CSS was fully logical: every reply ended with a right-to-left note naming the physical alternative in inline code (`text-align: left`), and the pattern matched it. The issue's guess (top: 0 on a positioned element) was not the cause. In css-rule-ease-in, all 5 runs routed to user-interface:write-animation, so the skill-fired grader, which named only write-css, failed every run.

Fix

  • no-physical no longer counts a physical property inside inline code, whether a bare declaration or a quoted whole rule; each alternative skips a match with a closing backtick later on the line. Physical declarations in CSS still fail it, including in a fenced block. Must-pass samples cover both prose forms.
  • The ease-in skill-fired grader accepts write-css or write-animation. Both carry the no-ease-in rule, and an animation request routes to write-animation by design (its Step 7 hands CSS to write-css).
  • Implements owner decision Q105 (2026-10-24, recorded on test(user-interface): css-rule-logical and css-rule-ease-in fail 0/5 in every arm #7129: test(user-interface): css-rule-logical and css-rule-ease-in fail 0/5 in every arm #7129 (comment)): an explicit request for an exit that accelerates away wins over the ease-in rule. write-css (SKILL.md, reference/motion.md) and the animation catalog say to use ease-in for that exit and warn that its slow start can read as lag. write-css also says the review script and audit-animation still report that ease-in, and that the finding is expected. The default is unchanged.
  • The css-rule-ease-in eval follows: no-ease-in and its samples are removed, and fast-start-curve becomes exit-request-honored, which passes an ease-out curve, or an honored request with the lag warning. The judge has four labeled samples.

Verification

  • Claude Code 2.1.310, claude plugin eval <copy>/plugins/user-interface --case <case> --ablation none --keep-temp --judge-model opus --threshold 0.8, 5 runs, run-validity verdict VALID:
    • css-rule-logical: before, no-physical failed 5/5 (score 0.67); after, 5/5 (score 1.00).
    • css-rule-ease-in on the final skill text: 5/5 (score 1.00), every grader passing. A run on the earlier wording ("say it starts slow") scored 4/5. The miss called the slow start a feature and never named the lag, so the skill text now asks for the lag warning.
  • exit-request-honored judge calibration (calibrate-judge.py, opus, 3 runs per sample): 12/12 agreement over 4 samples. Three independent labelers agreed with every sample label.
  • Offline: validate-cases.py plugins/user-interface/evals exits 0, checking each deterministic grader against its samples; the quoted-rule sample failed the old pattern first. validate-plugins.sh --only user-interface, check-changelog-fragments.sh --check and check-affected.sh --tests pass.

Related

🤖 Generated with Claude Code

https://claude.ai/code/session_01X1zWtmVHsCdqJk7AEeNE2b

kyle-sexton and others added 4 commits October 24, 2026 13:16
… and count write-animation firing

no-physical matched a right-to-left note in every reply (`text-align: left`
named in inline code) while the CSS itself was logical. A match opened or
closed by an inline-code backtick no longer counts; physical declarations in
CSS still fail. The ease-in case's skill-fired grader accepts write-animation,
which every run routed to.

Refs: #7129

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r the ease-in rule

An explicit request for an exit that accelerates away now wins over the
never-ease-in rule in write-css, with the slow start named. The default is
unchanged. The css-rule-ease-in eval passes an honored request with the
trade-off named.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 25, 2026 07:03
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-25T07:05:13.402182Z 740dd8a Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 3173c75d82864fd1a94ff0da252ecf814cbf2632: 0 finding(s), 0 as inline comments.

No security issues in the incremental change (8 files: eval prompt, grader, and samples; write-css and animation reference prose; a changelog fragment).

  • No manifests, workflows, hooks, scripts, trusted paths, or userConfig keys are touched, so no manifest <field> lines apply.
  • Instruction-surface lens: the ease-in wording changes from "say it starts slow" to "warn that its slow start can read as lag", and adds a note that the review script and audit-animation still report that finding. This deletes or softens no control. The ease-in rule is a UI-quality convention, not a protected class. The explicit-request exception was already in the earlier reviewed head. I did not read the instruction exception register, so I judged this by consequence.
  • Grader regex: the text-align alternative of no-physical now ignores a match followed by a backtick on the same line, as the other alternatives already did. Real physical declarations in fenced CSS still fail. This is an eval-quality matter with no exploit path.
  • PR context contains no embedded instructions aimed at the review.
  • Spend: eval reruns draw only on prepaid API credit, so there is no hold.

critical-findings: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 2 inline finding(s) on 740dd8aa6957d45dc7481934130c2c91c9773b70; the summary is in this lane's pull request comment.

claude-lane: claude-review

Comment thread plugins/user-interface/evals/css-rule-logical/graders/no-physical.md Outdated
Comment thread plugins/user-interface/skills/write-css/SKILL.md
@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 3173c75d82864fd1a94ff0da252ecf814cbf2632: 0 finding(s), 0 as inline comments.

Incremental review of the 8 files changed since 740dd8a: no findings.

  • Wording change. The reply must now "warn that its slow start can read as lag" instead of "name that it starts slow". The change fragment, prompt.md, catalog.md, write-css/SKILL.md and reference/motion.md all use the new wording.
  • Ease-in judge. graders/exit-request-honored.md was not touched. It still accepts "starts slow or reads as lag", which is slightly looser than the new expected_outcome but does not contradict it.
  • Lint note. The new SKILL.md text says the review script's ease-in finding on an accelerating exit is expected and to keep the curve. This matches the audit-animation note.
  • no-physical regex. The text-align alternative now uses (?![^\n]*) instead of (?!)`, which matches the other two alternatives.
    • I traced the new sample `.rtl { text-align: left; }`. The lookahead sees the closing backtick on the same line, so it does not match, and the pass sample holds.
    • The existing fail samples (.bubble { text-align: left; }, the margin and offset ones) contain no backtick, so they still match and still fail.
    • A physical declaration in a fenced block on its own line still fails. A one-line reply that puts CSS and a later inline-code span on the same line would slip through. That is an unlikely shape and not worth blocking.
  • Not run. I read the files and traced the regex by hand. I did not run validate-cases.py or any live eval. The PR's verification claims are the author's and unverified here. The PR itself says the opus judge for exit-request-honored is uncalibrated and was not re-run live.
  • Manifests. The diff changes no plugin.json or marketplace.json, so there are no manifest lines. The fragment's bump: patch fits an eval and docs-wording change.
  • Spend. The change is eval and doc content only. It adds no billing or paid-plan change, so there is no hold.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 740dd8aa69

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/user-interface/skills/write-css/SKILL.md Outdated
Comment thread plugins/user-interface/skills/write-css/SKILL.md Outdated
Comment thread plugins/user-interface/skills/write-css/SKILL.md
…in exception with the linter

- no-physical: the text-align alternative skips inline code anywhere later
  on the line, like the other two alternatives, so a quoted whole rule such
  as `.rtl { text-align: left; }` no longer fails a logical reply.
- write-css: the review script and audit-animation still report an honored
  ease-in; the rule and the lint step now say that finding is expected.
- The ease-in exception warns that the slow start can read as lag, matching
  what the exit-request-honored rubric asks for.
- exit-request-honored gets four labeled samples (three independent labelers
  agreed on all four); opus calibration agreed on 12/12 runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1zWtmVHsCdqJk7AEeNE2b
@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 25, 2026
Merged via the queue into main with commit 3b9ea19 Oct 25, 2026
53 checks passed
@kyle-sexton
kyle-sexton deleted the test/user-interface-css-rule-graders-7129 branch October 25, 2026 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(user-interface): css-rule-logical and css-rule-ease-in fail 0/5 in every arm

1 participant