Feature/skill creation docs - #24
Merged
Merged
Conversation
Tags used by the code-flow and data-model prompts were invisible to the verifier. --emit-summary prints a ready-to-paste Verification Summary block so docs never hand-count tags.
Single source of truth for the tag vocabulary and the mandatory verifier step; prompts reference it instead of carrying diverging copies.
- Remove flask rows from the mandatory Step 0c table (examples never shipped); add model-systems and nextjs rows - Collapse the two duplicate example-selection tables into Step 0c pointers - Add the verify.py step to 02/03/05 checklists; complete truncated tag tables - 01a: absences are NOT_FOUND, never [VERIFIED: no X found] or slash forms; add mandatory model-systems example read - 04: NEEDS_VERIFICATION rendering, inferred-classification honesty, point the sample source doc at pf-docs instead of a bundled example - 05: mandatory test-surface example read, Source Commit metadata row
…overlay 02a [VERIFIED: LSP operation] tags produced zero resolvable citations, so an LSP doc could never pass the verifier it mandates. Citations are now [VERIFIED: path:line — via <op>]. Restore the mandatory Step 0c example read, fix ../examples paths, add the verifier step to lsp/02, and rewrite lsp/02a as a thin overlay of the base prompt (with an explicit note that the outgoingCalls prohibition scopes to architecture docs only).
…oard mini app Examples must cite committed sources so citations resolve for every reader (the requests example previously cited a copy that was never committed). Slim pinned vendors: openai/whisper c0d2f62 (py sources only, assets pruned — see VENDORED.md) and requests 2.32.5 (7029833). linkboard is a purpose-built Next.js App Router mini app with deliberate warts, in the style of the existing tasktracker/slotbooker apps.
First example pair for the 01a model-centric overlay: the good doc shows weights-are-downloads and training-code-absence handled honestly; the bad doc demonstrates fame-based hallucination (documenting a famous model from prior knowledge instead of the repo). The nextjs pair covers App Router surfaces; its bad doc demonstrates App/Pages Router conflation. Both good docs pass verify.py at 100% citation resolution.
The code-flow prompt had no examples to enforce its mandatory read against. Good doc traces the real slotbooker booking flow (36/36 citations, 12/12 quotes); bad doc is an uncited specimen with annotated hallucinations (BookingService, queued mail, SMS, transactions).
…format Citations were systematically off by one line, two paths lacked the app/ prefix, one meta-tag was unparseable, and sections 4-5 used the exact ASCII arrow-flow pattern the prompt bans. Now 133/133 citations resolve and all 8 quotes match at ratio 1.00.
Every line citation had drifted 2-4 lines from the tasktracker app and one quote no longer existed in the file; the doc failed its own verifier. Now 104/104 citations and 13/13 quotes pass. The bad example gets the banner, per-section PROBLEMS callouts, and the false-claim-to-reality closing list.
Livewire resolved 20/26 citations and 1/18 quotes (bare blade filenames in the 3b map); react's quotes were shifted. Both good docs rebuilt to the prompt-01 skeleton — livewire 104/104 citations 17/17 quotes, react 90/90 and 15/15. Both bad examples annotated with grounded PROBLEMS callouts.
…n bad one The good doc had no metadata, invented tags, and prose evidence the verifier scored 0/0 — rebuilt to the contract (102/102 citations, 6/6 quotes). The bad doc's citations all resolve but its quotes are silently truncated to hide warts; annotations now expose each cut verbatim, with the headline lesson that resolving citations are not true claims.
…tbooker flow It cited BookingService and a confirmation email — the exact hallucinations the laravel bad example warns about. Now derived from the real booking flow (35/35 citations), including an absence test pinning that no email exists.
…cise phase 2 Line references had drifted; the doc also demonstrated only phase 1. Adds a quoted block so the canonical self-verifying example exercises the quote check, documents the new tags and --emit-summary, and syncs summary counts.
Citations were bare filenames resolving against nothing, with byte counts as evidence and hedged/disjunctive VERIFIED tags. Rewritten against the vendored 2.32.5 source: 125/125 citations, 6/6 quotes. Bad example gains per-section PROBLEMS callouts grounded in the vendored files.
scripts/verify-examples.sh maps each good doc to its source root and requires PASS; the workflow runs it on push and PRs. This is the guard that keeps teaching examples from drifting away from the code they cite — the failure mode this session found in every framework example.
…gger phrasing Skill descriptions were labels with no invocation phrasing; none mentioned the verifier. Every doc-producing skill now ends with the verify.py gate, LSP skills use the path:line citation form, map-flows reads the new code-flow example pair, and the SKILLS.md workflow gains the verify step.
Flask downloads 404ed silently into an empty dir that satisfied the mandatory example-read with nothing. Ships 00-verification-core.md, the laravel code-flow pair, nextjs and model-systems pairs, and stops swallowing errors on the verifier example fetch.
…erseded React/Vue/Livewire were listed as coming soon despite shipping; flask is gone; structure now shows the CI gate, skills, and new example dirs. The old guides/01 taught arrow-flow tracing and manual verification — now carries a superseded banner pointing at the current prompts.
|
Important Review skippedToo many files! This PR contains 101 files, which is 1 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (101)
You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.