Skip to content

feat: add a convention validator and CI workflow - #14

Open
Avicennasis wants to merge 1 commit into
RinDig:mainfrom
Avicennasis:feat/convention-validator
Open

feat: add a convention validator and CI workflow#14
Avicennasis wants to merge 1 commit into
RinDig:mainfrom
Avicennasis:feat/convention-validator

Conversation

@Avicennasis

Copy link
Copy Markdown

Filing this as a proposal rather than a fix. Adding CI to someone else's repo is opinionated, so please treat the workflow file as optional; bin/validate.py stands alone and is useful run by hand.

ICM's thesis is that the filesystem is the orchestration layer. The trade-off is that a code framework throws on a bad import path, whereas here a wrong path degrades silently: the agent guesses, or loads nothing, and the pipeline still appears to run.

There is currently nothing in the repo that mechanically enforces any rule in _core/CONVENTIONS.md, and no .github/. The PR checklist in the README is manual checkboxes.

What it checks

15 rules drawn from CONVENTIONS.md (Quality Guardrails, Naming Conventions, Patterns 1-4) and from defects actually found in the tree. Standard library only, no dependencies.

The four highest-value rules are the ones derived from real defects rather than from the spec text:

Rule Catches
Inputs-table paths resolve broken path references
Every workspace registered in both tables a workspace no routing table mentions
Markdown links have real targets (link-to-paper)
Directories described in the README exist documented-but-absent folders like _config/

Run against main today:

11/15 rules passed
FAIL  Inputs-table paths resolve                               9
FAIL  Every workspace is registered in README and root CLAUDE.md  1
FAIL  Markdown links have real targets                         1
FAIL  Directories described in the README exist                1

Those four failures are exactly the four classes fixed in the companion PRs. With all of them applied this goes green.

Vendored content

The three style rules (line length, em dashes, file naming) skip skills/ by default, since Pattern 9 says those files are copied verbatim from upstream and reformatting them defeats the point. --strict includes them and currently reports 26 further violations, all in the bundled pptx and remotion skills.

Whether the spec should exempt vendored content is your call, so the default takes the conservative reading and the flag makes the other view available.

Every rule was mutation tested

For each of the 11 rules passing on main, a deliberate defect was introduced in a throwaway copy and the rule was confirmed to flip to FAIL:

KILLED   CONTEXT.md under 80 lines
KILLED   Reference files under 200 lines
KILLED   No em dashes
KILLED   Empty persistent folders
KILLED   No spaces in file or folder names
KILLED   Names are lowercase-with-hyphens
KILLED   Stage folders use a zero-padded prefix
KILLED   Inputs, Process, Outputs in order
KILLED   Inputs rows carry a Section/Scope
KILLED   Stage cross-references are one-way
KILLED   Output folders contain only .gitkeep

All 11 killed, so none of the passes are vacuous. The remaining 4 are demonstrated by the live failures above.

Two rules I cut

Both after checking them against the spec rather than assuming:

  • "no placeholders in workspace-root CONTEXT.md" contradicts _core/placeholder-syntax.md:36, which permits placeholders in any markdown file in a workspace. {{?BUILD_STAGE}} in script-to-animation/CONTEXT.md is a documented conditional section, not a violation.
  • "stage CONTEXT.md has no sections beyond Inputs/Process/Outputs/Checkpoints/Audit" would flag ## When to Loop Back in voice-driven-animation/stages/05-render. Pattern 1 mandates that the three sections are present and ordered, not that nothing else may appear. Judgment call, left to you.

ICM's thesis is that the filesystem is the orchestration layer. The
trade-off is that a code framework throws on a bad import path, whereas here
a wrong path degrades silently: the agent guesses, or loads nothing, and the
pipeline still appears to run. There is currently nothing in the repo that
mechanically enforces any rule in _core/CONVENTIONS.md, and no .github/.

bin/validate.py checks 15 rules drawn from CONVENTIONS.md (Quality
Guardrails, Naming Conventions, Patterns 1-4) and from defects actually
found in the tree. Standard library only, no dependencies.

The four rules worth the most are the ones derived from real defects rather
than from the spec text:

  Inputs-table paths resolve                  catches broken path references
  Every workspace registered in both tables    catches a workspace no routing
                                               table mentions
  Markdown links have real targets             catches "(link-to-paper)"
  Directories described in the README exist    catches documented-but-absent
                                               folders such as _config/

Run against main today it reports 11/15, and the four failures are exactly
those four classes. Companion PRs fix each; with all of them applied this
goes green.

Vendored skills/ content is skipped for the three style rules (line length,
em dashes, file naming), since Pattern 9 says those files are copied
verbatim from upstream and reformatting them would defeat the point. Pass
--strict to include them; that currently reports 26 further violations, all
in bundled pptx and remotion skills. Whether the spec should exempt vendored
content is a call for the maintainer, so the default takes the conservative
reading and the flag makes the other view available.

Every rule was mutation tested: for each of the 11 rules passing on main, a
deliberate defect was introduced in a throwaway copy and the rule was
confirmed to flip to FAIL. All 11 were killed, so none of the passes are
vacuous. The remaining 4 are demonstrated by the live failures above.

Two rules from an early draft were cut after checking them against the spec
rather than assuming:

  - "no placeholders in workspace-root CONTEXT.md" contradicts
    _core/placeholder-syntax.md:36, which permits placeholders in any
    markdown file in a workspace. {{?BUILD_STAGE}} in
    script-to-animation/CONTEXT.md is a documented conditional section, not
    a violation.
  - "stage CONTEXT.md has no sections beyond Inputs/Process/Outputs/
    Checkpoints/Audit" would flag "## When to Loop Back" in
    voice-driven-animation/stages/05-render. Pattern 1 mandates that the
    three sections are present and ordered, not that nothing else may
    appear, so this is a judgment call and is left to the maintainer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant