Skip to content

Expand conformance suite to per-rule fixture coverage, gated (herb-embedded-ag7) - #48

Merged
jleo3 merged 1 commit into
mainfrom
herb-embedded-ag7
Aug 18, 2026
Merged

Expand conformance suite to per-rule fixture coverage, gated (herb-embedded-ag7)#48
jleo3 merged 1 commit into
mainfrom
herb-embedded-ag7

Conversation

@jleo3

@jleo3 jleo3 commented Aug 18, 2026

Copy link
Copy Markdown
Member

Summary

  • Adds one hand-written .html.erb fixture per rule the vendored bundle registers (100 rules — verified as the true, complete count against @herb-tools/linter's src/rules/, which has 106 files but 6 are shared utility modules, not rules) under spec/conformance/fixtures/rules/. Each was authored by reading the rule's TypeScript source and validated to trigger the rule via the real reference linter CLI (node_modules/.bin/herb-lint --only <rule>).
  • conformance_spec.rb's existing reference_offenses/bridge_offenses helpers gain optional only:/rules: args (extended, not duplicated) so a per-rule fixture can force-select just that rule the same way gu7/ada's specs already do — this also exercises rules not enabled by default. A new coverage-gate test enumerates bridge.rule_names and asserts every one has a fixture, so a future upstream rule with no fixture fails the build instead of going untested silently.
  • Three rules that only enable themselves for partial files (basename starting with _) get underscore-prefixed fixture filenames to trigger correctly.
  • Found and fixed a real bridge bug while running the new suite: __herbLint/__herbAutofix in js/ruby_backend.js construct the vendored Linter without its 4th constructor arg (allAvailableRules), which silently defaults to just the currently-selected rule subset. herb-disable-comment-unnecessary depends on that arg (via context.validRuleNames) to recognize other rules referenced in a herb:disable comment — under our one-rule-at-a-time execution model it could never recognize any rule but itself. Fixed by passing HerbLinter.rules explicitly at both call sites.
  • CHARTER.md's "conformance punch list isn't closed" open question is updated to reflect the fixture-coverage dimension is now closed, without overclaiming the separate .herb.yml config-application item it also named.

Acceptance criteria

  • One .html.erb fixture per built-in rule under spec/conformance/fixtures/rules/, hand-written, each verified to trigger its target rule via the real reference linter.
  • A new coverage-gate test enumerates every rule the vendored bundle registers and asserts each has a fixture — fails the build if any rule is uncovered.
  • All new fixtures pass the existing offense-for-offense diff against the reference linter, reusing (not duplicating) reference_offenses/bridge_offenses.
  • bundle exec rake passes, including the expanded conformance suite: 206 examples, 0 failures, 0 RuboCop offenses.
  • CHARTER.md updated to match, without adding scope beyond what this work actually changes.

Test plan

  • bundle exec rake passes locally (206 examples, 0 failures).
  • Full conformance suite alone: 107 examples (100 rule fixtures + 5 legacy fixtures + minimum-fixtures check + coverage-gate), 0 failures, ~16-20s.
  • Independently verified exactly 100 fixture files, all unique, 1:1 mapped onto bridge.rule_names with no extras and no gaps.
  • Verified the herb-disable-comment-unnecessary fix by re-running the full suite before (1 failure) and after (0 failures) the js/ruby_backend.js change.

🤖 Generated with Claude Code

…bedded-ag7)

Adds one hand-written .html.erb fixture per rule the vendored bundle
registers (100 rules, verified as the true complete count against
node_modules/@herb-tools/linter's src/rules/), under
spec/conformance/fixtures/rules/. Each fixture is authored by reading the
rule's TypeScript source and validated to trigger the rule via the real
reference linter CLI.

conformance_spec.rb's existing reference_offenses/bridge_offenses helpers
gain optional only:/rules: args (not duplicated) so a per-rule fixture can
run --only <rule> / rules: [rule] the same way gu7/ada's Bridge#lint specs
already do - this also lets not-enabled-by-default rules get exercised
directly. A new test enumerates bridge.rule_names and asserts every one has
a fixture, so a newly-added upstream rule with no fixture fails the build
instead of going untested silently. Three rules that only enable themselves
for partial files (basename starting with _) get underscore-prefixed
fixture filenames to trigger correctly.

Running the full suite surfaced a real, previously-undetected bridge bug:
__herbLint/__herbAutofix construct the vendored Linter without its 4th
constructor arg (allAvailableRules), which defaults to just the
currently-selected rule subset when omitted. herb-disable-comment-unnecessary
reads context.validRuleNames (derived from that arg) to recognize other
rules referenced in a herb:disable comment, so under our one-rule-at-a-time
execution model it could never recognize any rule but itself. Fixed by
passing HerbLinter.rules (the full registry) explicitly at both call sites.

CHARTER.md's "conformance punch list isn't closed" open question is updated
to reflect that the fixture-coverage dimension is now closed, without
overclaiming the separate .herb.yml config-application item it also named.
@jleo3
jleo3 merged commit 8b9b79c into main Aug 18, 2026
1 check passed
@jleo3
jleo3 deleted the herb-embedded-ag7 branch August 18, 2026 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant