fix(testing): stop enabled:false schema defaults from silently disabling harness tests - #408
Conversation
… disabling harness tests
check_plugin.py, render_plugin.py, and the pytest plugin matrix each built
config as {"enabled": True} then merged in config_schema.json's defaults on
top, letting a plugin's own enabled:false default (a reasonable choice for
a seasonal/opt-in plugin -- 15 of 23 real plugins ship one) silently win.
Every harness/CI render of those plugins was testing "disabled, do
nothing" rather than real behavior.
Extract build_full_config() into testing/loading.py (already the shared
home for plugin-discovery/config-default logic) and use it from all three
call sites: schema defaults, then a forced enabled=True, then harness.json's
config, then the caller's explicit config -- so a test can still
deliberately disable a plugin on purpose, it just can't happen by accident
via the plugin's own shipped schema default anymore.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
📝 WalkthroughWalkthroughAdds ChangesPlugin configuration flow
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 0 |
| Duplication | 0 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
1 participant
Summary
check_plugin.py,render_plugin.py, andtest/plugins/test_plugin_matrix.py(the real pytest CI matrix) each built config as{"enabled": True}then mergedconfig_schema.jsondefaults on top, letting a plugin's ownenabled: falsedefault silently win.enabled: falseas a reasonable default (seasonal/opt-in plugins like march-madness, odds-ticker, calendar, etc.) — every harness/CI render of those plugins was silently testing "disabled, do nothing" instead of real behavior.build_full_config()intotesting/loading.py(already the shared home for this kind of logic per its own docstring) and switched all three call sites to it: schema defaults → forcedenabled: True→ harness.json config → explicit caller config. A caller can still deliberately test the disabled path via an explicit override; it just can't happen by accident via the plugin's own shipped default anymore.Test plan
TestBuildFullConfigForcesEnabled(4 new tests) totest/plugins/test_harness.py— unit tests for the helper plus an end-to-endcheck_one()regression test with a fixture plugin whose schema defaultsenabled: false.pytest test/plugins/test_harness.py— 29 passed.pytest test/plugins/test_plugin_matrix.py(deselectingf1-scoreboard, which is independently rate-limited by its external API right now, unrelated to this change) — 23 passed, no regressions.check_plugin.py --plugin march-madness --sizes 192x48and--plugin odds-ticker --sizes 192x48, with no--configoverride, now render real (non-blank) content — previously required an explicit--config '{"enabled":true}'workaround.Summary by CodeRabbit
New Features
Bug Fixes
Tests