fix(ci): the invisible-character gate never matched anything - #50
Conversation
MEASURED 2026-08-27: this gate's pattern caught 0 OF 6 invisible-character test
cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi
override or word joiner.
ROOT CAUSE: the pattern used UTF-8 BYTE sequences (\xc2\xa0) while grep -P
matches CHARACTERS. Bytes c2 a0 are ONE character U+00A0; \xc2\xa0 asks for TWO
characters, U+00C2 then U+00A0, which is never present.
grep -P '\xc2\xa0' -> miss
grep -P '\x{a0}' -> MATCH
Only \x00 worked, being single-byte in both readings.
FIXED: codepoint escapes; C0 control characters \x01-\x08,\x0B,\x0C,\x0E-\x1F
added (TAB/LF/CR excluded); and grep -a, without which grep skips any NUL-bearing
file as binary.
The C0 range matters: a stray BACKSPACE byte made a workflow unparseable in
developer-ecosystem, so it never ran, and this linter called it clean.
Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
VERIFIED: YAML re-parsed, and the corrected pattern was confirmed to catch a real
NBSP before the change was kept.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📜 Recent review details⏰ Context from checks skipped due to timeout. (20)
🔇 Additional comments (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe workflow now detects invisible characters using code-point escapes, includes additional C0 controls, and scans binary files as text. ChangesInvisible-character gate
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to This localized CI workflow change corrects invisible-character detection, and no actionable merge-blocking risk remains beyond normal checks and review. Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Linked Issues checkExplanation The PR implements the codepoint escapes, C0 control range, and grep -a requirements from [ Resolution Add the separate byte-wise leading-BOM detection. Update stdlib/ByteDetector.affine and config.ncl with the matching C0-control logic. Verify that invalid characters are detected and that clean files and files containing tabs, CR, and LF are not flagged. Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🔍 Hypatia Security ScanFindings: 40 issues detected
View findings[
{
"reason": "Issue in codeql.yml",
"type": "missing_timeout_minutes",
"file": "codeql.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in push-email-notify.yml",
"type": "missing_timeout_minutes",
"file": "push-email-notify.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in scorecard-enforcer.yml",
"type": "scorecard_publish_with_run_step",
"file": "scorecard-enforcer.yml",
"action": "split_scorecard_publish_job",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "Issue in instant-sync.yml",
"type": "secret_action_without_presence_gate",
"file": "instant-sync.yml",
"action": "peter-evans/repository-dispatch",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "Required file missing (condition: public_repo)",
"type": "missing_requirement",
"file": "SECURITY.md",
"action": "create",
"rule_module": "cicd_rules",
"severity": "high"
},
{
"reason": "Download-and-execute pattern (curl|wget pipe to shell) -- verify integrity before execution (3 occurrences, CWE-494)",
"type": "shell_download_then_run",
"file": "/home/runner/work/live-files/live-files/setup.sh",
"action": "flag",
"rule_module": "code_safety",
"severity": "high"
},
{
"reason": "No SECURITY.md found in live-files",
"type": "SecurityPolicy",
"file": "/home/runner/work/live-files/live-files",
"action": "auto_fix",
"rule_module": "scorecard",
"severity": "medium",
"remediation": "Add SECURITY.md documenting how to report vulnerabilities.",
"scorecard_check": "Security-Policy"
},
{
"reason": "Code scanning (Hypatia): hypatia/scorecard/SecurityPolicy -- Hypatia scorecard: SecurityPolicy -- 3 day(s) old",
"type": "CSA001",
"file": "live-files",
"action": "review",
"rule_module": "code_scanning_alerts",
"severity": "medium"
},
{
"reason": "Code scanning (Hypatia): hypatia/cicd_rules/missing_requirement -- Hypatia cicd_rules: missing_requirement -- 3 day(s) old",
"type": "CSA001",
"file": "SECURITY.md",
"action": "update",
"rule_module": "code_scanning_alerts",
"severity": "high"
},
{
"reason": "Code scanning (Hypatia): hypatia/structural_drift/SD004 -- Hypatia structural_drift: SD004 -- 11 day(s) old [STALE]",
"type": "CSA001",
"file": ".machine_readable/6a2/PLAYBOOK.a2ml",
"action": "escalate",
"rule_module": "code_scanning_alerts",
"severity": "high"
}
]Powered by Hypatia Neurosymbolic CI/CD Intelligence |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Up to standards ✅🟢 Issues
|
![codacy-production[bot]](https://avatars.githubusercontent.com/in/56611?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull Request Overview
This PR successfully addresses a configuration issue where the invisible-character gate was ineffective due to invalid regex patterns. By transitioning from UTF-8 byte sequences to Unicode codepoint escapes and adding the -a flag to grep, the CI gate is now capable of detecting NBSP, BOM, and various C0 control characters even in files containing null bytes.
While the logic changes are correct and Codacy reports the PR is up to standards, there is a risk of future regressions because no automated test cases (dummy files with hidden characters) were added to the repository. The current verification remains manual.
About this PR
- Consider adding automated test files (dummy source files containing specific invisible characters like NBSP, BOM, and C0 controls) to the repository. Without these, it is difficult to programmatically ensure the gate remains functional across future CI changes.
Test suggestions
- Missing: Detect Non-breaking space (NBSP, U+00A0) in a source file
- Missing: Detect Zero-width space (U+200B) in a source file
- Missing: Detect Byte Order Mark (BOM, U+FEFF) in a source file
- Missing: Detect C0 control characters like Backspace (\x08) in a source file
- Missing: Scan and detect issues in a file containing null bytes (verifying -a flag functionality)
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Missing: Detect Non-breaking space (NBSP, U+00A0) in a source file
2. Missing: Detect Zero-width space (U+200B) in a source file
3. Missing: Detect Byte Order Mark (BOM, U+FEFF) in a source file
4. Missing: Detect C0 control characters like Backspace (\x08) in a source file
5. Missing: Scan and detect issues in a file containing null bytes (verifying -a flag functionality)
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| -o -name '*.idr' -o -name '*.zig' -o -name '*.v' -o -name '*.jl' \ | ||
| -o -name '*.gleam' -o -name '*.hs' -o -name '*.ml' -o -name '*.sh' \) \ | ||
| -exec grep -Prl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | ||
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null |
There was a problem hiding this comment.
⚪ LOW RISK
Suggestion: Using -exec ... {} \; is inefficient as it forks a new grep process for every file. Batching with {} + improves performance significantly. Additionally, the -r flag is redundant because find handles directory traversal and passes specific file paths to grep directly.
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | |
| -exec grep -aPl "$PATTERNS" {} + > /tmp/empty-lint-results.txt 2>/dev/null |
1 participant
Measured 2026-08-27: this gate caught 0 of 6 invisible-character test cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi override or word joiner.
Root cause
The pattern used UTF-8 byte sequences (
\xc2\xa0) whilegrep -Pmatches characters. Bytesc2 a0are one character U+00A0;\xc2\xa0asks for two, U+00C2 then U+00A0 — never present.Only
\x00worked, being single-byte in both readings. The gate ran, passed, and could not see what it exists to see.Fixed
\x01-\x08,\x0B,\x0C,\x0E-\x1Fadded (TAB/LF/CR excluded)grep -a— without it grep skips any NUL-bearing file as binaryThe C0 range matters: a stray backspace byte made a workflow unparseable in
developer-ecosystem, so it never ran — and this linter called it clean.Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
Verified: YAML re-parsed, and the corrected pattern was confirmed to catch a real NBSP before the change was kept.