fix(ci): the invisible-character gate never matched anything - #497
fix(ci): the invisible-character gate never matched anything#497hyperpolymath wants to merge 2 commits into
Conversation
MEASURED 2026-08-27: this gate's pattern caught 0 OF 6 invisible-character test
cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi
override or word joiner.
ROOT CAUSE: the pattern used UTF-8 BYTE sequences (\xc2\xa0) while grep -P
matches CHARACTERS. Bytes c2 a0 are ONE character U+00A0; \xc2\xa0 asks for TWO
characters, U+00C2 then U+00A0, which is never present.
grep -P '\xc2\xa0' -> miss
grep -P '\x{a0}' -> MATCH
Only \x00 worked, being single-byte in both readings.
FIXED: codepoint escapes; C0 control characters \x01-\x08,\x0B,\x0C,\x0E-\x1F
added (TAB/LF/CR excluded); and grep -a, without which grep skips any NUL-bearing
file as binary.
The C0 range matters: a stray BACKSPACE byte made a workflow unparseable in
developer-ecosystem, so it never ran, and this linter called it clean.
Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
VERIFIED: YAML re-parsed, and the corrected pattern was confirmed to catch a real
NBSP before the change was kept.
|
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📜 Recent review details⏰ Context from checks skipped due to timeout. (25)
🔇 Additional comments (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe ChangesInvisible-character gate
Estimated code review effort: 1 (Trivial) | ~5 minutes Merge Risk: ⚪ Minimal · up to This localized workflow change corrects invisible-character detection and adds binary-safe scanning; no actionable merge-blocking risk remains, so it is merge-ready after normal checks. Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.) Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Up to standards ✅🟢 Issues
|
![codacy-production[bot]](https://avatars.githubusercontent.com/in/56611?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull Request Overview
This PR correctly addresses the failure of the invisible-character CI gate by transitioning to PCRE-compatible Unicode escapes (\x{...}) and ensuring the scanner processes binary-flagged files. The addition of the -a flag is the most significant change; it prevents GNU grep from outputting 'Binary file matches' text which previously invalidated the filename processing loop.
While the logic changes are sound and the PR is up to standards, there is a lack of test assets (e.g., dummy files containing specific invisible characters) to verify the gate's efficacy. Without these assets, the fix is not explicitly validated within the repository's own test suite, which may lead to regressions.
About this PR
- The PR does not include any test files (e.g., a dummy file containing intentional invisible characters) to verify the fix or protect against future regressions. Validation currently relies on the CI's own output without explicit test assets in the diff.
Test suggestions
- Detect Non-breaking Space (NBSP) U+00A0 using \x{a0}
- Detect Zero-width space (ZWSP) U+200B using \x{200b}
- Detect C0 controls (e.g., Backspace \x08) while skipping allowed whitespace (LF/CR/TAB)
- Successfully scan a file containing a NULL byte (\x00) using the -a flag
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Detect Non-breaking Space (NBSP) U+00A0 using \x{a0}
2. Detect Zero-width space (ZWSP) U+200B using \x{200b}
3. Detect C0 controls (e.g., Backspace \x08) while skipping allowed whitespace (LF/CR/TAB)
4. Successfully scan a file containing a NULL byte (\x00) using the -a flag
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| -o -name '*.idr' -o -name '*.zig' -o -name '*.v' -o -name '*.jl' \ | ||
| -o -name '*.gleam' -o -name '*.hs' -o -name '*.ml' -o -name '*.sh' \) \ | ||
| -exec grep -Prl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | ||
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null |
There was a problem hiding this comment.
🟡 MEDIUM RISK
Suggestion: The -a (binary-as-text) flag is the critical fix for the issue where the gate produced no results. When searching for characters like \x00, grep identifies the file as binary and returns 'Binary file matches' strings that the filename processing loop cannot handle; -a ensures the scan returns clean paths. Additionally, the -r flag is redundant when used with find, and batching the execution with + is significantly more efficient than ;.
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | |
| -exec grep -aPl "$PATTERNS" {} + > /tmp/empty-lint-results.txt 2>/dev/null |
| # non-breaking spaces, null bytes, and other invisible Unicode in source files. | ||
| set +e | ||
| PATTERNS='\xc2\xa0|\xe2\x80\x8b|\xe2\x80\x8c|\xe2\x80\x8d|\xef\xbb\xbf|\xc2\xad|\xe2\x80\x8e|\xe2\x80\x8f|\xe2\x80\xaa|\xe2\x80\xab|\xe2\x80\xac|\xe2\x80\xad|\xe2\x80\xae|\x00' | ||
| PATTERNS='\x00|[\x01-\x08\x0B\x0C\x0E-\x1F]|\x{a0}|\x{ad}|\x{200b}|\x{200c}|\x{200d}|\x{200e}|\x{200f}|\x{202a}|\x{202b}|\x{202c}|\x{202d}|\x{202e}|\x{2060}|\x{feff}' |
There was a problem hiding this comment.
⚪ LOW RISK
Suggestion: Switching to \x{...} for Unicode code points is a significant improvement. It makes the intended character explicit to the PCRE engine, whereas the previous byte-based hex sequences could be misinterpreted depending on the environment locale settings.
1 participant
Measured 2026-08-27: this gate caught 0 of 6 invisible-character test cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi override or word joiner.
Root cause
The pattern used UTF-8 byte sequences (
\xc2\xa0) whilegrep -Pmatches characters. Bytesc2 a0are one character U+00A0;\xc2\xa0asks for two, U+00C2 then U+00A0 — never present.Only
\x00worked, being single-byte in both readings. The gate ran, passed, and could not see what it exists to see.Fixed
\x01-\x08,\x0B,\x0C,\x0E-\x1Fadded (TAB/LF/CR excluded)grep -a— without it grep skips any NUL-bearing file as binaryThe C0 range matters: a stray backspace byte made a workflow unparseable in
developer-ecosystem, so it never ran — and this linter called it clean.Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
Verified: YAML re-parsed, and the corrected pattern was confirmed to catch a real NBSP before the change was kept.