Repository navigation
feat: add missing validators - #48
Merged
Merged
Conversation
- Populate all 30+ previously empty arrays in valid_syntax.json with RFC-driven test cases covering every grammar production in RFC 3987 §2.2: ucschar, iunreserved, sub_delims, iuserinfo, h16, dec_octet, ls32, ipv6address (all 9 forms), unreserved, ipvfuture, ip_literal, ireg_name, ihost, port, iauthority, ipchar, isegment, ipath_abempty, isegment_nz, ipath_absolute, ipath_rootless, ipath_empty, ihier_part, iprivate, iquery, gen_delims, ifragment, isegment_nz_nc, ipath_noscheme, ipath, irelative_part, irelative_ref, reserved, scheme, pct_encoded, ipv4address - Add invalid test cases to 5 previously empty sections in invalid_syntax.json: isegment_nz_nc, ipath_noscheme, ipath, irelative_part, irelative_ref - Remove RFC-wrong entry: a new IRI entry added by the script using literal ASCII asterisks '******example.com:...' was invalid (asterisk is not a valid scheme character; scheme must start with ALPHA). Removed. - All existing entries preserved (no deduplication). - Source of truth: RFC 3987 §2.2 grammar exclusively. - No tests were run per task instructions.
…t data Re-serialize both JSON files with ensure_ascii=True so all non-ASCII characters (especially invisible private-use area chars like U+E000 and ucschar boundary values like U+00A0) appear as explicit \\uXXXX escape sequences. This makes the intent unambiguous in JSON source review.
The previous ensure_ascii=True commit introduced an ambiguous \u10ffFF representation for the existing iprivate test entry (U+10FF followed by literal 'FF'). Revert to raw UTF-8 (ensure_ascii=False) which matches the original file style and avoids this ambiguity. The 'reason' field for each entry explicitly identifies codepoints by name/number.
In isegment_nz_nc, ipath_noscheme and ipath invalid entries, update reason text to say 'ASCII space (U+0020)' rather than just 'space', to avoid confusion with U+00A0 NO-BREAK SPACE which IS valid as a ucschar within iunreserved and thus allowed in path segments.
Use explicit \uXXXX notation for U+00A0 (NO-BREAK SPACE) in value fields so they are visually distinct from ASCII space (U+0020). Also escape all private-use area characters (BMP E000-F8FF, Plane 15 F0000-FFFFD, Plane 16 100000-10FFFD) for clarity. Visible characters (CJK, Arabic, Hebrew etc.) remain as raw UTF-8.
Normalise lowercase \uXXXX to uppercase \uXXXX throughout both JSON files for consistency with the uppercase codepoint notation used in reason strings (e.g. "U+00A0").
Add "NO-BREAK SPACE" name to U+00A0 references in the ipath_noscheme and ipath invalid-space entry reason strings, matching the explicit style used elsewhere in the file.
Signed-off-by: Jan Kowalleck <jan.kowalleck@gmail.com>
Collaborator
Author
1 similar comment
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
is_valid_syntax_schemeis_valid_syntax_non_zerois_valid_syntax_pct_encodedtests from #42