fix: grammar according to rfc3987

CHANGELOG.md

@@ -6,9 +6,18 @@ All notable changes to this project will be documented in this file.
 
 <!-- unreleased things below this line -->
 
+* Fixed
+  * Align RFC 3987 grammar handling with the specification (via [#10])
+  * Fix RFC 3986-delegated IRI parsing edge cases (via [#10])
+* Changed
+  * Update grammar rules used by `is_valid_syntax(...)` and term-specific validators (via [#10])
 * Added
+  * Validate IPv6 compression (via [#10])
   * Assert static typing (via [#16])
+* Tests
+  * Add grammar regression tests (via [#10])
 
+[#10]: https://github.com/jkowalleck/rfc3987-syntax2/pull/10
 [#16]: https://github.com/jkowalleck/rfc3987-syntax2/pull/16
 
 ## v1.1.1 - 2026-07-14

README.md

@@ -20,8 +20,7 @@ The goal of `rfc3987-syntax2` is to provide a **lightweight, permissively licens
 - ✅ Designed for both **open source and proprietary use**
 - ✅ Powered by [Lark](https://github.com/lark-parser/lark), a fast, EBNF-based parser
 
-> 🧠 **Note:** This project focuses on **syntax validation only**. RFC 3987 specifies **additional semantic rules** (e.g., Unicode normalization, BiDi constraints, percent-encoding requirements) that must be enforced separately.
-
+> 🧠 **Note:** This project focuses on **syntax validation only**. RFC 3987 specifies **additional semantic rules** (e.g., Unicode normalization, BiDi constraints, percent-encoding requirements) that are intentionally out of scope.
 
 ## 📄 License, Attribution, and Citation
 
@@ -37,17 +36,16 @@ This project:
 
 ## ⚠️ Limitations
 
-The grammar and parser enforce **only the ABNF syntax** defined in RFC 3987. The following are **not validated** and must be handled separately for full compliance:
+The grammar and parser enforce **only the ABNF syntax** defined in RFC 3987 (and referenced RFC 3986 productions where applicable). The following are **not validated** and must be handled separately for full compliance:
 
 - ✅ Unicode **Normalization Form C (NFC)**
 - ✅ Bidirectional text (**BiDi**) constraints (RFC 3987 §4.1)
 - ✅ **Port number ranges** (must be 0–65535)
-- ✅ Valid **IPv6 compression** (only one `::`, max segments)
 - ✅ Context-aware **percent-encoding** requirements
+- ✅ Additional host/network semantics beyond ABNF syntax (for example, canonical representation expectations)
 
 ChatGPT 40 was used during the original development process. Errors may exist due to this assistance. Additional review, testing, and bug fixes by human experts is welcome.
 
-
 ## 📦 Installation
 
 ```sh
@@ -92,7 +90,7 @@ if is_valid_syntax_iri('http://github.com'):
 
 if not is_valid_syntax_iri('bob'):
     print("✗ Invalid IRI syntax")
-    
+
 if is_valid_syntax_iri_reference('bob'):
     print("✓ Valid IRI-reference syntax")
 ```
@@ -115,7 +113,7 @@ This grammar was derived from:
   → Defines IRI syntax and extensions to URI (e.g. Unicode characters, `ucschar`)  
   → https://datatracker.ietf.org/doc/html/rfc3987
 
-- **RFC 3986 – Uniform Resource Identifier (URI): Generic Syntax)**  
+- **RFC 3986 – Uniform Resource Identifier (URI): Generic Syntax**  
   → Provides reusable components like `scheme`, `authority`, `ipv4address`, etc.  
   → https://datatracker.ietf.org/doc/html/rfc3986
 
@@ -142,10 +140,10 @@ This grammar was derived from:
 | `ipchar`, `isegment` | RFC 3986 | Path characters and segments |
 | `isegment_nz_nc` | RFC 3987 | IRI-specific path constraint |
 | `iunreserved` | RFC 3987 | Includes `ucschar` |
-| `ucschar`, `iprivate`| RFC 3987 | Unicode support |
+| `ucschar`, `iprivate` | RFC 3987 | Unicode support |
 | `sub_delims` | RFC 3986 | Reserved characters |
 | `ip_literal` | RFC 3986 | IPv6 or IPvFuture in `[]` |
-| `ipv6address` | RFC 3986 | Expanded forms only |
+| `ipv6address` | RFC 3986 | RFC 3986 IPv6address production; syntax-level validation only |
 | `ipvfuture` | RFC 3986 | Forward-compatible |
 | `ipv4address` | RFC 3986 | Dotted-decimal IPv4 |
 | `ls32` | RFC 3986 | Final 32 bits of IPv6 |

src/rfc3987_syntax2/syntax_rfc3987.lark

@@ -57,8 +57,8 @@ ifragment: (ipchar | "/" | "?")*
 
 iunreserved: alpha | digit | "-" | "." | "_" | "~" | ucschar
 
-ucschar: /[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF]/
-iprivate: /[\uE000-\uF8FF]/
+ucschar: /[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF\U00010000-\U0001FFFD\U00020000-\U0002FFFD\U00030000-\U0003FFFD\U00040000-\U0004FFFD\U00050000-\U0005FFFD\U00060000-\U0006FFFD\U00070000-\U0007FFFD\U00080000-\U0008FFFD\U00090000-\U0009FFFD\U000A0000-\U000AFFFD\U000B0000-\U000BFFFD\U000C0000-\U000CFFFD\U000D0000-\U000DFFFD\U000E0000-\U000EFFFD]/
+iprivate: /[\uE000-\uF8FF\U000F0000-\U000FFFFD\U00100000-\U0010FFFD]/
 
 sub_delims: "!" | "$" | "&" | "'" | "(" | ")"
           | "*" | "+" | "," | ";" | "="
@@ -67,15 +67,15 @@ ip_literal: "[" (ipv6address | ipvfuture) "]"
 
 ipvfuture: "v" hexdig+ "." (unreserved | sub_delims | ":")+
 
-ipv6address: h16 ":" h16 ":" h16 ":" h16 ":" h16 ":" h16 ":" ls32
-           | "::" h16 ":" h16 ":" h16 ":" h16 ":" h16 ":" ls32
-           | h16 "::" h16 ":" h16 ":" h16 ":" h16 ":" ls32
-           | h16 ":" h16 "::" h16 ":" h16 ":" h16 ":" ls32
-           | h16 ":" h16 ":" h16 "::" h16 ":" h16 ":" ls32
-           | h16 ":" h16 ":" h16 ":" h16 "::" h16 ":" ls32
-           | h16 ":" h16 ":" h16 ":" h16 ":" h16 "::" ls32
-           | h16 ":" h16 ":" h16 ":" h16 ":" h16 ":" h16 "::" h16
-           | h16 ":" h16 ":" h16 ":" h16 ":" h16 ":" h16 ":" h16 "::"
+ipv6address:                                ( h16 ":" )~6 ls32
+           |                           "::" ( h16 ":" )~5 ls32
+           |                    h16?   "::" ( h16 ":" )~4 ls32
+           | ( ( h16 ":" )?     h16 )? "::" ( h16 ":" )~3 ls32
+           | ( ( h16 ":" )~0..2 h16 )? "::" ( h16 ":" )~2 ls32
+           | ( ( h16 ":" )~0..3 h16 )? "::"   h16 ":"     ls32
+           | ( ( h16 ":" )~0..4 h16 )? "::"               ls32
+           | ( ( h16 ":" )~0..5 h16 )? "::"               h16
+           | ( ( h16 ":" )~0..6 h16 )? "::"
 
 h16: hexdig
    | hexdig hexdig

tests/_data/valid_syntax.json

@@ -108,10 +108,60 @@
             "value": "пример/тест#часть2",
             "expect_lark": true,
             "reason": ""
+        },
+        {
+            "value": "ucschar/\u00A0-\uD7FF/\uF900-\uFDCF/\uFDF0-\uFFEF",
+            "expect_lark": true,
+            "reason": "`ucschar` ranges: %xA0-D7FF / %xF900-FDCF / %xFDF0-FFEF"
+        },
+        {
+            "value": "ucschar/\uD800\uDC00-\uD83F\uDFFD/\uD840\uDC00-\uD87F\uDFFD/\uD880\uDC00-\uD8BF\uDFFD",
+            "expect_lark": true,
+            "reason": "`ucschar` ranges: %x10000-1FFFD / %x20000-2FFFD / %x30000-3FFFD"
+        },
+        {
+            "value": "ucschar/\uD8C0\uDC00-\uD8FF\uDFFD/\uD900\uDC00-\uD93F\uDFFD/\uD940\uDC00-\uD97F\uDFFD",
+            "expect_lark": true,
+            "reason": "`ucschar` ranges: %x40000-4FFFD / %x50000-5FFFD / %x60000-6FFFD"
+        },
+        {
+            "value": "ucschar/\uD980\uDC00-\uD9BF\uDFFD/\uD9C0\uDC00-\uD9FF\uDFFD/\uDA00\uDC00-\uDA3F\uDFFD",
+            "expect_lark": true,
+            "reason": "`ucschar` ranges: %x70000-7FFFD / %x80000-8FFFD / %x90000-9FFFD"
+        },
+        {
+            "value": "ucschar/\uDA40\uDC00-\uDA7F\uDFFD/\uDA80\uDC00-\uDABF\uDFFD/\uDAC0\uDC00-\uDAFF\uDFFD",
+            "expect_lark": true,
+            "reason": "`ucschar` ranges: %xA0000-AFFFD / %xB0000-BFFFD / %xC0000-CFFFD"
+        },
+        {
+            "value": "ucschar/\uDB00\uDC00-\uDB3F\uDFFD/\uDB40\uDC00-\uDB7F\uDFFD",
+            "expect_lark": true,
+            "reason": "`ucschar` ranges: %xD0000-DFFFD / %xE0000-EFFFD"
+        },
+        {
+            "value": "iprivate?foo=\uE000-\uF8FF&bar=\uDB80\uDC00-\uDBBF\uDFFD&baz=\uDBC0\uDC00-\uDBFF\uDFFD",
+            "expect_lark": true,
+            "reason": "`iprivate` ranges: %xE000-F8FF / %xF0000-FFFFD / %x100000-10FFFD"
+        },
+        {
+            "value": "//[2345:0425:2CA1:0:0:0567:5673:23b5]/foo",
+            "expect_lark": true,
+            "reason": "`IPv6address` case: 6( h16 \":\" ) ls32"
+        },
+        {
+            "value": "//[2345::BEEF]/foo",
+            "expect_lark": true,
+            "reason": "`IPv6address` case: [ *5( h16 \":\" ) h16 ] \"::\" h16"
+        },
+        {
+            "value": "//[2041:0:140F::875B:131B]/foo",
+            "expect_lark": true,
+            "reason": "`IPv6address` case: [ *4( h16 \":\" ) h16 ] \"::\" ls32"
         }
     ],
     "reserved": [],
     "scheme": [],
     "pct_encoded": [],
     "ipv4address": []
-}
+}