1.2.0: fix streaming edge cases, linear-time scanner, packaging (no behaviour changes for valid input)

.github/workflows/tests.yml

@@ -9,7 +9,7 @@ jobs:
         runs-on: ubuntu-latest
         strategy:
             matrix:
-                python-version: ["3.9", "3.10", "3.11", "3.12"]
+                python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
         steps:
             - uses: actions/checkout@v4
             - uses: actions/setup-python@v5
@@ -20,5 +20,9 @@ jobs:
                   python -m pip install --upgrade pip
                   pip install -e .
                   pip install -r requirements-dev.txt
-            - name: Run tests
+            - name: Run tests (without optional json5 dependency)
               run: pytest -q
+            - name: Run tests (with optional json5 dependency)
+              run: |
+                  pip install -e '.[json5]'
+                  pytest -q

CHANGELOG.md

@@ -5,6 +5,83 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.2.0] - 2026-09-12
+
+No API changes. Every input that 1.1.0 parsed successfully parses to the same
+value in 1.2.0, verified by `tests/test_compat_1_1_0.py`, which runs the frozen
+1.1.0 parser next to the current one over every prefix of a corpus of documents.
+
+### Fixed
+
+- Numbers with an exponent (`1e5`, `2.5E-3`) inside an incomplete array or
+  object raised `JSONDecodeError`. They now parse; an exponent that has not
+  received its digits yet (`1e`, `1e-`) is dropped until it is complete.
+- In strict mode an unterminated string whose tail was an incomplete escape
+  (`"foo\`, `"foo\u00`) returned `""`, discarding text that had already
+  streamed. It now returns `"foo"`; only the unfinished escape is held back
+  (issue #8).
+- In strict mode a string cut between the two halves of a surrogate pair
+  (`"\ud83d`, half of an emoji) returned a lone surrogate, which raises
+  `UnicodeEncodeError` as soon as it is encoded. The high half is now held
+  back until its partner arrives.
+- The JSON5 parser raised on partial literals (`{"a": tr`, `[fals`, `[Inf`)
+  and on exponent numbers, and treated a comment that had only streamed its
+  first `/` as an unknown token. It now behaves like the JSON parser.
+- JSON5 string decoding no longer depends on whether the optional `json5`
+  package is installed; the same escapes (`\x41`, `\'`, line continuations,
+  surrogate pairs) decode the same way either way.
+- `bytes` and `bytearray` input, which `json.loads` accepts, no longer crash
+  the fallback parser with `AttributeError`. A chunk that ends in the middle
+  of a multi-byte UTF-8 character drops the incomplete bytes.
+- A leading UTF-8 byte-order mark no longer causes a `JSONDecodeError`.
+- `JSONParser.strict`, `.on_extra_token` and `.last_parse_reminding` are
+  readable and assignable again (assigning `strict` on a 1.x parser was
+  silently ignored), and the 0.x method names `parse_string`, `parse_number`,
+  `parse_array`, `parse_object`, `parse_true`, `parse_false`, `parse_null`
+  and `parse_space` are callable again.
+
+### Changed
+
+- The scanner works on string indexes instead of re-slicing the input at
+  every token, so a parse is linear in the input size. A 900 KB partial
+  document went from about 1 s to about 60 ms per `parse()` call.
+- A literal that is not a prefix of `true`/`false`/`null` (for example
+  `[trap]`, which 1.1.0 returned as `[True]`) now raises, matching
+  `json.loads`. Prefixes such as `[t`, `[tru` still parse.
+- `_JSON5Parser` is now a subclass of the JSON parser instead of a copy of it.
+- Packaging moved to `pyproject.toml` with `requires-python >= 3.8`,
+  classifiers and a `py.typed` marker; the package is fully type-annotated.
+- CI runs on Python 3.8 through 3.14, with and without the optional `json5`
+  dependency.
+
+## [1.1.0] - 2026-02-20
+
+### Added
+
+- JSON5 support: comments, unquoted keys, single-quoted strings, hex numbers,
+  `Infinity`/`NaN`, trailing commas. Available through
+  `create_json5_parser()` or `JSONParser(json5_enabled=True)`; install
+  `partialjson[json5]` for the optional `json5` fast path (issue #9).
+- `create_json_parser()` factory.
+
+## [1.0.0] - 2026-02
+
+### Added
+
+- `CITATION.cff`, `CODE_OF_CONDUCT.md`, `CONTRIBUTING.md`, JOSS paper draft,
+  GitHub Actions test workflow.
+
+### Changed
+
+- `JSONParser` became a thin facade over an internal implementation class.
+
+## [0.1.0] - 2025-01-28
+
+### Fixed
+
+- Incomplete escape sequences (`"\`, `"\u12`) at the end of a streamed
+  string no longer raise (issue #8).
+
 ## [0.0.8] - 2024-08-03
 
 ### Added

MANIFEST.in

@@ -0,0 +1,3 @@
+include LICENSE README.md CHANGELOG.md CITATION.cff
+graft tests
+global-exclude __pycache__ *.py[cod]

README.md

@@ -47,7 +47,30 @@ print(parser.parse(incomplete_json5))
 # {'name': 'Demo', 'version': 1.0, 'items': [1, 2, 3]}
 ```
 
-Install the optional `json5` dependency for full JSON5 support: `pip install partialjson[json5]`
+The optional `json5` dependency speeds up parsing of complete JSON5 documents: `pip install partialjson[json5]`. Partial documents parse the same way with or without it.
+
+### What you get while a string is still streaming
+
+Text that has already arrived is returned; only what cannot be decided yet is held back. With `strict=True` (the default) escapes are decoded and an unfinished escape or half an emoji is dropped until it is complete:
+
+```python
+parser.parse('{"msg": "caf\\u00')        # {'msg': 'caf'}
+parser.parse('{"msg": "caf\\u00e9"')      # {'msg': 'café'}
+parser.parse('{"msg": "hi \\ud83d"')      # {'msg': 'hi '}
+parser.parse('{"msg": "hi \\ud83d\\ude00"')  # {'msg': 'hi 😀'}
+```
+
+With `strict=False` the raw text of an unfinished string is returned untouched, backslashes included.
+
+### Extra tokens
+
+If the input contains a complete value followed by more text, the value is returned and the callback passed as `on_extra_token` is called with the input, the value and the leftover text. The default callback prints to stdout; pass `on_extra_token=None` to silence it, or read `parser.last_parse_reminding` afterwards.
+
+```python
+parser = JSONParser(on_extra_token=None)
+parser.parse('{"a": 1} trailing')   # {'a': 1}
+parser.last_parse_reminding         # ' trailing'
+```
 
 ### Installation
 
@@ -65,11 +88,13 @@ Also can be found on [pypi](https://pypi.org/project/partialjson/)
 ## Testing
 
 ```bash
-pip install -e .
+pip install -e '.[json5]'
 pip install -r requirements-dev.txt
 pytest -q
 ```
 
+`tests/test_compat_1_1_0.py` runs the frozen 1.1.0 parser next to the current one over every prefix of a corpus of documents, so behaviour changes for existing users show up as test failures.
+
 ## Citation
 
 If you use this software, please cite it using the metadata in `CITATION.cff`.
@@ -88,7 +113,7 @@ Please refer to each project's style and contribution guidelines for submitting
 
 1.  **Fork** the repo on GitHub
 2.  **Clone** the project to your own machine
-3.  **Update the Version** inside **init**.py
+3.  **Update the Version** inside `partialjson/__init__.py` and add a `CHANGELOG.md` entry
 4.  **Commit** changes to your own branch
 5.  **Push** your work back up to your fork
 6.  Submit a **Pull request** so that we can review your changes

partialjson/__init__.py

@@ -1,13 +1,14 @@
 """
 Partial Json.
 
-Parsing ChatGPT JSON stream response — Partial and incomplete JSON parser python library for OpenAI
+Parse partial and incomplete JSON, such as a streaming LLM response, without
+crashing: ``JSONParser().parse('{"a": [1, 2')`` returns ``{"a": [1, 2]}``.
 """
 
-from .json_parser import JSONParser, create_json_parser
 from .json5_parser import create_json5_parser
+from .json_parser import JSONParser, create_json_parser
 
-__version__ = "1.1.0"
+__version__ = "1.2.0"
 __author__ = "Nima Akbarzadeh"
 __author_email__ = "iw4p@protonmail.com"
 __license__ = "MIT"
@@ -16,8 +17,8 @@
 PYPI_SIMPLE_ENDPOINT: str = "https://pypi.org/project/partialjson"
 
 __all__ = [
+    "PYPI_SIMPLE_ENDPOINT",
     "JSONParser",
-    "create_json_parser",
     "create_json5_parser",
-    "PYPI_SIMPLE_ENDPOINT",
+    "create_json_parser",
 ]