Skip to content

1.2.0: fix streaming edge cases, linear-time scanner, packaging (no behaviour changes for valid input) - #10

Merged
iw4p merged 1 commit into
mainfrom
improve/1.2.0
Sep 11, 2026
Merged

iw4p merged 1 commit into
mainfrom
improve/1.2.0

Conversation

@iw4p

@iw4p iw4p commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Summary

A backward-compatible 1.2.0. Every input that 1.1.0 parsed successfully parses to the same value here. That is not a promise but a test: tests/legacy_1_1_0_parser.py is the 1.1.0 parser frozen verbatim, tests/test_compat_1_1_0.py runs it next to the new parser over every prefix of a document corpus in both strict modes, and tests/fuzz_compat_1_1_0.py does the same over random documents (1.08 million prefixes checked locally, zero unexplained differences). The only allowed differences are the fixes below.

Fixed

  • Exponent numbers crashed the fallback parser. [1e5, {"a": 2.5E-3 raised JSONDecodeError. An exponent that has no digits yet (1e, 1e-) is held back until it does.
  • Strict mode lost already-streamed text. {"a":"foo\u00 returned {"a": ""}; now {"a": "foo"}. Only the unfinished escape is held back (Throws when in middle of escaped token #8).
  • Half an emoji came out as a lone surrogate. Cut between \ud83d and \ude00, the result raised UnicodeEncodeError on encode. The high half is now held back until its partner arrives.
  • JSON5 mode was less tolerant than JSON mode. Partial literals ({"a": tr, [fals, [Inf) and exponents raised, and a comment that had only streamed its first / was treated as a stray token.
  • JSON5 string decoding depended on whether json5 was installed. Escapes now decode identically either way; the optional package is only a fast path for complete documents.
  • bytes/bytearray input crashed the fallback parser; a leading BOM raised.
  • JSONParser.strict, .on_extra_token, .last_parse_reminding are readable and assignable again (assigning strict on 1.x was silently ignored), and the 0.x method names (parse_string, parse_number, ...) are callable again.

Changed

  • Linear-time scanner. The parser walks indexes instead of re-slicing the input at every token. 900 KB partial document: about 1 s per parse() in 1.1.0, about 60 ms here. Structure and all quirks of the old control flow are preserved on purpose.
  • Literals that are not a prefix of true/false/null now raise like json.loads ([trap] returned [True] in 1.1.0). Prefixes such as [t, [tru still parse. In valid JSON nothing but true can start with t, so returning True for a prefix is safe; the only change is that garbage is rejected instead of guessed.
  • _JSON5Parser subclasses _JSONParser instead of duplicating it.
  • pyproject.toml replaces setup.py: requires-python >= 3.8, classifiers, py.typed, full type hints (mypy --strict clean).
  • CI matrix: 3.8 through 3.14, each run with and without the optional json5 extra.
  • CHANGELOG entries for 1.0.0, 1.1.0 and 1.2.0; README documents streaming-string semantics and on_extra_token.

Deliberately not changed

  • The default on_extra_token still prints to stdout. Worth moving to warnings in a 2.0.
  • Non-strict mode still returns the raw text of an unfinished string (backslashes included) and decoded text once it closes; test_string_relaxed_returns_raw_content_when_incomplete pins this.

Verification

  • 176 tests pass on Python 3.8, 3.9 and 3.14, with and without json5.
  • ruff and mypy --strict clean.
  • sdist and wheel build; py.typed and the compatibility oracle ship.

Suggested release steps

  1. Let CI go green on this PR.
  2. Merge, tag v1.2.0, publish. Nothing in the public API changed, so a minor bump is right.

🤖 Generated with Claude Code

Behaviour-preserving release. Every input 1.1.0 parsed successfully parses
to the same value; tests/test_compat_1_1_0.py runs the frozen 1.1.0 parser
next to the new one over every prefix of a corpus, and
tests/fuzz_compat_1_1_0.py does the same for random documents.

Fixed
- Exponent numbers (1e5, 2.5E-3) inside an incomplete container raised.
- Strict mode returned "" for an unterminated string ending in an
  incomplete escape, losing already-streamed text (issue #8).
- Strict mode returned a lone surrogate when cut inside an escaped
  surrogate pair.
- JSON5 mode raised on partial literals and exponents, and mis-handled a
  comment that had only streamed its first '/'.
- JSON5 string decoding depended on whether the optional json5 package
  was installed.
- bytes input crashed the fallback parser; a leading BOM raised.
- JSONParser.strict / on_extra_token / last_parse_reminding are readable
  and assignable again; 0.x method names are callable again.

Changed
- Index-based scanner: parse time is linear in input size (900 KB partial
  document: ~1 s -> ~60 ms).
- Literals that are not a prefix of true/false/null now raise like
  json.loads instead of being accepted.
- _JSON5Parser subclasses _JSONParser instead of duplicating it.
- pyproject.toml, requires-python >= 3.8, classifiers, py.typed, full type
  hints; CI on 3.8-3.14 with and without json5.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@iw4p
iw4p merged commit 7e5dcac into main Sep 11, 2026
14 checks passed
@iw4p
iw4p deleted the improve/1.2.0 branch September 11, 2026 22:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant