Skip to content

JSON: Add opt-in comments and trailing comma flags - #23336

Draft
otar wants to merge 15 commits into
php:masterfrom
otar:feature/json-comments-and-trailing-comma-parsing
Draft

otar wants to merge 15 commits into
php:masterfrom
otar:feature/json-comments-and-trailing-comma-parsing

Conversation

@otar

@otar otar commented Aug 17, 2026

Copy link
Copy Markdown

Configuration files often use a narrow JSON extension with JS-style comments, trailing commas, or both. PHP currently requires a separate parser or a string-aware preprocessing pass. This PR adds native parser support. Strict JSON remains the default.

Public API

Constant Value Accepted by Effect
JSON_ALLOW_COMMENTS 1 << 23 json_decode(), json_validate() Allows // and non-nesting /* ... */ comments at JSON whitespace boundaries.
JSON_ALLOW_TRAILING_COMMAS 1 << 24 json_decode(), json_validate() Allows one final comma in a nonempty array or object.

The flags are independent. Neither flag applies to json_encode() or is enabled by default.

Semantics

  • Line comments end at LF, CRLF, lone CR, or EOF. Block comments do not nest and end at the first */.
  • Comments are allowed before and after the root value and between tokens. They are not allowed inside a token. Comment-like text in a JSON string is unchanged.
  • U+2028/U+2029 and valid controls, including embedded NUL, are comment content. Malformed UTF-8 follows the existing decode flags. Validation still accepts JSON_INVALID_UTF8_IGNORE, but not JSON_INVALID_UTF8_SUBSTITUTE.
  • Unterminated block comments produce JSON_ERROR_SYNTAX at the opening slash.
  • A trailing comma is allowed only after the last element or member of a nonempty container. Empty or missing-element forms such as [,], {,}, and [1,,] remain invalid.
  • The implementation retains existing behavior for depth, duplicate keys, bigints, associative objects, exceptions, error codes, and original-input line and column numbers.

@otar
otar requested review from bukka and kocsismate as code owners August 17, 2026 17:07
@otar
otar marked this pull request as draft August 17, 2026 17:08
@otar

otar commented Aug 17, 2026

Copy link
Copy Markdown
Author

Sorry @bukka and @kocsismate, I accidentally tagged you on this PR. Please ignore this draft...

Comment on lines +35 to +36
} catch (JsonException $e) {
echo get_class($e), ': ', $e->getCode(), ': ', $e->getMessage(), "\n";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
} catch (JsonException $e) {
echo get_class($e), ': ', $e->getCode(), ': ', $e->getMessage(), "\n";
} catch (Throwable $e) {
echo $e::class, ': ', $e->getCode(), ': ', $e->getMessage(), "\n";

Comment on lines +31 to +57
} catch (JsonException $e) {
echo $e->getCode(), ': ', $e->getMessage(), "\n";
}

?>
--EXPECT--
NULL
5
array(1) {
[0]=>
string(2) "ab"
}
0
array(1) {
[0]=>
string(5) "a�b"
}
0
NULL
5
int(1)
0
bool(false)
5
bool(true)
0
5: Malformed UTF-8 characters, possibly incorrectly encoded near location 1:9

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
} catch (JsonException $e) {
echo $e->getCode(), ': ', $e->getMessage(), "\n";
}
?>
--EXPECT--
NULL
5
array(1) {
[0]=>
string(2) "ab"
}
0
array(1) {
[0]=>
string(5) "a�b"
}
0
NULL
5
int(1)
0
bool(false)
5
bool(true)
0
5: Malformed UTF-8 characters, possibly incorrectly encoded near location 1:9
} catch (Throwable $e) {
echo $e::class, ': ', $e->getCode(), ': ', $e->getMessage(), "\n";
}
?>
--EXPECT--
NULL
5
array(1) {
[0]=>
string(2) "ab"
}
0
array(1) {
[0]=>
string(5) "a�b"
}
0
NULL
5
int(1)
0
bool(false)
5
bool(true)
0
JsonException: 5: Malformed UTF-8 characters, possibly incorrectly encoded near location 1:9

Comment on lines +67 to +96
} catch (JsonException $e) {
echo $e->getCode(), ': ', $e->getMessage(), "\n";
}

?>
--EXPECT--
[1]
[1]
[1]
{"a":1}
[[1],{"a":2}]
error 4: Syntax error near location 1:2
error 4: Syntax error near location 1:2
error 4: Syntax error near location 1:4
error 4: Syntax error near location 1:4
error 4: Syntax error near location 1:8
error 4: Syntax error near location 1:4
error 4: Syntax error near location 1:4
error 4: Syntax error near location 1:4
[1]
{"a":1}
error 2: State mismatch (invalid or malformed JSON) near location 1:4
error 2: State mismatch (invalid or malformed JSON) near location 1:8
bool(true)
bool(true)
bool(false)
bool(true)
bool(false)
bool(false)
4: Syntax error near location 1:4

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
} catch (JsonException $e) {
echo $e->getCode(), ': ', $e->getMessage(), "\n";
}
?>
--EXPECT--
[1]
[1]
[1]
{"a":1}
[[1],{"a":2}]
error 4: Syntax error near location 1:2
error 4: Syntax error near location 1:2
error 4: Syntax error near location 1:4
error 4: Syntax error near location 1:4
error 4: Syntax error near location 1:8
error 4: Syntax error near location 1:4
error 4: Syntax error near location 1:4
error 4: Syntax error near location 1:4
[1]
{"a":1}
error 2: State mismatch (invalid or malformed JSON) near location 1:4
error 2: State mismatch (invalid or malformed JSON) near location 1:8
bool(true)
bool(true)
bool(false)
bool(true)
bool(false)
bool(false)
4: Syntax error near location 1:4
} catch (Throwable $e) {
echo $e::class, ': ', $e->getCode(), ': ', $e->getMessage(), "\n";
}
?>
--EXPECT--
[1]
[1]
[1]
{"a":1}
[[1],{"a":2}]
error 4: Syntax error near location 1:2
error 4: Syntax error near location 1:2
error 4: Syntax error near location 1:4
error 4: Syntax error near location 1:4
error 4: Syntax error near location 1:8
error 4: Syntax error near location 1:4
error 4: Syntax error near location 1:4
error 4: Syntax error near location 1:4
[1]
{"a":1}
error 2: State mismatch (invalid or malformed JSON) near location 1:4
error 2: State mismatch (invalid or malformed JSON) near location 1:8
bool(true)
bool(true)
bool(false)
bool(true)
bool(false)
bool(false)
JsonException: 4: Syntax error near location 1:4

try {
json_validate('{}', flags: RELAXED | $invalidFlag);
} catch (ValueError $e) {
echo get_class($e), "\n";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you want to assert the message here too. Same format as above.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NickSdot thanks for the feedback. I really appreciate it and will take it into consideration.

…ma-parsing

Resolve the conflict in sapi/fuzzer/fuzzer-json.c: keep the option sets
that master added in 60ddc8f (php#23877) and append the
JSON_ALLOW_COMMENTS and JSON_ALLOW_TRAILING_COMMAS sets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaWra1qpfXbgU9JZ4LozYB
Catch Throwable instead of a specific exception class, so that an
unexpected exception type shows in the test output instead of escaping
the try block. Print $e::class, the code and the message, and also assert
the ValueError message for the invalid json_validate() flags.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XaWra1qpfXbgU9JZ4LozYB
claude and others added 11 commits October 12, 2026 17:50
Outside comments, the JSON scanner treats a bare CR as whitespace and
counts only LF and CRLF as line endings. The comment rules also counted
a bare CR as a line ending. As a result, an error location after a bare
CR in a comment did not agree with the same input without the comment.

A bare CR still ends a line comment, but it no longer starts a new line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCee9RRsbzYcC4ovMxhU6H
Outside comments and inside strings, an embedded NUL byte is a control
character error. The comment rules skipped it as comment text. Some
other parsers end a comment at a NUL byte, so the same input decoded to
different values in PHP and in those parsers.

An embedded NUL in a line or block comment now gives
JSON_ERROR_CTRL_CHAR at the NUL byte. Other control characters stay
valid comment text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCee9RRsbzYcC4ovMxhU6H
The trailing-comma rules add a default reduction after a comma. Without
JSON_ALLOW_TRAILING_COMMAS, this reduction also runs when the scanner
returns an error, and only the errcode check in its action keeps the
scanner's error code. The test ran only with the relaxed flags, so it
did not run this check.

Run the cases also without flags, add an object case, and check
json_validate() too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCee9RRsbzYcC4ovMxhU6H
The block comment case had the invalid byte in the comment and also in
the string. Without flags, the string also gave error 5, and the test
did not print the location. Thus the test did not show that the comment
itself reports the error. A block comment rule that always skipped
invalid bytes also passed the test. The line comment case did not test
JSON_INVALID_UTF8_SUBSTITUTE or json_validate().

Put the invalid byte only in the comment, run the same checks for both
comment types, and print the error message so that the location is
also checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCee9RRsbzYcC4ovMxhU6H
Two cases used '\n' in single quotes, so the input contained a backslash
and an "n" instead of a newline. Use double quotes so that each case
tests the input that its comment describes. The expected output does
not change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCee9RRsbzYcC4ovMxhU6H
Add regression tests for comment delimiters that touch: an empty block
comment, an extra '*' before the closer, and a '/' right after the
opener. Also test a comment directly after a float and an exponent,
and a comment inside a number.

All cases already behave correctly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCee9RRsbzYcC4ovMxhU6H
The "member ','" and "element ','" rules had the same action. When
trailing commas were not allowed, each action freed the value by hand
before YYERROR, because bison does not run %destructor for the symbols
of the rule whose action calls YYERROR.

Move the comma into its own trailing_comma rule, as object_end and
array_end already do. YYERROR then removes only the comma, and the
%destructor frees the member or element value. php_json_yyerror()
keeps an error code that the scanner already set.

There is no change in behavior.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCee9RRsbzYcC4ovMxhU6H
While the scanner is in COMMENT_BLOCK, the rules jump to the condition
label and do not pass the std label, so s->token keeps the position of
the "/*" opener. The comment_token copy and its restore had no effect.

Keep the line and line start copies, because the line ending rules
change them. Correct the code comment, which said that comments are
always fully consumed before the function returns.

There is no change in behavior.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCee9RRsbzYcC4ovMxhU6H
The dictionary had "//" and "/**/", but not the block comment
delimiters alone, a trailing comma before a closer, or a bare CR, which
ends a line comment. Add "/*", "*/", ",]", ",}" and "\x0d".

libFuzzer only accepts \\, \" and \xNN escapes in a dictionary, so the
CR is written as "\x0d".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCee9RRsbzYcC4ovMxhU6H
Describe the JSON_ALLOW_COMMENTS and JSON_ALLOW_TRAILING_COMMAS flags
under New Features, the new json_decode() and json_validate() flags
under Changed Functions, and the two constants under New Global
Constants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCee9RRsbzYcC4ovMxhU6H
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants