Repository navigation
The parser doesn't allow parse_error() to return true and continue parsing. #3989
Description
Description
The documentation has contradictory information:
Here, it says that parse_error() must return false but here, it says that "the return value indicates whether the parsing should continue, so the function should usually return false."
However it seems that the code expects parse_error() to return false and doesn't continue parsing if it returns true. See parser::sax_parse_internal() where it propagates the return value of parse_error() instead of only propagating false values.
// case token_type::value_float:
if (JSON_HEDLEY_UNLIKELY(!std::isfinite(res)))
{
return sax->parse_error(...);
}Should be
// case token_type::value_float:
if (JSON_HEDLEY_UNLIKELY(!std::isfinite(res)))
{
if (!sax->parse_error(...))
{
return false;
}
break;
}This would allow a SAX handler to deal with non-finite numbers, for example by stringifying them instead of failing parsing altogether. Of course a cleaner way would be for the parser to handle that natively by introducing a new method virtual bool number_nonfinite(const string_t& s) = 0;
Reproduction steps
Have a SAX handler return true for some parse_error()
Expected vs. actual results
Expected: the parser to continue parsing the rest of the input.
Actual: the parser stops.
Minimal code example
No response
Error messages
No response
Compiler and operating system
N/A
Library version
developer branch
Validation
- The bug also occurs if the latest version from the
developbranch is used. - I can successfully compile and run the unit tests.
Activity
nlohmann commentedon Mar 22, 2023
The documentation is not clear enough: the function must return
false. Allowing the parser to proceed after an error needs more work.francislan commentedon Mar 22, 2023
Thanks for clarifying. Is there a bug that tracks "allow the parser to proceed after an error" or "allow parsing non-finite numbers"?
nlohmann commentedon Mar 24, 2023
Not that I am aware of.
nlohmann commentedon Nov 29, 2024
@francislan Is #4522 what you have in mind?
francislan commentedon Nov 29, 2024
@nlohmann Yes, exactly! Thanks!
nlohmann commentedon Nov 30, 2024
I am not entirely sure how this can be useful, but maybe I misunderstand your use case.
Assume an input like
[{1}, "a"]. This is wrong on many levels, and so far, parsing would stop after{was parsed but not string key was seen, so we would see the events[json.exception.parse_error.101] parse error at line 1, column 3: syntax error while parsing object key - unexpected number literal; expected string literal), last token:1If we would continue parsing, we would continue with the events:
[json.exception.parse_error.101] parse error at line 1, column 4: syntax error while parsing object separator - unexpected '}'; expected ':'), last token:1}[json.exception.parse_error.101] parse error at line 1, column 5: syntax error while parsing value - unexpected ','; expected '[', '{', or a literal), last token:1},[json.exception.parse_error.101] parse error at line 1, column 9: syntax error while parsing value - unexpected string literal; expected end of input), last token:"a"I think we would need to have a means to "stabilize" the parser - for instance, we could ignore the invalid value
{1}and continue parsing the array and return["a"]as result. This is of course very dependent on the client's requirements. But I am not sure how this could be realized.@francislan What is your use case?
nlohmann commentedon Jan 19, 2025
@francislan Can you give your input here?
Repair complete items in binary formats when parse_error() returns tr…
Fix CI: no global std::string in the #3989 regression test
Fix CI: clang-tidy and JSON_NOEXCEPTION in the #3989 changes
Fix CI: do not hide base class methods in the #3989 test SAX parsers
Fix CI: keep the binary readers' recovery code out of from_*()
Fix CI: no semicolon after CAPTURE() in the #3989 tests
Use the file-scope float_json alias in the #3989 test
Keep the MinGW test objects below the linker's section limit