Repository navigation
Conversation
nlohmann
added this pull request to stack #5636
September 29, 2026 14:20
nlohmann
removed this pull request from stack #5636
September 30, 2026 13:19
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
from
September 30, 2026 13:20
2202ac0 to
b6f0daf
Compare
nlohmann
added this pull request to stack #5739
September 30, 2026 13:21
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
3 times, most recently
from
September 30, 2026 18:06
d50ae18 to
61afa69
Compare
nlohmann
marked this pull request as ready for review
September 30, 2026 18:16
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
from
September 30, 2026 19:19
61afa69 to
1df24a0
Compare
nlohmann
removed this pull request from stack #5739
October 6, 2026 09:26
This was referenced Oct 6, 2026
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
from
October 6, 2026 09:28
2db0e30 to
64c3dbd
Compare
nlohmann
added this pull request to stack #5768
October 6, 2026 09:29
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
from
October 7, 2026 14:44
64c3dbd to
accba4d
Compare
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
3 times, most recently
from
October 14, 2026 15:38
15031b9 to
2b9644d
Compare
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
from
October 14, 2026 15:55
2b9644d to
fa01cf2
Compare
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
from
October 16, 2026 06:46
fa01cf2 to
f81cfaf
Compare
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
from
October 16, 2026 17:42
f81cfaf to
1ef5204
Compare
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
from
October 17, 2026 06:02
1ef5204 to
7456054
Compare
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
from
October 17, 2026 06:32
7456054 to
0f75bf1
Compare
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
from
October 17, 2026 12:28
0f75bf1 to
601afa7
Compare
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
from
October 17, 2026 21:25
601afa7 to
ea0afb4
Compare
The default dump() (no indentation, no ensure_ascii) gets its own writer that makes the same walk and produces the same output: - the write position stays in a local variable instead of a member, so the compiler keeps it in a register across stores through aliasing char pointers; - strings and number tokens are copied with fixed-size 32-byte moves wherever enough source bytes remain, instead of one memcpy call per token; - the innermost open container lives in local variables; a stack that starts as a local array of 32 entries holds the rest; - unedited documents are walked through the node array in order, and integer tokens are read from the source directly. On top of that, float tokens of at most 15 significant digits are written straight from their digits via zmij::to_shortest() and write_shortest(), without converting to a double and back: such decimals are farther apart than a double's rounding interval, so the token's digits are the double's shortest digits. Tokens of 16+ digits, or edited values, still go through decimal_to_float(). The view's own NEON write_decimal() is removed in favor of the shared writer, and the dump output now grows in 64 KiB steps instead of being resized to its estimate at once. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Test the template options through enabled() so that conditions combined with them are not constant, and mark the nav::value() null dereference as a false positive like materialize.hpp does. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
A float node of an image loaded with image_check::bounds can hold a token whose bytes are not digits, so its value need not have the digit count recorded in the node. write_double_at() passed that count to write_short_decimal(), which indexes its table of powers of ten by it: an assertion failure in debug builds, an out-of-bounds read in release builds. Use the counted overload only if the value has exactly that many digits, otherwise count them. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
…nstant Signed-off-by: Niels Lohmann <mail@nlohmann.me>
… compile Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The MinGW linker of the Windows clang jobs cannot link object files with more
than 32767 sections ("relocation truncated to fit: IMAGE_REL_AMD64_REL32
against `.rdata'"). unit-json_view_edit.cpp has 35409 sections since the fast
dump instantiates more code for editable documents, so its test cases "views
and values", "deeply nested values", "pointers below a null value", and
"strings of other documents are checked" move into
unit-json_view_edit_values.cpp. The helper exception_of_call() that both
files use moves into json_view_test_helpers.hpp.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The dump and comparison tests moved to their own file in the dump pull request; the tests of the fast dump() join them there. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
nlohmann
force-pushed
the
json-view/22-view-dump-fast
branch
from
October 18, 2026 02:27
ea0afb4 to
2ca2cf0
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the stack for the zero-copy view (#5295). This PR gives the default
json_view::dump()a dedicated writer and makes it write floats from their digits. It builds on #5632 and on the Żmij float writer below it.Summary
A dedicated writer for the default dump()
The default
dump()(no indentation, noensure_ascii) gets its own writer,dump_compact. It makes the same walk and produces the same output, with these changes:charpointers may alias anything, so with the position as a member of the output buffer, the compiler reloads it after every store.output_buffergetscursor(),limit()andset_cursor()for this.memcpycall per token. Up to 256 bytes, several 32-byte moves are used. Longer copies, and copies near the end of the source, go through an out-of-linememcpy. The buffer keeps 64 bytes of slack for the overshoot, and the size estimate ofdump()includes it (+160 instead of +64), so the buffer does not grow just before the end.small_stack: it starts in a local array of 32 entries and moves to the heap only for deeper nesting, and its address does not escape, so its pointers stay in registers. Shallow documents need no allocation besides the output. The general writer (indented output,ensure_ascii) uses the same stack.-0is written as0, asparse()reads it; withnumber_format::sourcethe token is copied as it is).write_decoded: runs without a quote, a backslash or a control character are copied, and the rest is escaped.plain_output_runfinds these runs eight bytes at a time. It does not validate non-ASCII bytes: strings of a document are valid UTF-8, and a damaged image loaded withimage_check::boundswrites them unchanged here.Indented output and
ensure_asciistill use the general writer. Editable documents take the new path too (through the navigation policy); strings and tokens written by edits are copied with a plainmemcpy.Floats written from their digits
For
double(where it is IEEE 754 binary64; otherwise the general writer is used),json_view::dump()writes a float token of at most 15 significant digits from its digits, without converting it to a double and back. The output is unchanged: it is whatjson::dump()writes for the same value.DBL_DIG. So the token's digits, without trailing zeros, are the shortest digits of its double. The library writes exactly those (Żmij, viazmij::to_shortest()anddtoa_impl::write_shortest()); with Grisu2 this would not hold.decimal_to_float(), so the token is not read twice, and then written from the bits of the double, without the checks of the generalto_chars()path. Non-finite values are written asnull.write_short_decimal()(new into_chars.hpp, withpowers_of_ten_16()) extends the digits to 16 and hands them towrite_shortest(). The overload with a digit count is used when the count from the token's layout matches the value; otherwise the digits are counted. A token from an image loaded withimage_check::boundscan have bytes that are not digits, so the layout may not describe the value; the check keeps such a token from reaching the counted overload, anddump()can no longer read out of bounds for it.Output growth
dump()'s output now grows in steps of 64 KiB within its reserved size estimate (the source extent), instead of being resized to the estimate at once.resize()zero-fills, and for a pretty-printed source the estimate is far larger than the compact output. A step does not go beyond the reserved capacity, so a good estimate is never copied to a larger allocation.Performance
Measured on Apple M1 Max (Apple clang -O2) and an x86-64 KVM VPS (Haswell, GCC 13 / Clang 18 -O2, pinned to one core, about ±5–10% noise), µs, best of 3 interleaved rounds of 15 runs; files from nativejson-benchmark.
dump()of the whole document, with the general writer (before) and with this PR (after):Documents of mostly strings and integers gain the most (the copies and the registers); canada is dominated by writing 17-digit floats and gains the least.
Core library (json::parse / dump)
json::dump()is not affected. This PR addspowers_of_ten_16()andwrite_short_decimal()toto_chars.hpp; only the view uses them. The other changes are ininclude/nlohmann/detail/view/serializer.hppandjson_view.hpp.Tests
Some edit tests move from
unit-json_view_edit.cppto a new file,tests/src/unit-json_view_edit_values.cpp, so that no test object exceeds the 32,767 sections that the MinGW linker of the Windows clang job accepts.unit-json_view_dump.cpp: 20,000 random float tokens with 1 to 17 significant digits, in every spelling (with and without a point,e/Eexponents, signs, leading and trailing zeros, exponents from about 1e-320 to 1e300), checked againstjson::dump()of the same text. Thenumber_format::sourceoutput is also checked indented and withensure_ascii. The existing tests comparedump()withordered_json::dump()on generated documents, edited documents (the differential test), and loaded images.unit-json_view_image.cpp: float tokens of an image loaded withimage_check::boundswhose bytes are not digits, or whose digit count differs from the recorded layout.dump()must still write a number.unit-json_view_edit.cpp: float types other thandouble(herefloat) are written asbasic_jsonwrites them, with non-finite values asnull.unit-to_chars.cpp:write_short_decimal()againstto_chars()of the same double, for examples and for 100,000 random digit strings, exponents and trailing zeros, with and without the digit count.powers_of_ten_16()is checked as well.Public API
No breaking changes. The output of
json_view::dump()is unchanged. The newdtoa_impl::write_short_decimal()andpowers_of_ten_16()are indetail; the public API does not change.Written by Claude Code.
🤖 Generated with Claude Code