Compare json_view with yyjson, simdjson, and Boost.JSON

[nlohmann]

Part of the stack for the zero-copy view (#5295). This PR adds a benchmark comparison of json_view with other libraries, and a workflow to run it on GitHub-hosted runners. It is the last PR of the stack: it measures the parser and serializer work of the PRs below it.

Summary

tests/benchmarks/json_view/ holds a comparison with other libraries. It answers the question users ask when they pick a library. It is not built by CMake and not run by CI.

• bench_view.cpp: parse, traverse, select, and dump of twitter, citm_catalog, canada, jeopardy, a single tweet, and a JSON-RPC request.
  • Engines: json_view, yyjson, simdjson (DOM and On-Demand), Boost.JSON, and json::parse.
  • All engines must agree on every document (value count, bytes of all strings and keys, sum of numbers) before anything is timed, and they run interleaved in every round. The best round is reported.
• bench_corpus.cpp: parse, traverse, and dump of any list of files. Files that cannot be opened are reported instead of aborting the run.
• bench_edit.cpp: parses, applies the same logical edits with each library's own API (a handful at fixed places, or one in every record), and serializes; all outputs must describe the same value.
• compare.py: reproduces the numbers with one command.
  • It builds the three programs against include/, using the libraries of the system (pkg-config or Homebrew) or pinned downloads (--download): yyjson 0.13.0, simdjson 4.6.11, and Boost 1.92.0.
  • Downloads have a timeout and go to a .part file. Each archive is hashed in chunks and checked against its SHA-256; an archive that does not match is removed. It is extracted into a temporary directory that is renamed when complete, so an interrupted run is not mistaken for a finished one.
  • --data, --corpus and --build-dir are made absolute, since the benchmarks run in the build directory. CXX and CC may contain arguments (they are split with shlex).
  • It writes the results to results/<date>-<host>.md and .csv, together with the date, commit, CPU, OS, compiler, flags, and library versions.
  • Only the Python 3 standard library is used.
• README.md: how to run the comparison, what is measured, and which features each engine has, so that the numbers can be read correctly.
• .github/workflows/json_view_benchmarks.yml: runs compare.py --download on GitHub-hosted Ubuntu runners and shows the results as the job summary and as an artifact.
  • It never runs on its own: it is started by hand (workflow_dispatch: x86-64 or AArch64, GCC or Clang), or by adding the label benchmark to a pull request (both architectures, GCC).
  • It has timeout-minutes: 60 and a concurrency group keyed by the label, so adding an unrelated label does not cancel a running benchmark.
  • Shared runners are noisy, so the numbers show where json_view stands on another architecture; they are not for publication.
• .github/labeler.yml labels changes under tests/benchmarks/json_view/.

Fair comparison

• Each engine runs once untimed before each timed call. Otherwise the engine after json::parse pays for the allocator cleaning up the tens of thousands of nodes json::parse just freed (with glibc, this made json_view look 1.7 times slower than yyjson on citm_catalog traverse).
• bench_corpus compares the dump with source numbers (number_format::source) against yyjson writing numbers read as raw text (YYJSON_READ_NUMBER_AS_RAW); both write the same bytes.
• Columns "json_view" (fresh document) and "json_view (reused)" (parsed into the same document, timed for traverse and select too) are compared with "simdjson DOM" (parser reused) and "simdjson DOM (fresh)".
• Boost.JSON is optional (JSON_VIEW_BENCH_BOOST; --no-boost).

No results are committed. They should come from a quiet machine and include x86-64; the benchmark workflow gives a first x86-64 impression (label benchmark).

Performance

No library code changes in this PR. The numbers below were measured with this harness at the top of the stack.

Measured on an x86-64 KVM VPS (Haswell with AVX2, Ubuntu 24.04), compare.py --download --rounds 30 (-O3), fresh documents, µs; yyjson and simdjson DOM from the same run:

	json_view GCC 13	json_view Clang 18	yyjson GCC / Clang	simdjson DOM GCC / Clang
twitter parse	400	367	433 / 435	331 / 337
citm_catalog parse	948	939	1098 / 995	770 / 816
citm_catalog traverse	1138	1144	1228 / 1178	1066 / 1075

With twitter and citm_catalog, json_view is ahead of yyjson with both compilers, and 6–23% behind simdjson, which uses AVX2.

Dump with the numbers of the input (µs; same output bytes), x86-64 / Apple M1:

	json_view (source numbers)	yyjson (raw numbers)
canada	903 / 407	793 / 804
citm_catalog	283 / 134	307 / 181
twitter	169 / 73	201 / 107
jeopardy (ms)	72.2 / 15.0	73.6 / 20.5

On canada on x86-64, yyjson's lead is mostly the zero-fill of std::string::resize(), which yyjson's malloced output does not have.

bench_edit (parse, edit, serialize), Apple M1: twitter 309 µs (yyjson takes 1.65 times as long), citm_catalog 567 µs.

What tuning does not close: canada traverse is about 10% behind yyjson (the lazy float conversion runs at fast_float's speed; yyjson converts during the parse); canada dump is about 1.9x behind yyjson (converting each 17-digit token to a double during the dump).

Tests

None: this PR adds benchmarks, not unit tests. The benchmarks require all engines to agree on every document's content before any timing starts. compare.py is not run by CI.

Public API

No breaking changes, and no change to the API: this PR only adds files under tests/benchmarks/json_view/, a GitHub Actions workflow, and a labeler rule. Nothing in include/ changes.

---

Written by Claude Code.

🤖 Generated with Claude Code