Skip to content

Keep JSON_DIAGNOSTICS parent pointers of ordered_json members after erase() and update() - #5552

Merged
nlohmann merged 5 commits into
developfrom
claude/silly-rhodes-f0f089
Sep 25, 2026
Merged

nlohmann merged 5 commits into
developfrom
claude/silly-rhodes-f0f089

Conversation

@nlohmann

Copy link
Copy Markdown
Owner

With JSON_DIAGNOSTICS enabled, erase() and update() can leave members of an ordered_json object with a null parent pointer. The next assert_invariant() on that object then aborts, for example when the object is copied. In NDEBUG builds nothing aborts, but diagnostic messages silently lose part of the path, e.g. (/x) instead of (/b/x). nlohmann::json isn't affected, because std::map doesn't move its entries.

Minimal reproducer (aborts on develop):

#define JSON_DIAGNOSTICS 1
#include <nlohmann/json.hpp>
using J = nlohmann::ordered_json;
int main()
{
    J a = J::parse(R"({"a":{"c":{"d":{}}}})");
    a.merge_patch(J::parse(R"({"a":{"c":{"c":"s","d":null},"e":"s"}})"));
}

Root cause

ordered_json stores its members in a vector. Two operations move members without restoring their parent pointers:

  1. Erasing a member that is not the last one. ordered_map::erase rebuilds every later member in place (destroy + placement-new). The basic_json move constructor doesn't carry m_parent over, and none of the object branches of basic_json::erase (by key, iterator or range) repaired this. merge_patch() with a null member and patch() with a remove operation go through the same code.
  2. update() adding a key that makes the vector grow. Reallocation copy-constructs the other members, because std::pair<const std::string, basic_json> isn't nothrow-movable. Those copies get a null parent, and update() only set the parent of the member it inserted. The it2->second.set_parents() added for json::update() with merge_objects==true may trigger JSON_ASSERT for some objects #4813 repaired this only for the nested object during a merge, not for the object update() is called on. The json::update() with merge_objects==true may trigger JSON_ASSERT for some objects #4813 test still passed because it only uses CHECK_NOTHROW and never copies the result.

In the reproducer, /a/c becomes [d, c], and erasing d shifts c into slot 0 with a null parent. The assertion fires when inserting e makes /a's vector reallocate: that reallocation copies /a/c, and the copy constructor checks the source.

Changes

  • A new private helper, set_parents_after_object_erase(), runs set_parents() after an erase if the object type is vector-backed (detail::is_ordered_map). It does nothing unless JSON_DIAGNOSTICS is on. All four object erase paths call it: both erase_internal overloads, erase(iterator) and erase(iterator, iterator).
  • update() now uses the existing set_parent(), which already refreshes all members for vector-backed objects. This makes the json::update() with merge_objects==true may trigger JSON_ASSERT for some objects #4813 workaround redundant, so it's removed.
  • A new section in tests/src/unit-diagnostics.cpp covers erase (key, iterator, range), patch remove, update(j), update(j, true) and the reproducer above, all on ordered_json. Each case checks two things: a diagnostic path read through const access (so operator[] can't repair the pointers; this catches the bug under NDEBUG) and a deep copy (which runs assert_invariant() on every node).

Testing

  • The new test aborts against the old header with asserts on. Under NDEBUG, six of its checks fail on the truncated path.
  • A randomized stress test (30,000 seeds, mixing operator[], all erase forms, update, merge_patch, patch, emplace and insert on ordered_json, with a deep copy after every step) passes with the fix. The old header aborts almost immediately.
  • Twenty-four related unit suites, built with and without -DJSON_DIAGNOSTICS=1, give the same results as the old header. The only difference is the new test.
  • Built with Apple clang 21 under C++11, C++17 and C++20 with -Wall -Wextra -Wpedantic -Werror. GCC wasn't tested locally.

Interaction with #5547

#5547's merge_patch rewrite erases through basic_json::erase, so it picks up this fix automatically. Its update_members() and update_members_iteratively() still use the old operator[](key).m_parent = ... pattern, and the iterative version never repairs the top-level object. When #5547 is rebased onto this, both functions should switch to set_parent(...) and drop their set_parents() workarounds.

Breaking changes

None. The public API is unchanged: the new helper is private, and the behavior only changes when JSON_DIAGNOSTICS is enabled.


This pull request was written by Claude Code.

🤖 Generated with Claude Code

@nlohmann nlohmann added the review needed It would be great if someone could review the proposed changes. label Sep 23, 2026
@nlohmann
nlohmann force-pushed the claude/silly-rhodes-f0f089 branch from af31bf0 to f3ee159 Compare September 24, 2026 15:14
…rase() and update()

ordered_json stores its members in a vector, and two operations moved
members without restoring their parent pointers afterwards:

- ordered_map::erase() re-constructs every member after the erased one in
  place. The basic_json move constructor leaves m_parent at nullptr, and
  none of the object branches of basic_json::erase() (by key, iterator, or
  iterator range) called set_parents(). This also affected merge_patch()
  with a null member and patch() with a remove operation.
- update() only set the parent pointer of the inserted member. Adding a key
  can reallocate the vector, which copies all other members and leaves
  their m_parent at nullptr. The set_parents() call added for #4813 only
  repaired this for the nested object of a merge, not for the target.

The next assert_invariant() on such an object (for instance, when copying
it) aborted, and diagnostic messages lost the path prefix above the moved
member. std::map-based json was not affected, because its nodes do not
move.

Erasing from an ordered_map object now calls set_parents(), and update()
uses set_parent(), which already refreshes all members for vector-based
objects. This makes the #4813 workaround redundant.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
…ter test

The merge_patch() case parses its input, so with JSON_DIAGNOSTIC_POSITIONS
the exception message also carries the byte range of the parsed value.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
…t-pointer test

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The iterative path of update() only set the parent pointer of the member
it inserted, like the recursive one did before. It now uses set_parent()
too, so ordered_json members that move when a nested object grows keep
their parents, and the set_parents() calls that patched this up after
each nested merge are gone.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
@nlohmann
nlohmann force-pushed the claude/silly-rhodes-f0f089 branch from f3ee159 to 3993137 Compare September 24, 2026 15:19
@nlohmann nlohmann added 🚀 ready to merge Ready to merge - just waiting for CI to complete. and removed review needed It would be great if someone could review the proposed changes. labels Sep 24, 2026
@nlohmann nlohmann added this to the Release 3.13.0 milestone Sep 24, 2026
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
@nlohmann
nlohmann merged commit 01b53c8 into develop Sep 25, 2026
4 of 160 checks passed
@nlohmann
nlohmann deleted the claude/silly-rhodes-f0f089 branch September 25, 2026 06:29
nlohmann added a commit that referenced this pull request Sep 25, 2026
Resolve the conflict in json.hpp by keeping both the comparison helpers and
set_parents_after_object_erase() from #5552, and regenerate the amalgamation.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

L 🚀 ready to merge Ready to merge - just waiting for CI to complete. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants