Repository navigation
Conversation
|
CI fixes for this PR (commit 958fbc3, plus the lower fixes merged in):
Verified with — posted by Claude Code on behalf of @nlohmann |
🔴 Amalgamation check failed! 🔴The source code has not been amalgamated and/or formatted correctly, or |
29c684f to
4d99fe9
Compare
5dce18c to
a8beec0
Compare
a8beec0 to
e17e23a
Compare
5286c28 to
0adb778
Compare
0adb778 to
77acd45
Compare
c6c0a6d to
237370a
Compare
29b3c0c to
9e71c7e
Compare
0539b2d to
2ab1482
Compare
bc7a04e to
130e073
Compare
gregmarr
left a comment
There was a problem hiding this comment.
There are too many commits to try to review this commit by commit, since things keep changing. I'm going to submit these now. I hope they still make sense. I was reading commit "Make view lookups last-wins and chained access safe" or earlier when I wrote these. I'm going to switch to reading all the commit notes and then review all of the code at once.
|
|
||
| ## Exception safety | ||
|
|
||
| Strong exception safety: if an exception is thrown, there are no changes to the view or the document it refers to. |
There was a problem hiding this comment.
Per previous PR discussion, remove the "if an exception is thrown"?
There was a problem hiding this comment.
|
|
||
| 1. The function can throw the following exceptions, both with the same message as the corresponding call to | ||
| [`BasicJsonType::at`](../basic_json/at.md): | ||
| - Throws [`type_error.304`](../../home/exceptions.md#jsonexceptiontype_error304) if the value is not an object. |
There was a problem hiding this comment.
Would it be worthwhile to reference the new enum in all the docs? That would probably require moving the enum out of detail. Totally separate PR for that I think.
There was a problem hiding this comment.
Agreed, and for the basic_json pages as well, so in a separate PR: exception_id would first have to move out of detail. I'll leave the view pages like the basic_json ones for now.
— posted by Claude Code on behalf of @nlohmann
|
|
||
| None of these exceptions carry a [`JSON_DIAGNOSTICS`](../macros/json_diagnostics.md) path: the view has no | ||
| `BasicJsonType` value to point at, so the exception is created without one, even if `BasicJsonType` was built with | ||
| `JSON_DIAGNOSTICS` enabled. |
There was a problem hiding this comment.
Is the data not available through the index? Maybe this can be put alongside that other item to maybe add diagnostics to materialize() in the future.
There was a problem hiding this comment.
|
|
||
| 1. Linear in the number of members: as for [`ordered_json`](../ordered_json.md), members are compared one after | ||
| another, in document order, stopping at the first match. Each comparison first checks the key's length -- | ||
| already known from the index, without reading the key bytes -- before comparing its content. |
There was a problem hiding this comment.
If it were to iterate in the reverse order, would it produce the same results of a basic_json::parse() where later ones overwrite earlier ones?
There was a problem hiding this comment.
I see in another docs section that there are no reverse iterators. I suspect that means that this isn't possible.
There was a problem hiding this comment.
and yet in the next commit, there it is
There was a problem hiding this comment.
and then it disappears again
There was a problem hiding this comment.
Yes, the policy went back and forth across the commits. It is first-wins again since 9cc0ff9 (lookups stop at the first match, and there are no reverse iterators), with the reasons in Duplicate keys. Sorry for the churn while you were reading.
— posted by Claude Code on behalf of @nlohmann
|
|
||
| ## Exception safety | ||
|
|
||
| Strong exception safety: if an exception is thrown, there are no changes to the view or the document it refers to. |
There was a problem hiding this comment.
exceptions again, should probably just look at all of them.
|
|
||
| !!! danger "Lifetime issues" | ||
|
|
||
| As for `BasicJsonType::items()`, calling `items()` on a temporary view (or a temporary document) is dangerous: |
There was a problem hiding this comment.
It isn't needed: the range returned by items() holds a copy of the view, so v["a"].items() in a range-for is safe. Calling root() on a temporary document is already rejected (root() const&& is deleted). The "Lifetime issues" box was wrong; it says this now (7b75573).
— posted by Claude Code on behalf of @nlohmann
| [runtime assertion](../../features/assertions.md)) for a missing key on a **const** value, this operator always | ||
| returns a safe, testable result: a [discarded](is_discarded.md) view, which is `#!cpp false` in a boolean context. | ||
| There is also no non-const overload that inserts a missing key or extends an array -- a view never modifies the | ||
| document. |
There was a problem hiding this comment.
I feel the foreshadowing...
| a string representation of the type ([`value_t`](../basic_json/value_t.md)): | ||
|
|
||
| | Value type | return value | | ||
| |-----------------------------------------------------|---------------| |
| @@ -24,7 +24,7 @@ Constant. | |||
|
|
|||
| For an object, iteration visits **every** member, including all occurrences of a duplicate key -- unlike | |||
There was a problem hiding this comment.
This whole paragraph now needs to be rewritten.
There was a problem hiding this comment.
Probably was in a later commit, after the decision was reversed.
| ## Return value | ||
| An iterator to the member with key `key`, or [`end()`](end.md) if there is none or the value is not an object. |
There was a problem hiding this comment.
Settled on "first" (see the reply on at.md); find.md says "the first one" now.
— posted by Claude Code on behalf of @nlohmann
130e073 to
426dec2
Compare
gregmarr
left a comment
There was a problem hiding this comment.
These functions can produce a different result in basic_json_view than the materialized basic_json will. Does this need to be called out in the see also text?
|
|
||
| - [count](count.md) returns the number of occurrences of a key | ||
| - [contains](contains.md) checks whether a key exists | ||
| - [basic_json_view::find](../basic_json_view/find.md) - the same lookup on a zero-copy view |
There was a problem hiding this comment.
Agreed. The see-also entries on the four basic_json pages now say how the view differs (7b75573): find/operator[] resolve a duplicate key to the first member while parse() keeps the last, operator[] returns a discarded view instead of undefined behavior, and front/items follow document order and visit duplicates.
— posted by Claude Code on behalf of @nlohmann
| ## See also | ||
|
|
||
| - [back](back.md) to access the last element | ||
| - [basic_json_view::front](../basic_json_view/front.md) - the same access on a zero-copy view |
There was a problem hiding this comment.
but can produce a different result?
|
|
||
| - [begin](begin.md) returns an iterator to the first element | ||
| - [end](end.md) returns an iterator to one past the last element | ||
| - [basic_json_view::items](../basic_json_view/items.md) - the same range on a zero-copy view (`#!cpp const auto`, |
There was a problem hiding this comment.
But can produce different iteration order.
| - see [`at`](at.md) for access by reference with range checking | ||
| - see [`value`](value.md) for access with default value | ||
| - [basic_json_view::operator[]](../basic_json_view/operator%5B%5D.md) - the same access on a zero-copy view (always | ||
| returns a discarded view instead of assuming undefined behavior) |
There was a problem hiding this comment.
But can produce a different element
|
|
||
| Linear in the size of the array or object: unlike [`front()`](front.md), which only ever looks at the first | ||
| element, `back()` has to walk every element to find where they end, since elements are not a fixed size in the | ||
| index. |
There was a problem hiding this comment.
Does this happen enough that it should have an index entry? I assume this would make the entry bigger. This bigger entry might make some of the other difficult things easier, but will make it take up more space, and might have other performance impacts due to larger reads or more reads.
There was a problem hiding this comment.
Not with the current node: it is 16 bytes, containers already use off, len and next, and the 16-bit extra holds the hash index number of a large object (#5628). A wider node would cost every value for one rarely used function. I'd keep back() linear and documented, and revisit if a benchmark shows it matters.
— posted by Claude Code on behalf of @nlohmann
|
|
||
| Unlike [`BasicJsonType::back()`](../basic_json/back.md), which has undefined behavior for an empty array or object, | ||
| `back()` throws `invalid_iterator.214` in that case -- the same way it already does for `#!json null` and for a | ||
| discarded view, where `BasicJsonType::back()` also throws. |
There was a problem hiding this comment.
Seems overly complex.
Unlike BasicJsonType::back(), which has undefined behavior for an empty array or object, back() throws invalid_iterator.214 for an empty array, an empty object, a null value, or a discarded view.
|
|
||
| Because objects are iterated in document order rather than sorted by key, the order seen here can differ from what | ||
| iterating the [`materialize()`](materialize.md)d `BasicJsonType` value would produce: a `#!cpp basic_json` object | ||
| (`std::map`-backed by default) sorts its keys, while a view does not. |
There was a problem hiding this comment.
Mention that ordered_json doesn't sort either, but does remove duplicates?
| an `ObjectType` that allows multiple entries per key, `count()` here never does: it is exactly | ||
| [`contains()`](contains.md) as `#!cpp 0`/`#!cpp 1`. This holds even if the source text has a duplicate key -- see the | ||
| [Notes on duplicate keys](operator[].md#notes) of `operator[]` -- because a `#!cpp count() > 1` result would require | ||
| counting every member with a matching key (the lookup functions resolve to the *first* one). |
There was a problem hiding this comment.
The only reason to use this function over contains is to find out how many actually exist, so it seems like it should be the actual result. This matches the complexity of unordered_multimap.
| Returns an iterator to one past the last element of an array, one past the last member value of an object, in | ||
| **document order** -- see [`begin()`](begin.md). A primitive value iterates as a range of one element (itself); | ||
| `#!json null` and a [discarded](is_discarded.md) view iterate as an empty range, so `#!cpp begin() == end()` for | ||
| them. |
There was a problem hiding this comment.
It should be end() = begin() + 1 for a primitive or you can't iterate over them.
There was a problem hiding this comment.
It is already: for a primitive, end() is one past the node (m_node + 1), so begin()/end() is a range of one element. Only null and a discarded view are empty ranges, as for basic_json. end.md says so in its first lines.
— posted by Claude Code on behalf of @nlohmann
| return index_status::unresolved; | ||
| } | ||
| std::uint64_t v = 0; | ||
| for (std::size_t i = 0; i < s.size(); ++i) |
| { | ||
| if (is_discarded()) | ||
| { | ||
| return basic_json_view(); |
There was a problem hiding this comment.
Can this just be return {} to avoid repeating the return type?
|
|
||
| /// any other integer type, see operator[]; negative values are out of | ||
| /// range | ||
| template < typename IntegerType, typename std::enable_if < detail::view::is_index_type<IntegerType>::value, int >::type = 0 > |
There was a problem hiding this comment.
That spacing (template < typename T, typename std::enable_if < ... >::type = 0 >) is what the pinned astyle produces for this pattern; json.hpp has the same. make pretty leaves it like this.
— posted by Claude Code on behalf of @nlohmann
| /// the value a JSON pointer refers to converted to T, or the default | ||
| /// value if the pointer cannot be resolved. Throws type_error.306 if this | ||
| /// is neither an object nor an array. | ||
| template < typename T, typename std::enable_if < !std::is_same<typename std::decay<T>::type, const char*>::value, int >::type = 0 > |
There was a problem hiding this comment.
Same as above: astyle's output for this pattern.
— posted by Claude Code on behalf of @nlohmann
| } | ||
|
|
||
| template<typename T> | ||
| T& get_to(T& v) const |
There was a problem hiding this comment.
I thought the whole point of get_to<> was to use the existing storage of v.
Oh, also to add the json's data to the object, which may have values already, using from_json().
There was a problem hiding this comment.
Good point, it differed from basic_json::get_to(). Since 7b75573, types that get<T>() converts directly (numbers, strings, std::vector, maps with string keys, ...) are assigned, and all others go to materialize().get_to(v), i.e. from_json with the existing value, so a from_json that sets only the fields present keeps the others. There's a test for exactly that case.
— posted by Claude Code on behalf of @nlohmann
Give basic_json_view the read-only access functions of basic_json: operator[] and at() with keys and indices, front()/back(), find(), contains(), count(), begin()/end() and cbegin()/cend(), items() with structured bindings from C++17 on, and type_name(). Exceptions have the ids and messages of the const functions of basic_json. Where basic_json has undefined behavior the view answers safely: operator[] with a missing key or an out-of-range index returns a discarded view, and front()/back() of an empty container throw invalid_iterator.214. Objects are iterated in document order, and all members are visited; duplicate-key lookups find the first member (as yyjson and simdjson do), while parse(), materialize(), and the map conversions keep the last value, as parse() does. Keys of up to 16 bytes are compared with two overlapping loads. Add value conversions: get<T>()/get_to() for arithmetic types, bool, nullptr_t, strings (std::basic_string copied, string_view_t without a copy), BasicJsonType, views, std::vector, and maps with string keys; get_string() for the string without a copy; number_token() for the number exactly as written in the source; value() with keys and JSON pointers; and operator[]/at()/ contains() with JSON pointers. Everything else, including types with from_json(), goes through materialize() of that subtree. get<T>() of arithmetic types is inlined down to the conversion, so reading an integer needs no call. detail::json_pointer_access exposes a pointer's reference tokens to code outside basic_json. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
* Lookups (operator[], at, find, contains, count, value, JSON pointers) return the last member of a duplicate key, as materialize() and parse() keep it. * operator[] on a discarded view returns a discarded view instead of throwing, so v["a"]["b"] is safe for a missing "a". * operator[] and at() take any integer type (not only int and size_t), fixing ambiguous calls with unsigned, long, std::int64_t, ... * Fix the operator[] documentation, which claimed a discarded view for a type mismatch where type_error.305 is thrown. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
…d its tests A view is tested with is_discarded(); the lookups in at(), value() and contains(json_pointer) and the unit tests no longer rely on the explicit conversion to bool. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
…mporary root() of an rvalue document is deleted, because the views would dangle. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
…xamples The explicit conversion to bool of basic_json_view is gone. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Looking up the last member of a duplicate key cannot stop at a match, so every lookup scanned the whole object (1.6 to 3.4 times slower for small objects). operator[](key), at, find, value, contains, count, and JSON pointer resolution return the first member again, as yyjson and simdjson do; materialize() and get<map>() keep the last value, as parse(). The documentation says so in the feature page and on each lookup page, and explains how to get the value parse() would give. The integer index templates and the discarded chaining of operator[] stay. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
__forceinline makes MSVC report C4714 for function templates it cannot inline (get<std::string>), an error under /WX. The forced inlining is only a performance hint, so use plain inline there. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Mark the includes that are needed by design (json.hpp, the macro unscope, <tuple> for the tuple_size specialization), take size_t from <cstring>, and drop the forward declaration of basic_json_document that the friend declaration makes redundant. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
ci_single_binaries runs IWYU with --error on every header. For json_view.hpp it suggested adding <array> (std::array is used), the headers that json.hpp already provides (abi_config, abi_macros, input_adapters, json_pointer, cpp_future, string_concat, value_t, json_fwd), and <version> for std::nullptr_t, and removing <cstddef>. - include <array> - keep <cstddef> (nullptr_t, size_t; IWYU attributes them to <version> and <cstring>) - tell IWYU not to suggest the headers that json.hpp provides: the amalgamated json_view.hpp only includes json.hpp, so including them here would duplicate their definitions Signed-off-by: Niels Lohmann <mail@nlohmann.me>
The same changes as on the dump branch, where they were first made, so that this branch passes clang-tidy on its own. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
… view Develop's #5800 made basic_json report a JSON pointer array index such as "/a" as parse_error.109 instead of out_of_range.404, as longer ones were; the view's array_index() follows, with "-" and the empty token still unresolved. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
426dec2 to
f2582da
Compare
From gregmarr's review of #5622: - count() counts every member with the key, so count(key) > 1 detects duplicate keys; it was contains() as 0/1 before. - get_to() passes types that get<T>() does not convert directly to from_json with the existing value, as basic_json::get_to() does, so that a from_json which sets only the fields present keeps the others. - array_index() checks the empty token and "-" first; the results are unchanged. - Docs: no "if an exception is thrown" where a view never changes anything; the borrowed source must also stay unmodified; items() on a temporary view is safe; back()/front() notes shortened; the basic_json see-also entries say how the view's results can differ. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
Part of the stack for the zero-copy view (#5295). This adds element and member access to
basic_json_view, and the value conversions built on it.Summary
basic_json_viewgains the read-only access functions ofbasic_json:operator[]andat()with keys, indices, and JSON pointers,front()andback(),find(),contains(), andcount(),begin()/end()andcbegin()/cend(),items()with structured bindings from C++17 on, andtype_name().operator[]andat()accept an index of any integer type (int,unsigned,long,std::int64_t, ...) through a template; a singleintoverload would have been ambiguous forunsigned,long, andint64_t. Negative values are out of range.basic_json. Wherebasic_jsonhas undefined behavior, the view answers safely:operator[]with a missing key or an index out of range returns a discarded view;front()/back()of an empty container throwinvalid_iterator.214.operator[](key, index, JSON pointer) on a discarded view returns a discarded view, sov["a"]["b"][0]can be tested once at the end.at()still throws.parse()keeps the last value, and so domaterialize()and the conversions to maps. This is documented prominently in a "Duplicate keys" section of the feature page, with a recipe for getting the last member. Returning the last member would mean scanning every object to its end on each lookup, which made lookups in small objects 1.6 to 3.4 times slower.operator[](index)is linear in the index, because elements are not fixed-size in the flat index. The docs say so.get<T>(),get_to(): convert directly, without building abasic_jsonvalue: arithmetic types (booleans become 0/1, as inbasic_json);boolandstd::nullptr_t; strings (std::basic_stringas a copy,string_view_twithout a copy);BasicJsonType(the same asmaterialize()); views themselves;std::vector, and maps with string keys, e.g.std::map<std::string, json_view>. Everything else, including user types withfrom_json(), goes throughmaterialize()of that subtree. Floats are bit-identical toparse().get_string(): the string without a copy, pointing into the source text or, for strings with escapes, into the document's buffer of decoded strings.number_token(): the number as it appears in the source ("1.50","1E2", all digits of a long integer) --basic_jsoncannot provide this.value(): with keys and with JSON pointers; the pointer overload also accepts arrays, as since Extend value to arrays when using JSON pointers #5223.operator[],at(), andcontains(). The exceptions are those ofbasic_json, message included; where constbasic_json::operator[]has undefined behavior, the result is a discarded view;value()returns the default whereverbasic_json::value()catchesout_of_range;contains()never throws; array indices followjson_pointer's rules (parse_error.106/109,out_of_range.404/410).json.hpp,json_pointer.hpp, andvalue_t.hppgain two small internal helpers.detail::json_pointer_accessgives code outsidebasic_jsonthe reference tokens of a pointer, so they need not be parsed fromto_string()again.detail::value_type_name(value_t)holds the names thatbasic_json::type_name()used to switch over;type_name()now calls it, so the view uses the same names (no behavior change).NLOHMANN_VIEW_ALWAYS_INLINEis plaininlineon MSVC, because__forceinlinetriggers C4714 (an error under/WX) for templates MSVC cannot inline. IWYU pragmas are added. Docs: API pages and examples for every new function, the "Duplicate keys" and "Getting values out without copying" sections, and fixes of theoperator[]andis_discardedpages.Performance
Measured on Apple M1 Max (Apple clang,
-O2) and an x86-64 KVM Haswell VPS pinned to one core (GCC 13 / Clang 18,-O2; the VPS is noisy, about ±5-10%). µs, best of 3 interleaved rounds of 15 runs; files from nativejson-benchmark. Traversing every value (numbers read asdouble, strings copied) at this PR:get<T>()of arithmetic types is inlined down to the conversion, so its checks of the node kind merge with those of the caller, and reading an integer needs no call.The core library is unaffected: the three core files only gain the helpers above, and
basic_json::type_name()returns the same strings.Tests
ordered_json.basic_json.parse()fordoubleand forfloat.at(),operator[],contains(), andvalue()compared withbasic_json, also withJSON_DIAGNOSTICSandJSON_DIAGNOSTIC_POSITIONS.Public API
No breaking changes to the existing API: this PR adds members of
basic_json_view(access functions and value conversions), plus the internaldetail::json_pointer_accessanddetail::value_type_name().basic_json::type_name()behaves as before. The view API is new in this stack, so its decisions are not breaking relative to a release, but they are visible to users:operator[]andat()accept any integer type as an index.operator[]on a missing key, an out-of-range index, an unresolvable pointer, or a discarded view returns a discarded view instead of invoking undefined behavior as constbasic_json::operator[]does;at()throws.parse(),materialize(), and map conversions keep the last.JSON_DIAGNOSTICSpath.Written by Claude Code.
🤖 Generated with Claude Code