Skip to content

Keep the field order in Json.newBuilder - #1482

Open
linyiru wants to merge 1 commit into
playframework:mainfrom
linyiru:fix/json-newbuilder-order
Open

linyiru wants to merge 1 commit into
playframework:mainfrom
linyiru:fix/json-newbuilder-order

Conversation

@linyiru

@linyiru linyiru commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Pull Request Checklist

  • Have you read through the contributor guidelines?
  • Have you squashed your commits?
  • Have you added copyright headers to new files? (no new files)
  • Have you updated the documentation? (no API change; the documented example has 2 fields and is unaffected)
  • Have you added tests for any changed functionality?

Purpose

Json.newBuilder collects the fields into Map.newBuilder[String, JsValue]. Above four fields that map is a Scala HashMap, so the resulting JsObject loses the insertion order. JsObject.apply, Json.obj and Json.parse all keep the order. On main (Scala 3.3.8):

val b = Json.newBuilder
(1 to 8).foreach(i => b += (s"f$i" -> i))
Json.stringify(b.result())
// {"f7":7,"f6":6,"f1":1,"f8":8,"f5":5,"f3":3,"f4":4,"f2":2}

The existing test uses 2 fields, and JsObject equality ignores order, so nothing caught this.

The builder now collects into ImmutableLinkedHashMap.newBuilder, the same map JsObject.apply uses. The result keeps the order, and result() wraps the accumulated map without the second copy (Map → JsObject) it made before.

Background Context

ImmutableLinkedHashMap.newBuilder (2.13+) handed its LinkedHashMap to the result and kept mutating it. Its internal uses build once, so that was safe there. A public builder can be reused, though, and then += or clear() after result() would have changed an object that was already returned. The builder now copies the map on the first change after result(), like Scala's own map builders do. So reuse behaves as before: adding after result() keeps the earlier fields, and the earlier result is untouched. It also reports knownSize, which JsObjectBuilder forwards. The 2.12 builder already buffers and copies on every result(), so it needed no change there.

JsObjectBuilder also gets append(key, value), which adds a field without the caller allocating a tuple (per review, #1482 (comment)). It saves 16 B per field and is about 10–20% faster than += (JMH, JDK 17, Scala 2.13; numbers in the thread). The builder stays private[json], so Json.newBuilder callers cannot reach it yet; exposing it is left for later.

Memory: an object from the builder now takes exactly what JsObject(fields) takes, for 1 to 20 fields (new JsonMemoryFootprintSpec case). For 1 to 4 fields that is more than before: those fit Scala's specialised Map1–Map4 (e.g. 40 vs 160 bytes for one field). I think matching the other constructors and the parser is the right trade, but I'm noting it.

Checked locally (Docker, JDK 21): validateCode, +play-jsonJVM/testFull, +mimaReportBinaryIssues, docs/testFull, +play-jsonJS/Test/compile, +play-jsonNative/Test/compile. With the tests only and the old implementation, the order and footprint tests fail as expected. After adding append: validateCode, +play-jsonJVM/testOnly play.api.libs.json.JsonSharedSpec and +mimaReportBinaryIssues.

3.0.x has the same JsObjectBuilder. I'm happy to open a backport if you want one.

References

Found while replying to the review of #1479 (#1479 (comment)). The builder was added in #876.

AI disclosure: I directed and verified this work; Claude Code (Claude Opus) assisted with the investigation, the patch, tests and this description.

// Keeps the fields in insertion order, as JsObject.apply does.
private val fs = ImmutableLinkedHashMap.newBuilder[String, JsValue]

def +=(elem: (String, Json.JsValueWrapper)): this.type = {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

def append(key: String, v: Json.JsValueWrapper): this.type when optimization without tuple is wanted?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, and I measured it to check (JMH, JDK 17, Scala 2.13, -prof gc, building an object field by field):

fields += (this PR) append
4 12,706 ops/ms · 480 B 15,678 ops/ms · 400 B
16 3,966 ops/ms · 1,376 B 5,040 ops/ms · 1,104 B
64 996 ops/ms · 4,864 B 1,200 ops/ms · 3,824 B

So append saves 16 B per field and is about 10–20% faster, even after escape analysis. (Removing the second tuple that += builds internally saves nothing once C2 has warmed up, so I left that alone.)

One catch: Json.newBuilder returns MBuilder[(String, JsValueWrapper), JsObject] and JsObjectBuilder is private[json], so callers can't reach append today. Narrowing the return type breaks binary compatibility, so it would need either a new public entry point now or a change in the next major. Since either way it's an API addition, I'd like to keep this PR to the ordering fix and do append in a follow-up. Which would you prefer?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me, the API compatibility about append can be considered later.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added append to JsObjectBuilder (2.12 and 2.13+) in c1b4710, with += delegating to it. The builder stays private[json]; exposing it is left for later as you suggest. Re-running the benchmark on this exact implementation gives the same numbers as above.

@linyiru linyiru mentioned this pull request Oct 7, 2026
5 tasks done
JsObjectBuilder accumulated into Map.newBuilder, so above four fields the
result was backed by a Scala HashMap and lost the insertion order that
JsObject.apply and Json.parse keep. Accumulate into
ImmutableLinkedHashMap.newBuilder instead: the result keeps the order and
takes the same memory as JsObject(fields), with no second copy.

That builder handed its LinkedHashMap to the result and kept mutating it,
which was fine for its internal one-shot uses but not for a public builder
that can be reused. Copy the map on the first change after result(), as
Scala's own map builders do, and report knownSize.

Also add JsObjectBuilder.append(key, value), which adds a field without
the caller allocating a tuple. The builder stays private[json]; exposing
it is left for later.
@linyiru
linyiru force-pushed the fix/json-newbuilder-order branch from bbc6e45 to c1b4710 Compare October 7, 2026 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants