Fix #1680: overridable escape-scan fast path for JSON generators

[pjfanning]

Alternative/extension to #1681 (by @franz1981), targeting 3.x since it adds new protected API.

Extracts the ASCII fast-scan loops into overridable protected methods, one virtual call per segment (per-character hooks would defeat the optimization once any override is loaded):

• UTF8JsonGenerator._writeUnescapedAscii(char[]/String, ...) — scan+copy, used by both _writeStringSegment fast loops
• WriterBasedJsonGenerator._findFirstToEscape(char[], int, int) — scan only, used by _writeString2, _writeSegment, _writeString(char[],...)

Default implementations include #1681's loop-invariant specialization: when _outputEscapes == CharTypes.get7BitOutputEscapes() the loop tests ch < 0x20 || ch > 0x7F || ch == '"' || ch == '\\' directly (no table load; C2 unswitches on the invariant). Custom CharacterEscapes, ESCAPE_FORWARD_SLASHES and alternate quote chars keep the original table lookup, so output is unchanged everywhere. Sub-classes with statically known escaping rules can override with a simple || check of their own.

Relative to #1681: also covers WriterBasedJsonGenerator, and exposes the extension point; happy to rebase on top of #1681 if that lands in 3.1 first.

Fixes #1680

🤖 Generated with Claude Code

[pjfanning]

@franz1981 Claude AI suggested this as it keeps the configurability of the escapes but if you keep the standard escapes, it uses the if statement that you suggested. I'll see if I can get jmh numbers.

[pjfanning]

I got Claude AI to do a quick benchmark for the utf8 generator and it says that for Java 17, there is little gain. There is a measurable gain in Java 21 (8-15% when the hardcoded if kicks in).

[franz1981]

Ran #1682 against #1681 on the reproducer at https://github.com/franz1981/c2-writestring-spill-jmh
(SingleBench.serialize, JDK 25, jackson-core 3.1.5, 3 forks, 5x5 iterations, pinned). The two are
indistinguishable:

	clean	String.charAt profile polluted
stock	352.6 ± 3.7	443.8 ± 4.7
#1681	341.1 ± 2.3	350.6 ± 11.8
#1682	342.3 ± 5.2	354.4 ± 6.9

With a single implementation loaded the virtual call is devirtualized and inlined, so #1682 compiles
to the same loop - it costs nothing and covers WriterBasedJsonGenerator too, so it looks like the
better vehicle to me.

One thing I did not measure: the bimorphic case. Once a subclass override is loaded the call site is
no longer monomorphic and the call becomes real. It is per segment rather than per character so it
should be cheap, but it is the case the extension point exists for and it is currently unmeasured.

[pjfanning]

@franz1981 I checked and the settable character escapes will need to be retained.

On JsonFactoryBuilder:

1. characterEscapes(CharacterEscapes) — fully custom: the user's getEscapeCodesForAscii() supplies the whole 128-entry table, arbitrary contents.
2. quoteChar(char) — e.g. apostrophe instead of " changes which table get7BitOutputEscapes(quoteChar, ...) returns.
3. JsonWriteFeature.ESCAPE_FORWARD_SLASHES — switches to the sOutputEscapes128WithSlash variant.

And despite factories being immutable in 3.x, setCharacterEscapes() is still a mutable setter on the generator itself — it can be called mid-stream (the JsonGeneratorBase._characterEscapes field comment says it can't be final because of JSONP-style use cases). So the table can even change during a generator's lifetime.

[franz1981]

@pjfanning that spread may be profile state rather than JDK version.

The size of the win depends on whether String.charAt has ever seen a UTF-16 String in that JVM.
Its branch profile is a single process-wide MethodData, and C2 prunes the isLatin1 branch only at a
zero UTF-16 count - so one UTF-16 charAt anywhere leaves a cold StringUTF16.charAt call in every
ASCII charAt loop compiled afterwards, which disables unrolling and range check elimination.

On the reproducer (jackson-core 3.1.5, JDK 25, SingleBench.serialize, 3 forks, pinned):

	stock	with this change
no UTF-16 charAt anywhere in the JVM	352.6 ± 3.7	341.1 ± 2.3 (~3%)
after a single UTF-16 charAt in setup	443.8 ± 4.7	350.6 ± 11.8 (~21%)

A benchmark that only ever writes ASCII keeps the profile clean and so measures the small number; a
real application usually does not. So it may be worth checking whether your 17 and 21 runs differ in
that rather than in the JIT - this is a guess, but it is what we measure here.

FasterXML/jackson-databind#6182 is one common way it gets polluted: StdDateFormat's static
initializer builds a SimpleDateFormat, which calls String.charAt on the locale per-mille sign, so
merely constructing an ObjectMapper is enough.

[franz1981]

@pjfanning if you are converging to an hybrid solution which can still do the best of the 2 worlds I am happy to close my PR ❤️

---

Some numbers on how the two changes interact, in case it's useful for the discussion.

Setup: a real Quarkus app doing reflection-free Jackson serialization (GET returning a list of beans), Quarkus main + Jackson 3.1.4, Temurin 25.0.2, 40s @ 100 connections, app pinned to 2 cores. Each configuration run 3+ times, interleaved. I measured the escape-table removal using #1681; this PR takes the same approach.

configuration	req/s	vs previous
neither fix	140,073 (sd 485, n=3)	—
databind #6182 only	147,246 (sd 2,494, n=6)	+5.1%
#6182 + escape-table removal	151,154 (sd 2,177, n=3)	+2.7%

Total +7.9%.

The C2 assembly explains why. Numbers below are for the ASCII copy loop as it appears in the method that actually runs -- the generated serializer, which inlines writeString (in several places, so there are multiple copies of the loop; the figures are the same in each):

configuration	chars per iteration	stack accesses per char	escCodes[] loads per char	instructions per char
neither	1	15	1	33
#6182 only	2	0	1	16
both	4	0	0	13.8

Without #6182 the loop is not unrolled and is badly register-starved: 15 stack accesses to emit one character, with the String object and the index reloaded from the stack on every iteration. The cause is that String.charAt has a single process-wide MethodData, and one UTF-16 call from StdDateFormat's <clinit> leaves a non-zero count on its non-LATIN1 branch. The LATIN1 case is still inlined to a plain byte load -- the hot path is fine -- but C2 can no longer prune the other branch, so it keeps an out-of-line copy of it. That path is never executed for ASCII data and costs no profiler samples; the damage is that C2 will not unroll a loop containing a call, and the allocator must keep values live across it. Fixing the <clinit> makes the spilling disappear entirely and the loop unrolls to 2 chars/iteration.

Removing the escape-table lookup is a separate win on top: it takes the per-character escCodes[] load and its bounds check out of the loop, and the loop unrolls to 4 chars/iteration.

So the escape-table change doesn't look redundant once the profile pollution is fixed -- the two address different costs, and the +2.7% is measured with #6182 already applied.

Standalone JMH reproducer with the same three configurations, on both Jackson 2.22.0 and 3.1.5: https://github.com/franz1981/c2-writestring-spill-jmh

The compiled copy loop there is identical in the two Jackson versions: 1 char/iteration and 13 stack accesses per character with neither fix, 2 chars/iteration and 2 with the databind fix, 4 chars/iteration and no table load with both. On the inlined benchmark #6183 is worth ~20% and #1681 a further 3-5%, and the not-inlined control never regresses.

[franz1981]

FYI @pjfanning #1680 (comment) is further explaining how this PR interact with the already merged change which pollute all String::charAt calls and where the performance diff is more visible.
In addition to that I have proposed there an alternative approach using a static final table which deliver the same or better performance than this and is more similar to the existing approach although requires to manually "unswitch" the loop, but is necessary to allow the JIT to access the lookup table without any bound check and allowing 4x loop unrolling

[pjfanning]

@franz1981 feel free to update your PR or create a new one and include the changes from here if they help.
We can't lose support for configurable output escapes but we can do fixes like this that optimise the code if certain output escapes are used. If this PR is ok as is, I can spend more time benchmarking it.

[franz1981]

The overall benefits between this or the other approach I have suggested is still minimal, and very CPU dependent at this point. so, feel free to go for it - if not happy I will push the alternative one, if you spot any regression with this one.

Just remember to try both with and without my other change in databind as it will impact the chance that any unrolling would happen 🙏

I can share a specific benchmark with custom serializers if you want which is removing reflection out of the equation and create the inlining condition where this PR deliver a very big benefit

[github-actions]

📈 Overall Code Coverage

Metric	Coverage	Change
Instructions	84.31%	📈 +0.020%
Branches	77.53%	📈 +0.050%

Overall project coverage from JaCoCo test results. Change values compare against the latest base branch build.

[pjfanning]

JMH results comparing this PR's UTF8JsonGenerator / WriterBasedJsonGenerator (commit 41b264e, copied verbatim into a benchmark project as renamed classes) against the current 3.3.0-SNAPSHOT (2026-09-12 build, does not contain this change).

Benchmark: writeString(String) × 100 values into an array, standard escape settings, no CharacterEscapes. Content types:

• ascii-short: 8–32 char ASCII, nothing to escape
• ascii-long: 512–1024 char ASCII, nothing to escape
• ascii-escapes: 512–1024 char ASCII with ", \ or \n roughly every 40 chars
• unicode: 512–1024 char ASCII with a non-ASCII char roughly every 40 chars

JDK 17.0.19, JMH 1.36, throughput (ops/ms, higher is better), 2 forks × 10 iterations, error is 99.9% CI. Old Intel i7-4770HQ MacBook, so absolute numbers are low; look at the relative delta.

Generator	content	3.3.0-SNAPSHOT	PR #1682	Δ
UTF8	ascii-short	210.1 ± 3.7	220.6 ± 3.3	+5.0%
UTF8	ascii-long	7.36 ± 0.32	7.91 ± 0.07	+7.4%
UTF8	ascii-escapes	6.20 ± 0.06	6.28 ± 0.19	+1.2% (within error)
UTF8	unicode	3.96 ± 0.04	3.92 ± 0.03	−1.0% (within error)
Writer	ascii-short	178.4 ± 2.6	196.8 ± 1.7	+10.3%
Writer	ascii-long	7.13 ± 0.05	7.92 ± 0.03	+11.0%
Writer	ascii-escapes	4.27 ± 0.06	5.23 ± 0.03	+22.6%
Writer	unicode	7.14 ± 0.07	8.77 ± 0.04	+22.7%

Takeaways:

• UTF8JsonGenerator: 5–7% faster on pure-ASCII content (where the fast loop dominates). No measurable change when escapes or non-ASCII break the loop frequently, as expected since those paths are untouched.
• WriterBasedJsonGenerator: 10–11% faster on pure ASCII and ~22% faster on the escape/unicode cases. The larger gain there is presumably from the restructured _writeString2 / _writeSegment loops (no per-iteration _outputTail field write, and the c < escLen bound check is gone for the standard-escapes case, so non-ASCII chars stay in the fast loop).
• No regressions observed in any configuration.

Output was verified byte-identical between the two implementations for random mixed content (ASCII, control chars, quotes, backslashes, 2/3/4-byte UTF-8) via writeString(String), writeString(char[],..) and writeName.

Benchmark source: https://github.com/pjfanning/double-reader-writer (JsonStringGeneratorBenchmark).

[franz1981]

@pjfanning it is with FasterXML/jackson-databind#6183 in? Or without?

[pjfanning]

@pjfanning it is with FasterXML/jackson-databind#6183 in? Or without?

my testing and microbenchmarks are with jackson-core only

[franz1981]

Thanks — then those are all clean-profile numbers. With jackson-core only, String.charAt never sees a UTF-16 String in those forks (JMH forks per param set, so the unicode set doesn't pollute the ascii-* forks either), and that's the state where the escape-table change matters least (~3% on my reproducer). It's also not the state most users are in: on 3.x without FasterXML/jackson-databind#6183, just constructing an ObjectMapper pollutes String.charAt via StdDateFormat.<clinit>, and the ASCII loop compiles very differently (no unrolling, the out-of-line StringUTF16.charAt call kept inside the loop, spills).

Could you add a @Param to the benchmark that forces the pollution, so both states show up in the same run? Something like:

@Param({"false", "true"})
public boolean pollutedCharAtProfile;

@Setup(Level.Trial)
public void pollute() {
    if (pollutedCharAtProfile) {
        // a single call is enough: String.charAt has one process-wide
        // MethodData, and C2 prunes the non-LATIN1 branch only at a
        // zero count - that's what makes profile pollution so scary
        if ("\u2030".charAt(0) != '\u2030') {
            throw new AssertionError();
        }
    }
}

Level.Trial runs once per fork before any measurement, so true reproduces what new ObjectMapper() does today and false is the post-#6183 state.

[pjfanning]

@franz1981 I see the effect that the 'pollution' has on String.charAt - maybe we need to try to remove any code that uses String.charAt. jackson-databind#6183 only removes one way that the 'pollution' happens.

[franz1981]

It's true @pjfanning and I had the same impression as well; that said if you need to iterate an unknown relatively long String object to know what could escape, I cannot see how to do it without a double iteration (copy into the recycler chars and iterate over them) which seems an overkill for larger strings which can escape :(

The problem is on the JIT and I am working with the IBM openJDK to provide a fix for that ie. by making the JIT able to trust the coder field (which state the asciness of the string) and hoist it out of the loop allowing the loop to be duplicated instead of containing the utf16 (maybe untaken!) read char call which prevent loop unrolling to happen.
In any case this patch still allows, for the happy path, a value as it increase the loop unrolling by simplifying the body loop, from a JIT pov