Repository navigation
Blackbird rewrite: ClassFile generation of unrolled codec class, optimize record / builder / field - #359
Conversation
|
Ok: is JDK-25 hard requirement for building/releasing Blackbird module? JDK 21 already needed for building jackson-core 3.1+, everything else built with 17. EDIT: yes, should have read properly; Blackbird does rely on JDK 25 after this rewrite. |
Code review (Claude Code)I found 10 issues and one minor note, each checked against stock jackson-databind behavior. Nothing was run, and none of these cases is covered by the existing blackbird tests. High
Medium
Low
SummaryThe most urgent are #10, which affects every user below Java 25, and #1–2, which lose data or fail on reads that work today. Most of the rest come from the generated code skipping things stock 🤖 Generated with Claude Code |
|
@cowtowncoder Thank you for the review, I will do another iteration on this soon to address the findings. That said, the last Blackbird release still works same as it always has - such users could pin it, at least for now until Jackson API drifts too far. (I expect not many changes will hit 2.x at least at this point?). On top of that, with modern reflection and Jackson improvements, the classic Blackbird hardly wins in a meaningful way anymore. I have never run the Jackson release process so I want to make very sure I will not break anything for you. The goal here is that only the build JDK must be 25, all other modules still run fine with their existing 17 baseline, but I am not 100% sure how to test every path you use. I am of course always excited to "ship it" but I do not want to push anything too fast or break everything all at once. I see the risk as manageable since there is always the "bug found, disable blackbird for now" break-glass option. Regarding 3.3.0 vs 3.4, I will accept your judgement as wise and all-powerful maintainer :) I will try to remain responsive to your concerns as we decide whether to merge before 3.3 is cut or sometime later. Once I address feedback, my next step will be to run this in test and then production on our own platform, and I will report the experience back here. |
|
@stevenschlansker Excellent, sounds like a plan -- we are in full agreement here. Performance improvement improvement looks really good so looking forward to getting this merged. |
The next commits replace the Blackbird engine with code generators that use the ClassFile API (JEP 484, final in Java 24). Blackbird requires Java 25, the current LTS release. Set up the build for that first: - Pin the reactor to --release 17. The parent chain sets only source and target, so a build on a newer JDK could link against newer APIs. - Build blackbird and blackbird-tests only on JDK 25 or later (profile blackbird-jdk25). The other modules still build on JDK 17. - Fail a release build (-Prelease) on an older JDK, so that a release cannot leave out jackson-module-blackbird without an error. - Lay out blackbird as a multi-release jar: src/main/java25 compiles at release 25 into META-INF/versions/25, and all other classes stay at release 17. The directory is empty until the generators arrive. - Declare osgi.ee JavaSE 17 explicitly. bnd derives it from the newest class file, and bnd 6 maps class file 69 to UNKNOWN, which no framework resolves. - Run the release build of CI on JDK 25, so the deployed reactor has Blackbird.
…ders The LambdaMetafactory engine replaced individual property setters inside the stock BeanDeserializer loop. The new engine replaces the loop: for each eligible bean it emits one hidden class with the ClassFile API, which matches names with the parser's PropertyNameMatcher (the fused nextNameMatchAndToken call), dispatches on the match index with a tableswitch, reads String, int, long and boolean values with direct parser calls, stores through constant MethodHandles, and calls the codec of a nested bean directly. This commit covers POJOs that stock databind builds with its default creator. Records and builder-based beans keep the stock deserializer until the next commit. Design: - The codec is a stock BeanDeserializer. GeneratedReaderBase extends it and is a copy of the resolved stock deserializer, and the generated class overrides only deserialize(). Inputs the loop does not model go to the stock implementation of the same instance. Other modifiers, which often cast to BeanDeserializer, and the instanceof checks of databind see the type they expect. A placeholder (also a BeanDeserializer) stands in from the modifier to resolution, because generation needs the resolved properties. - One rule decides what the codec handles: a bean or property gets generated code only where that code does what stock does. Everything else keeps the stock deserializer (a gate) or the stock SettableBeanProperty inside the codec (a STOCK arm): Object Ids, non-standard creation (creators, external type ids, unwrapped properties), merge, polymorphic values, custom deserializers, property wrappers such as managed references, final fields. Ignorals, aliases and case-insensitive matching come from the resolved stock state. - Failures go through the stock helpers (wrapAndThrow, the instantiator failure handling, the stock property message), so exceptions and their reference paths match stock. - Member access uses handles unreflected with the public lookup after databind's own fixAccess, so Blackbird reaches exactly what stock can invoke and grants no access of its own. Codecs are defined in Blackbird's own module and unload with the mapper. - @JSONVIEW: a codec of a bean that declares views keeps view-active calls on the generated path through a per-view property bitmask. - A generated method over 7000 bytes is discarded, and the bean keeps the stock deserializer: HotSpot does not compile methods over 8000 bytes. - Generation failures log at DEBUG through System.Logger and leave the stock deserializer. The test suites set tools.jackson.module.blackbird.failOnCodegenError, so a generator bug fails the build. blackbird.debug.codegen traces the gate decisions and blackbird.debug.dumpDir writes the generated class files. The generator is in src/main/java25. On Java 17 to 24 it is not visible, and Blackbird cannot generate codecs. A separate service provider, internal.DiscoveredBlackbirdModule, makes module discovery (findAndAddModules) skip Blackbird there with one warning, so discovery still registers every other module. An explicit registration of BlackbirdModule throws IllegalStateException, also when discovery runs before or after it: the provider has the registration id of BlackbirdModule only on a runtime that generates codecs, so elsewhere it cannot replace the explicit registration. blackbird-jpms-tests adds a foreign named module that requires Blackbird and checks codec engagement for its package-private beans on the module path.
Records: the codec collects components into typed locals in canonical order and calls the canonical constructor through a constant handle. When the document leaves out a component, a primitive component reads a null, FAIL_ON_NULL_CREATOR_PROPERTIES is enabled, or the constructor throws, the stock PropertyBasedCreator completes the arguments (absent values such as Optional.empty(), required checks) and builds the record, so the result and the exception are the stock ones. Names that no component takes are handled or buffered and replayed like the stock property-based loop does. A record gets a codec only when databind selected the canonical constructor with plain creator properties and the record has no any-setter, injectables, external type id, unwrapped properties or delegating creator. Builders: the stock ValueInstantiator creates the builder, and setters apply through handles. A non-null fluent return becomes the new builder, and a null one keeps the current builder, as in stock deserializeSetAndReturn. The build method runs through a constant handle. GeneratedBuilderBase extends BuilderBasedDeserializer, for the same reason that GeneratedReaderBase extends BeanDeserializer. A failing build method goes through wrapInstantiationProblem, as in stock finishBuild.
The generated writer for a bean is a copy of the resolved stock BeanSerializer (GeneratedWriterBase) whose serialize() writes the properties in a straight line: pre-encoded name constants, getter or field reads through constant handles, direct writes of String, int, long and boolean values, and direct calls of the generated writer of a nested bean. When the bean itself is written with a type id, the generated serializeWithType writes the type wrapper the stock way. As on the reader side, everything the generated code does not reproduce exactly keeps stock code. A property keeps its stock PropertyWriter when it has a type serializer, an applied view, null or value suppression, a non-default null serializer, or a custom serializer. A bean keeps the stock serializer when it has Object Ids or a filter. Write failures go through the stock wrapAndThrow. A direct self-reference gets the stock "Direct self-reference leading to cycle" error, or a null with WRITE_SELF_REFERENCES_AS_NULL. A StackOverflowError from a deep or cyclic graph gets the stock "Infinite recursion" error. Scalar writes go through small per-kind helper methods in one generated class, WriteHelpers, that all writers share. The helpers are padded with nops to a size that neither HotSpot compiler inlines. On aarch64, C2 copies of writeName and the NumberOutput digit code inlined into writer bodies that hold list loops spill loop state to the stack. With the padding, RecordBenchmark.recordSer is 24-45% faster and PojoBenchmark.mediaItemSer 12-31% faster on aarch64 (JDK 25 and 27-ea), and BuilderBenchmark.builderSer 23% faster on x86_64 (JDK 25). A copy of the helpers in each writer class compiles once per class, and on x86_64 the extra machine code causes instruction-cache misses: the shared class makes recordSer 12% and mediaItemSer 7% faster. -XX:+PrintInlining confirms that the helpers stay out of line. With both sides generated, Blackbird no longer uses a caller lookup, and CrossLoaderAccess is unused. blackbird-tests now also runs the packaged jar in child JVMs: on the build JDK, and on each Java 17-24 home that -Dblackbird.test.oldJdks lists (CI passes 17 and 21). The runs cover module discovery on the class path and the module path, with a second module provider on the path, and explicit registration with discovery before it, after it and without it. Another test loads Jackson and Blackbird in a class loader that the system class loader cannot see, as an application server does.
Remove CrossLoaderAccess and its generator, the util package, and the moditect descriptor that the build no longer reads. Nothing in the module uses java.util.logging now, so the module no longer requires java.logging. Export only tools.jackson.module.blackbird from the OSGi bundle, as the JPMS descriptor does. The codecs are defined in Blackbird's own module, so no user code links to the internal packages. PackagingTest checks the module descriptor, the bundle exports, the release guard of the root POM, and that the member handles grant no access of their own.
Each scenario runs through a stock mapper and a Blackbird mapper built from the same configuration. The two must give the same value, or the same exception class and message with the same reference path. Then the scenario asserts which path Blackbird took: the generated codec, a stock property inside it, or the stock (de)serializer of the whole bean. The check looks at the (de)serializer that the mapper uses, not at the placeholder that a modifier sees. The scenarios cover polymorphic values and default typing, non-standard creation, property wrappers, ignoral sets, absent record components, construction, setter and build failures, builder setters that return null, null serializers, self-references, and beans whose generated methods would be too large for the JIT.
Build the benchmarks against the current version, and select the engine with a mode parameter (stock, blackbird, afterburner) instead of one subclass per engine. A classic-blackbird profile builds the same benchmarks against Blackbird 3.2.2, the last release of the LambdaMetafactory engine, for comparison. The uber jar is a multi-release jar, so that Blackbird sees its release-25 generators. Add benchmarks for records, builder-based beans, beans with 40 and 200 properties, and the cold start of 500 types in a fresh JVM. Remove the JDK 14 result files of the old engine.
Rewrite the Blackbird README for the generated codecs: requirements (Java 25 at run time, the behavior on Java 17-24, JDK 25 for the build, Multi-Release: true in an uber jar), what the codecs handle and where Blackbird keeps stock code, debugging switches, and benchmark results against stock databind and Blackbird 3.2.2. Add release notes that also list the removed public types, and update the POM descriptions and the blackbird-tests README.
blackbird-tests/corpus runs the whole jackson-databind test suite with BlackbirdModule registered in every mapper, and compares the failures with a list of expected failures. The hook is a 10-line patch to the ObjectMapper(MapperBuilder) constructor of databind, active only with -Dblackbird.corpus=true. The runner passes the Blackbird jar, the module exports and the strict code generation switch to the databind build on the Maven command line, and changes no POM. The build does not run the corpus: databind publishes no test jar, and a CI job that patches databind main code breaks when databind changes that constructor. Maintainers run it manually before they submit a Blackbird change. The blackbird-tests README says how. With databind f75dc9039fe0 the only failures are 7 tests that assert the exact list or number of registered modules, which the harness changes.
e569e65 to
1d91be5
Compare
|
I refreshed the PR with fixes to all problems reported so far. Working on production testing now (have to upgrade our app to Jackson 3 first...) |
|
The new refresh also includes some manual tests (not hooked up to CI currently) to re-run the full Jackson test suite with Blackbird force-installed. This helps makes sure we do not break any core behavior. |
Unreflected varargs members yield variable-arity handles, whose asType to an erased (Object) trailing parameter builds a collector that casts the passed array to its element type. Records with a varargs canonical constructor therefore always failed with ClassCastException on the generated path. Make constructor, setter and builder-setter handles fixed-arity before adapting them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Class-data entries are loaded through condy constants named after JSON property names, and DynamicConstantDesc validates those as member names, which also forbids '<' and '>'. A property such as "a<b>" made code generation fail, so the bean silently lost its generated reader and writer. Replace both characters along with '.', ';', '[' and '/'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Going through this again with CC; finding & fixing some issues. |
With an active view, generated writers (MASK strategy) skipped hidden properties outright. Stock FilteredBeanPropertyWriter passes a property hidden by its declared views to serializeAsOmittedProperty, which calls writeOmittedProperty on generators that cannot omit properties (CSV), so generated output differed from stock there. Do the same for properties with declared views; view-less properties excluded by DEFAULT_VIEW_INCLUSION still write nothing, as in stock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codec class names are derived from the bean's class name, and a hidden bean class is named "Foo/0x...". '/' is not allowed in a binary name, so ClassDesc.of rejected it and such beans lost their generated reader and writer. Replace '/' with '$'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tryGenerate in both factories caught Throwable, so OutOfMemoryError or StackOverflowError while generating a codec was logged at DEBUG and the bean silently fell back to stock. Rethrow VirtualMachineError; other failures still demote as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bean's (de)serializer for good, paying a volatile read and an extra dispatch per call. Contextualization now hands out stock instead: the original serializer on the writer side (it shares the placeholder's property writers, which resolution fills in), and an exact-class BeanDeserializer copy of the resolved state on the reader side. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(de)serializers by class-name strings, so a databind rename would silently turn them all off. Compare against the class literals, which are public in exported packages, so the coupling is compile-checked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ontext The builder codec captured the parser's stream context at entry to report the failing property name as stock does. After a buffered type id the codec runs over the buffer and then the live stream, so that context was stale. Each arm now reads the input name itself (only for beans with aliases or case-insensitive matching), and the entry token kind is recorded at the first match; the name is resolved on the failure path only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings FasterXML#366, which patched the LambdaMetafactory engine this branch replaced: keep this branch's engine (CreatorOptimizer and ReflectionHack stay removed) and its ChildClassloaderTest, and drop TestReflectionHack. Port the FasterXML#366 tests to the generated-codec engine, which handles every scenario: assert generated readers and writers instead of per-property classes, and drop the old engine's skip-logging test. Lookups are no longer read, so drop the lookup helpers the 3.x port added and run those tests with and without module reads instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@cowtowncoder The two JDK17-24 cases are split in the PR as it stands:
Does this seem acceptable to you? I think it is an okay compromise, and am open to changing it to something you like, or in response to a future bug report. |
That sounds like a very good way to do it! Let's start with that. I will try to get this PR merged today; only (?) complication is concurrent 2.x work I am merging in first to 2.x then forward. But 1 of 3 worked fine. |
Brings in the 2.x port of generated builder setters (FasterXML#364) for the old LambdaMetafactory engine. This branch's generated-codec engine already handles builders, so its BBDeserializerModifier is kept and the Settable*Property classes stay deleted. The new builder tests are adapted to it: they check for a generated reader instead of optimized properties, drop the cases that construct the removed property classes, and expect the List property (a stock arm) to go through reflection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since db7183e, the static initializer of ClassFileEmitters initializes the release-25 emitters and lets their errors propagate. Module discovery first uses ClassFileEmitters in DiscoveredBlackbirdModule.getRegistrationId(). Thus a LinkageError in an emitter initializer made findAndAddModules() fail for every module, and each later call failed with "Could not initialize class ClassFileEmitters". A jackson-core without tools.jackson.core.type.WritableTypeId shows the failure on JDK 25. BeanWriterGenerator loads that class in its initializer, and stock databind does not load it for plain beans. No released jackson-core or jackson-databind 3.x (3.0.4, 3.1.8 and 3.2.4 checked) lacks a class that the emitter initializers load. A later release that moves or removes one of them would cause the failure. A LinkageError from the initialization now makes the emitters unavailable, as on Java 24. Discovery logs one warning and registers the other modules. An explicit registration throws IllegalStateException. The warning and the exception name the error and attach it. A VirtualMachineError still propagates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
I fixed one regression with the recent changes around LinkageError which was pointed out in an earlier review round, and added a test to ensure no further breaks. Updated PR description and unless you find any more review feedback I think this is good to go! |
Brings in FasterXML#367 (don't retry failed bean setters through reflection) for the old LambdaMetafactory engine. Its Settable*Property classes stay deleted here. The new TestSetterFailures checks for a generated reader instead of optimized properties, and drops the two tests that build the removed property classes by hand (and their helper class). TestAccessFallback is rewritten for the generated writers, which have no stock fallback on getter failure: it checks that a failing getter runs once and fails as plain databind does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… store - Generated POJO readers now return the bean, as stock's vanilla loop (_vanillaDeserializeWithUnknown) does, when a token that is neither a name nor END_OBJECT follows an unknown name; only when the copied stock state takes the vanilla loop (DEFAULT_VIEW_INCLUSION on) - emitScalar and emitChildArm emit only the stock set when a property has no generated store, instead of failing with NPE during generation - WideBeanTest restores a pre-existing blackbird.debug.dumpDir - Fix stale PkgBean comment on where the codec is defined Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…surefire - RecordReferring (the copy of stock BeanReferring) now drops a forward reference that resolved before construction when the record is null, as stock does after databind#6325; BBCodecStockChecksTest compares the two and is updated for the new stock outcome - failOnCodegenError is set through surefire systemPropertyVariables in blackbird, blackbird-tests and blackbird-jpms-tests, so every test class runs strict whatever loads first; the static blocks remain for IDE runs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…plan - Record codecs treat a token that is neither a name nor END_OBJECT as the end of the object and build the record, as stock's property-based loop does, instead of reporting it as unexpected - The "seen an unknown name" tracking for odd tokens is emitted only for POJO codecs whose stock deserializer takes its vanilla loop: the placeholders pass the resolved _vanillaProcessing into the plan, and the runtime _unexpectedAfterUnknown check is gone - BBCodecStockChecksTest compares stock and Blackbird for the null record with a resolved forward reference instead of hard-coding the outcome Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…g back - ReaderCodegen.generate fails generation when a non-STOCK arm outside record mode has no store handle, so a factory slip fails strict-mode tests instead of quietly taking the stock path - Drop the null-store guards from emitScalar and emitChildArm: the STOCK case of the arm switch is the only place a property goes stock - Store no longer carries builderMode; emitStore takes the same flag as the stock fallback - New ReaderPlanCheckTest covers the plan check Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Still finding things to improve upon although hoping to wind things down, get merged today if at all possible. |
… stock store errors - No codec is generated for array-shaped beans (reader, builder and writer placeholders): stock hands them to its as-array (de)serializer, which never calls a codec - Private builder classes are no longer gated: the codec never names the builder class - ReaderSupport.storeException calls stock _throwAsJacksonE through a probe instead of a copy of it - A stack overflow during generation is logged (debug trace and DEBUG log) once it has unwound - CodegenDebug.log(String) for constant messages; no lambdas in the CodegenFallbacks slow-path logging - New tests: ArrayShapeNoCodecTest, private builder in BBCodecEngagementTest Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ories - The StackOverflowError handlers in BBReaderFactory and BBWriterFactory only record the (de)serializer; the next generation logs it from a normal stack, so logging can no longer overflow again (possibly inside a class initializer) and escape tryGenerate - The array-shape check moves from the three placeholders into the factories' gates; its comment now states which stock paths still call back into the placeholder - CodegenFallbacks builds log messages only when DEBUG (or the codegen trace) is enabled Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No point in generation is known to have a shallow stack (databind drives the nested resolution), so even deferred logging could overflow again, fail an unrelated bean, or pin a class loader through the static slot. A generation that runs out of stack again just returns null and the bean keeps the stock path, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- With failOnCodegenError set (the test suites), a StackOverflowError during generation is rethrown instead of demoting the bean, since there it more likely means unbounded recursion in the generator. The check reads the property through its constant name, so it initializes no Blackbird class on the nearly exhausted stack - The tryGenerate trace runs after the try, so an overflow while tracing can no longer discard a codec that generated fine Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README's Debugging section and the CodegenFallbacks Javadoc now say that a generation running out of stack keeps the stock path without any report (it fails the test in strict mode), and how it shows in the codegen trace. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- README lists array-shaped beans (annotation or config override) among the beans that keep the stock deserializer and serializer - A builder without a build method now logs its own gate line, so a resolve line with no outcome in the codegen trace means only a stack overflow, as the README says - BBWriterFactory gets a gate(String) helper like BBReaderFactory's Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The StackOverflowError handler in BBReaderFactory and BBWriterFactory called Boolean.getBoolean, which can overflow again on the nearly exhausted stack and escape tryGenerate in production. Each factory now reads the property once at class initialization, so the handler only reads a static field. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodegenFallbacks.FAIL_ON_ERROR reads failOnCodegenError once, and both generationFailure and the factories' stack overflow handlers (through their own copy of it) use it, so strict mode is the same for every check in a JVM. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- ReaderCodegen and WriterCodegen rethrow RuntimeException and Error from the generated codec's constructor as they are, so a StackOverflowError reaches the silent overflow handler and an OutOfMemoryError is not demoted; only checked throwables are wrapped - The factories' FAIL_ON_STACK_OVERFLOW copy now documents why it exists: it initializes CodegenFallbacks early, so the overflow handler never runs that class's initializer on an exhausted stack - Test-base comments say when the IDE fallback setProperty takes effect - BBReaderOddTokenTest builds its baseline mapper from newVanillaJSONMapper() Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@stevenschlansker Phew! Finally got this merged, along with a few minor fixes to Afterburner/Blackbird and Now, as far as I could quickly test, observed performance improvement is still there. But if you have time and interest, it'd be great to verify that, before 3.3.0 release (hopefully later this week!). |
This PR replaces the Blackbird engine. The module coordinates, the
BlackbirdModuleAPI and the registration do not change. TheLambdaMetafactoryaccessors are gone. For each bean type, Blackbird now generates one hidden class per direction with the JDK ClassFile API (JEP 484), and installs it as the deserializer and the serializer of that bean.The branch starts with 9 commits on
3.x. Each of these commits compiles and passes its tests, so you can review the series one commit at a time:After these commits, the branch has follow-up commits from Tatu (review fixes in the generators and placeholders, the deprecation of the unused lookup hooks, and CI changes) and merges of
3.x.This version also addresses the review of 2026-09-23 and the
UnsupportedClassVersionErrorreport in FasterXML/jackson-future-ideas#108. I reply to each point separately.JDK requirement
Blackbird needs Java 25 or later at run time. The code generators use the ClassFile API (final in Java 24) and compile at release 25, the current LTS release. The other modules of this repository keep the Java 17 baseline.
The jar is a multi-release jar. The two generators (
BeanReaderGenerator,BeanWriterGenerator) compile at release 25 intoMETA-INF/versions/25. All other classes,module-infoand theServiceLoaderentry compile at release 17. On Java 17 to 24:MapperBuilder.findAndAddModules()) skips Blackbird with one logged warning, and it registers every other module. In the previous version of this PR, the class file 69 entry stopped theServiceLoaderloop withUnsupportedClassVersionErrorfor all modules.addModule(new BlackbirdModule())throwsIllegalStateException, with a message that names the Java version. It also throws whenfindAndAddModules()runs before or after the explicit registration.A BOM upgrade on Java 17 or 21 thus does not break an application that has Blackbird on its path. Users who need Blackbird acceleration on Java 17 to 24 can pin
jackson-module-blackbird3.2.x.The build needs JDK 25 for Blackbird. On an older JDK, the reactor leaves out
blackbird,blackbird-testsandblackbird-jpms-tests, and the other modules build as before.The release must be built on JDK 25. A release build on an older JDK would publish no Blackbird artifact. To prevent that, a
releaseprofile in the root POM has an enforcer rule for JDK[25,).maven-release-pluginpasses-Prelease(fromoss-parent), so a release on an older JDK fails. The CI release and deploy job now runs on JDK 25.Design notes
Stock base classes. A generated reader extends
GeneratedReaderBase, which extends stockBeanDeserializer(GeneratedBuilderBaseextendsBuilderBasedDeserializer, andGeneratedWriterBaseextendsBeanSerializer). The codec is a copy of the resolved stock (de)serializer, and the generated class overridesdeserialize()orserialize(). A writer also overridesserializeWithType()when the bean has no@JsonTypeIdproperty, and the codec of a bean that has views (as the resolved stock (de)serializer reports them) overrides_computeViewMask(). Each input that the generated loop does not model goes to the stock implementation of the same instance. Other modifiers, which often cast toBeanDeserializer, and theinstanceofchecks in databind see the type they expect. The placeholder that stands in from the modifier to resolution is also a stock subclass. When a bean stays on stock, the placeholder gives the stock (de)serializer to databind at contextualization, so that later calls do not go through the placeholder. This design also removedStockDeserializerProbe,StockSerializerProbeand theThreadLocalof the previous version. Three small nested probes remain for protected members:BBWriterFactory.WriterProbe(fields ofBeanPropertyWriter),GeneratedWriterBase.SelfReferenceProbe(the self-reference handling ofBeanPropertyWriter) andReaderSupport.InstantiatorProbe(StdValueInstantiator.rewrapCtorProblem).One rule for gates. The generated code handles a bean, or a property of it, only where that code does what stock databind does, and a differential test shows it. Everything else keeps stock code: the whole bean keeps the stock (de)serializer (a gate), or one property keeps its stock
SettableBeanPropertyorPropertyWriterinside the codec. This covers polymorphic values and default typing, custom null providers and null serializers, property wrappers (managed and back references, wrappers from other modifiers), external type ids, a delegating creator next to a default creator, and filtered writers that another modifier changed (a call with an active view then goes to stock). Decisions that stock databind already made (ignored and included names, aliases, case-insensitive matching, the selected creator, the build method) come from the resolved stock deserializer. Blackbird does not compute them again. For records, Blackbird builds its own name matcher, but it puts in only the names and aliases for which the stock lookup (PropertyBasedCreator.findCreatorProperty) finds the component. Failures go through the stock helpers (wrapAndThrow,wrapInstantiationProblem, the instantiator failure handling), so the exception class, the message and the reference path match stock, also for "Infinite recursion". A generated method over 7000 bytes is discarded: HotSpot does not compile methods over 8000 bytes.Discovery provider. The
ServiceLoaderentry isinternal.DiscoveredBlackbirdModule, a subclass ofBlackbirdModule. Only this class is lenient on Java 17 to 24, or when code generation cannot initialize on JDK 25 (for example, a jackson-core or jackson-databind of another version that lacks a class the generators load). The module can tell discovery from an explicit registration without a stack walk. Databind keeps one module for each registration id, the one added last. On Java 25, the provider has the registration id ofBlackbirdModule, so discovery plus an explicit registration installs Blackbird once. On Java 17 to 24, the provider has its own id, so it cannot replace an explicit registration, which must fail.Release guard. See above.
PackagingTestchecks that the root POM keeps the rule.Exports. The JPMS module and the OSGi bundle export only
tools.jackson.module.blackbird. The codecs are defined in Blackbird's own module (MethodHandles.lookup()of the generators), so no user module links to the internal packages. The bundle declaresosgi.eeJavaSE 17 explicitly, because bnd derives it from the newest class file and maps class file 69 toUNKNOWN. The bundle imports the ClassFile API packages as optional, so it also resolves on Java 17 to 24. The module no longer requiresjava.logging: generation failures log atDEBUGthroughSystem.Logger.Member access. Generated code reaches members through
MethodHandleconstants that Blackbird unreflects with the public lookup after databind ranfixAccess. Blackbird thus reaches exactly what stock databind can invoke, and grants no access of its own. A user-supplied lookup is not necessary any more. The twoBlackbirdModuleconstructors that take a lookup stay@Deprecated(since 3.1).findLookup(),findLookupSupplier()and_wrapWithJdkClassCheck()stay for source compatibility, but they are now@Deprecated, and module setup never calls them, so an override has no effect. Hidden classes are defined withoutClassOption.STRONG, so codecs unload with the mapper (#147).Shared, padded write helpers. Scalar writes go through small static helpers in one generated class,
WriteHelpers, that all writers share. The helpers are padded with nops pastFreqInlineSize(325 bytes, C2) andC1MaxInlineSize(35 bytes, C1), so that neither compiler inlines them intoserialize(). On aarch64, C2 copies ofwriteNameand theNumberOutputdigit code inlined into writers that hold list loops spill loop state to the stack. With the padding,RecordBenchmark.recordSeris 24-45% faster andPojoBenchmark.mediaItemSer12-31% faster on aarch64 (JDK 25 and 27-ea, 5 forks, padding on/off/on/off), andBuilderBenchmark.builderSer23% faster on x86_64. An earlier version of this PR emitted a copy of the helpers into each writer class. Each copy compiled separately, and on x86_64 the extra machine code caused instruction-cache misses (perfnorm: the same instruction count, 7% more cycles, IPC 3.7 against 4.0). The shared class makesrecordSer12% andmediaItemSer7% faster on x86_64, and it removes about 2 KB of nops from each writer class, which lowers the cold-start cost per type by about 0.1 ms.-XX:+PrintInliningconfirms that the helpers stay out of line. Graal uses other inlining rules. The comment atBeanWriterGenerator.INLINE_PADrecords this.Core and databind versions. Generated readers call
JsonParser.nextNameMatchAndToken. Stock databind 3.3 calls it too, so Blackbird needsjackson-coreandjackson-databindof the same version, as the README says.Tests
Run on JDK 25:
blackbird(523 tests): the existing suite, plus engagement, demotion and behavior tests per feature. The suite runs withtools.jackson.module.blackbird.failOnCodegenError=true, so a generator bug fails the test instead of a silent fallback to stock.blackbird-tests(classpath mode, 74 tests with 1 skipped in a run on JDK 25 without-Dblackbird.test.oldJdks): engagement for beans in the unnamed module; differential tests (each scenario runs through a stock mapper and a Blackbird mapper, compares the value or the exception class, message and reference path, and asserts which path Blackbird took); and the packaged jar in child JVMs on JDK 25 and on each JDK that-Dblackbird.test.oldJdkslists (discovery on the class path and the module path with a second module provider on the path, and explicit registration with discovery before it, after it and without it), the module descriptor, the bundle exports, the release guard, and an isolated class loader. Without-Dblackbird.test.oldJdks, the old-JDK cases are skipped. CI passes Temurin 17 and 21, and the JDK 25 jobs fail if either JDK is missing. The workflow that runs on databind pushes now also builds and tests Blackbird on JDK 25.blackbird-jpms-tests(2 tests): a foreign named module that requires Blackbird, with package-private beans, on the module path.Each finding of the review with testable behavior maps to a test that fails on the previous head and passes now. Our final review of this version found a builder setter, declared to return the builder, that returns null: stock keeps the current builder, and the generated codec failed. That case is fixed, and
ReaderErrorsDiffTest.builderSetterReturningNullKeepsBuilderfails without the fix. The OSGi fix has a check of the manifest only, without an OSGi framework.The databind test corpus is a manual check, not part of the build.
blackbird-tests/corpusruns the whole jackson-databind test suite with Blackbird registered in every mapper, through a 10-line patch to databind'sObjectMapper(MapperBuilder)constructor. Theblackbird-testsREADME says how to run it. Maintainers run it before they submit a Blackbird change. At42bc8bb04f, with databindf69415e590e4(upstream3.x), 7592 tests ran, and 7 failed. The 7 assert the exact list or number of registered modules, which the harness changes by one. An earlier head of this PR had 72 failures. The corpus is manual because databind publishes no test jar and the hook patches databind main code, so a CI job would break when databind changes that constructor. If databind adds a supported test hook (a system property that names extra modules to register), a CI job for Blackbird and Afterburner becomes simple.Benchmarks
The suite is in
blackbird/benchmarks(records, builders, setter POJOs, 40- and 200-property beans, cold start). Aclassic-blackbirdprofile builds the same benchmarks against Blackbird 3.2.2. Throughput relative to stock databind on JDK 25.0.4, x86_64 (Ryzen 7 5800X3D). Each ratio is from one JMH run that also measured stock, and the 99.9% confidence intervals are 1 to 6% of the scores. We measured these ratios and the cold-start costs at1d91be5d7b, before the later codegen changes on this branch, which emit more code per property:Two rows are slower than classic, and in both this PR keeps the stock deserializer. A POJO with a property-based
@JsonCreatorstays on stock (1.02x, classic 1.26x), because classic accelerated that creator. The 200-property read stays on stock (0.99x, classic 1.04x), because its generated reader would be over 7000 bytes, and HotSpot interpreted that reader at 0.27x of stock before the size limit. Cold start costs about 1 ms more per bean type than stock (stock 0.83 ms, this PR 1.85 ms, classic 1.82 ms). The README has the full table, andblackbird/benchmarks/README.mdhas the raw scores and the commands.Compatibility notes
deser.CreatorOptimizer,deser.ObjBooleanConsumer, the function interfaces ofBBDeserializerModifier,ser.ToBooleanFunction, and theutilpackage.BBDeserializerModifierandBBSerializerModifiertake no arguments. The package-privateCrossLoaderAccessis removed (closes Remove CrossLoaderAccess companion-class slow path from Blackbird (deprecated in 3.2) #350; its methods were deprecated for removal since 3.2).WrongMethodTypeExceptionwith blackbird #142: the affected code path is gone.-Dblackbird.debug.dumpDir=<dir>and decompile the class files.-Dblackbird.debug.codegen=trueprints each gate decision.The release target (3.3.0 or 3.4) is your decision. The release notes in
release-notes/VERSIONare under 3.3.0 for now.