feat: optionally compose @JsonSubTypes parents into oneOf

[vpelikh]

Pull Request

Thank you for contributing to swagger-core!

Please fill out the following information to help us review your PR efficiently.

---

Description

Adds an opt-in that composes a @JsonSubTypes / @Schema(subTypes) polymorphic parent into a self-referencing oneOf: the parent gets a oneOf to its subtypes while the subtypes keep allOf to the parent.

This cyclic allOf <-> oneOf shape is off by default because several tools (swagger-ui, openapi-generator, ...) dislike it, so the default output is unchanged from 3.0.0. It can be enabled with -Djson-subtypes-oneof or ModelResolver.jsonSubTypesOneOf = true.

When enabled, the parent model is replaced with a ComposedSchema whose oneOf references the resolved subtypes (using the resolved @JsonView-aware names); an explicit @Schema(oneOf) wins over the automatic composition to avoid a recursive allOf <-> oneOf.

resolveSubtypes() is now protected instead of private, providing an extension point for downstream subclasses (see #5030).

Related: #4991, #5030, #5331, #5332

Type of Change

• 🐛 Bug fix
• ✨ New feature
• ♻️ Refactor (non-breaking change)
• 🧪 Tests
• 📝 Documentation
• 🧹 Chore (build or tooling)

Checklist

• I have added/updated tests as needed
• I have added/updated documentation where applicable
• The PR title is descriptive
• The code builds and passes tests locally
• I have linked related issues (if any)

Screenshots / Additional Context

• Default (opt-in disabled) output is identical to 3.0.0; no existing expectations change.
• Tests: adds opt-in coverage to JsonSubTypesAndSchemaOneOfTest (composition when enabled, no composition by default, and the resolveSubtypes() override opt-out).
• With a WRAPPER_OBJECT parent, the composed oneOf appears nested inside the wrapping property; that interaction is handled separately in fix: keep oneOf when WRAPPER_OBJECT wrapping is applied #5331.

[vpelikh]

CC @Mattias-Sehlstedt

[Mattias-Sehlstedt]

Could we share what tools that prefer a cyclic polymorphic structure? As shared in #5314 (comment) my experience is that most actually prefer it to not exists, and that even OAS documentation has previously documented it as "the two different polymorphic structures (e.g., the oneOf and the allOf approach).

To me

Early skip: @Schema(oneOf) annotation processing is skipped when @JsonSubTypes is present, preventing a duplicate oneOf

feels like the opposite of how the annotations function in general? Isn't the general idea that first heuristics are used to derive the OAS structure, but that we then provide these annotations so that a user can force the structure to follow something that the user explicitly would like to correct? Now we instead force the generation to follow the internal interpretation, and ignore any explicit structure that the user has asked for? Basically copying what happened in springdoc/springdoc-openapi#2915?

If someone would like to restore the previous behavior, have we thought about making that easily-achievable? I would recommend ensuring that there are good extensions points for it, and that it is illustrated with a test similar to #5030.

[vpelikh]

Thanks for the detailed review, @Mattias-Sehlstedt. I've addressed both of your points with two follow-up commits on this branch.

1. Extension point to restore the previous behavior (easy opt-out)

resolveSubtypes() was private; it's now protected, so a downstream ModelResolver subclass can override it to return false and opt out of the automatic @JsonSubTypes → oneOf composition — restoring the previous behavior. This is illustrated with a test:

• JsonSubTypesAndSchemaOneOfTest.extensionPointAllowsSubclassToRestorePreviousBehavior

which shows a subclass override preventing the oneOf conversion (same pattern suggested in PR #5030).

2. Explicit @Schema(oneOf) now wins over @JsonSubTypes

Agreed on the philosophy point. The previous "early skip" inverted user intent: it skipped the explicit @Schema(oneOf) whenever @JsonSubTypes was present, forcing the internal composition over the user's explicit structure.

I changed this so:

• The explicit @Schema(oneOf) is always processed, even when @JsonSubTypes is present (the early-skip was removed).
• resolveSubtypes() now returns early when a class declares its own @Schema(oneOf), so the explicit composition takes precedence over the @JsonSubTypes-derived one.

This also removes the cyclic allOf <-> oneOf structure: when a class declares @Schema(oneOf), the parent keeps only its shared discriminator property, and subtypes become plain object schemas (no recursive allOf → parent ref).

On "which tools prefer a cyclic polymorphic structure"

The original motivation for the @JsonSubTypes → oneOf combination was to move away from the allOf-heavy form (including requests for @JsonSubTypes-driven generation). I agree the cyclic allOf + oneOf hybrid is often undesirable. With @Schema(oneOf) now winning, users who want the explicit non-cyclic form get it; the auto-composition remains only for @JsonSubTypes-only types, and users can always opt out via the resolveSubtypes extension point.

Happy to take further direction if you'd prefer a different default.

[Mattias-Sehlstedt]

What should be default I will leave to swagger-core to decide. I am very happy as long as there is an easy way to control the behavior with configurations/extensions.

As a human I prefer the more logical allOf <-> oneOf, since it makes sense to the parts to be aware of each other. But I have gotten used to it always being exactly one of them, since swagger-ui and openapi-generator both dislike cyclic structures (I have not tried scalar to see if it handles it properly).

I can see if I have time to investigate further the history behind it and what might actually be considered best-practice. Do note that any findings should not impact this PR in any way, since it gives full control to the user, and thus findings will not impact code but at most recommendations for users.

[Mattias-Sehlstedt]

I've searched around a bit and all I can find point to the best approach being to not have them together. If one wants to create a "link" between the parent and the children, the best way to express that is with the discriminatorMapping (i.e., basically what is done in #4983).

One can for example query de-detoxified Stack overflow (read: AI) with a biased prompt "I want to strengthen the relation between parent and child with oneOf + allOf because..." and it will generally even advice you against doing it even if you are biased and insistent on it being correct. This issue with it being that the oneOf becomes entirely a validation constraint, and that the validation cycle introduced cannot terminate (A Dog is a Pet and a Pet needs to be a Dog...). The recommendation instead is a self-contained oneOf union, e.g.,

Pet:
  oneOf:
    - $ref: '#/components/schemas/Dog'
    - $ref: '#/components/schemas/Cat'
  discriminator:
    propertyName: petType
    mapping:
      dog: '#/components/schemas/Dog'
      cat: '#/components/schemas/Cat'

So my personal suggestion would be to not make oneOf + allOf as the default, but instead rather make it an easy opt-in for those that find it beneficial.

[vpelikh]

@Mattias-Sehlstedt, thanks for your input! Now let's wait for a reply from the team regarding the @Schema/@JsonSubTypes allOf/oneOf relationship.

[vpelikh]

Thanks @Mattias-Sehlstedt, that feedback changed the direction of this PR. Summary of where it landed:

1. Cyclic allOf + oneOf is no longer the default. You convinced me. The @JsonSubTypes → oneOf composition is now opt-in, off by default, so the default output is unchanged from 3.0.0. It's enabled with -Djson-subtypes-oneof or ModelResolver.jsonSubTypesOneOf = true (same pattern as the existing enums-as-ref flag). No existing expectations change.

2. Explicit @Schema(oneOf) wins. The early-skip that inverted user intent is gone: an explicit @Schema(oneOf) is honored and takes precedence over the @JsonSubTypes-derived composition, so users who want the self-contained, non-cyclic union get exactly that.

3. Extension point + test. resolveSubtypes() is now protected instead of private, so a downstream ModelResolver subclass can override only that method to customize or disable subtype resolution (the hook you asked for). It's covered by JsonSubTypesAndSchemaOneOfTest.extensionPointAllowsSubclassToRestorePreviousBehavior.

On "which tools prefer a cyclic structure": I don't have a concrete list to cite. Your references (swagger-ui, openapi-generator) and the discriminator-mapping approach in #4983 settled it for me, so the composition is opt-in rather than default.

With this PR a @JsonSubTypes user has two options:

• Default (flag off) or an explicit @Schema(oneOf): the self-contained oneOf union with discriminator + mapping (the shape you prefer).
• Flag on (-Djson-subtypes-oneof): the linked/cyclic oneOf + allOf form, for those who want the parent to reference its children.

If the team wants the linked form to be the default, that's a small follow-up: flip the flag's default.