Repository navigation
Conversation
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The changes preserve ordered, single-threaded archive access while bounding parallel work, with no unresolved findings from static review.
No open issues 🎉
What changed in this PR
Extends the parallel remapping approach used for bytecode to shadowed sources JAR generation, while preserving deterministic entry order.
Changes:
- Remaps sources on
Dispatchers.Defaultthrough a bounded 128-entry queue, keeping file reads and ZIP writes on the calling thread. - Adds a regression test covering 500 sources and their companion resources.
- Clarifies that relocators must support concurrent source remapping.
| File | Description |
|---|---|
| src/test/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/ShadowedSourcesJarTest.kt | Tests source and resource output across multiple queue windows. |
| src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/relocation/Relocator.kt | Extends thread-safety documentation to source remapping. |
| src/main/kotlin/com/github/jengelman/gradle/plugins/shadow/internal/ShadowedSourcesJar.kt | Adds bounded parallel source remapping with ordered ZIP writes. |
Review effort: Balanced
Goooler
force-pushed
the
g/20261025/parallel-shadowed-sources
branch
from
October 25, 2026 11:54
90a14ef to
6551b87
Compare
Apply the same sliding-window coroutine approach used for bytecode remapping in ShadowCopyAction to source file remapping in generateSourcesJar, keeping entry order deterministic. Resources are written directly after draining the pending sources, so they are never buffered.
Share the sliding-window ordered writer between ShadowCopyAction and the shadowed sources JAR generation instead of duplicating the pending entry queue in both.
Goooler
force-pushed
the
g/20261025/parallel-shadowed-sources
branch
from
October 25, 2026 12:02
6551b87 to
939c356
Compare
Goooler
added this pull request to stack #2347
October 25, 2026 12:09
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #2265, applying the same approach as #2302 to the shadowed sources JAR.
Source files are read on the calling thread and remapped on
Dispatchers.Default, through a bounded sliding window (128 pending entries) that writes entries in their original order. When a resource comes up, the pending sources are written first and then the resource is written directly, so resources are never buffered and the output stays deterministic.Benchmark Environment & Configuration
ktfmt:ktfmt:shadowJar --rerun(enableAutoRelocation = true,generateSourcesJar = true)cf71351a(sequential baseline) vs6551b877(parallel source remapping)The measured time includes the main JAR. Bytecode remapping for the main JAR is already parallel in both versions (#2302), so the difference comes from generating the sources JAR (~14.5 MB). Both versions produce byte-identical main and sources JARs.
Performance Comparison
cf71351a(Sequential Baseline)6551b877(Parallel Optimization)Summary
:ktfmt:shadowJardropped from ~28.9s to ~13.3s (~54.1% less).