Skip to content

Avoid reopening jars per resource request and deleteOnExit per upload - #1217

Open
pjfanning wants to merge 3 commits into
apache:mainfrom
pjfanning:bugs
Open

Avoid reopening jars per resource request and deleteOnExit per upload#1217
pjfanning wants to merge 3 commits into
apache:mainfrom
pjfanning:bugs

Conversation

@pjfanning

@pjfanning pjfanning commented Aug 22, 2026

Copy link
Copy Markdown
Member

Two independent resource-handling fixes in the file and resource directives, one commit each (plus a follow-up commit that makes the first one configurable), so they can be split if you prefer separate PRs.

Motivation

Jar resources are reopened per request. ResourceFile.apply opened a java.util.zip.ZipFile for every request to a resource inside a jar, only to read the entry's size and time. That re-parses the whole central directory of the jar on each request, and getFromResource/getFromResourceDirectory served out of a jar is the usual production layout for static resources. The result of getEntry was also dereferenced without a null check, so a missing entry would throw an NPE rather than reject.

Uploads register a deleteOnExit per file. fileUploadAll called File.deleteOnExit() on each temporary upload file. The JVM keeps every path passed to deleteOnExit in a global set for the lifetime of the process, and the entry is not removed when the file is deleted after the stream is consumed, so a long-running server accepting uploads grows its heap by one entry per upload.

Modification

  • Read jar entry metadata from the JarURLConnection and leave its cache enabled, so the JDK reuses the same open jar the class loader already holds; guard against a null or vanished entry; share the plain URLConnection handling with the fallback branch via a small helper.
  • Add a pekko.http.routing.use-jar-file-cache setting (on by default) for that behaviour. With it off, the connection reading the metadata owns its jar file and closes it again, and the entity stream is opened through a connection with caches disabled too. ResourceFile.apply(url) keeps its previous meaning and uses the cache; the new ResourceFile.apply(url, useJarFileCache) overload takes the flag.
  • Place upload temp files in a directory of their own and register a single shutdown hook that removes it recursively, instead of one deleteOnExit per file.

Result

No jar is opened or parsed per request for jar-hosted resources, and a missing entry rejects instead of throwing. Deployments that need to replace jar files while the server runs (an open jar file cannot be replaced on Windows) can set use-jar-file-cache = off — which the previous implementation could not offer at all, since it opened its own ZipFile for the metadata but still streamed content through URL.openStream, which uses the JDK caches.

The on-exit cleanup that fileUploadAll documents is unchanged but now costs one shutdown hook per JVM rather than one permanent global entry per upload; the dedicated directory also gets the owner-only permissions Files.createTempDirectory applies. Note that addShutdownHook throws IllegalStateException once shutdown is in progress, so an upload arriving during shutdown fails — deleteOnExit threw in the same situation, so this is parity rather than a regression.

Scala and Java DSL settings are in parity, @since 2.0.0 is on the new public methods, and MiMa filters are added for the two new RoutingSettings members (needed on Scala 3).

Tests

  • sbt "http-tests/testOnly org.apache.pekko.http.scaladsl.server.directives.FileAndResourceDirectivesSpec" - pass; new tests assert that the jar entry metadata matches the bytes actually served (a wrong length fails at toStrict), that a second request to the same route still works, and that a jar resource is still served correctly with use-jar-file-cache = off
  • sbt "http-tests/testOnly org.apache.pekko.http.scaladsl.server.directives.FileUploadDirectivesSpec" - pass; new test asserts the temp files share one directory
  • sbt http-tests/test - pass (1485 tests; TimeoutDirectivesSpec flaked once in a full run and passes on its own)
  • sbt +http/compile - pass on 2.13.18 and 3.3.8
  • sbt +http/mimaReportBinaryIssues - pass
  • sbt http/scalafmt http-tests/Test/scalafmt - clean

References

None - avoids reopening jars per resource request and an unbounded deleteOnExit registration per upload

Motivation:
`ResourceFile` opened a `java.util.zip.ZipFile` for every request to a
resource that lives in a jar, only to read the entry's size and time. That
parses the whole central directory of the jar again per request, and
`getFromResource`/`getFromResourceDirectory` served from a jar is the usual
production layout for static resources. The result of `getEntry` was also
dereferenced without a null check.

Modification:
Read the metadata from the `JarURLConnection` instead and leave its cache
enabled, so the JDK reuses the same open jar file that the class loader
already holds. Guard against a null entry, and share the plain
`URLConnection` handling with the fallback branch.

Result:
No jar is opened or parsed per request for resources served from a jar, and
a missing entry rejects the request instead of throwing.

Tests:
- sbt "http-tests/testOnly org.apache.pekko.http.scaladsl.server.directives.FileAndResourceDirectivesSpec" - pass, 1 new test asserting the entry metadata matches the bytes served
- sbt http-tests/test - pass
- sbt +http/compile - pass
- sbt http/mimaReportBinaryIssues - pass
- sbt http/scalafmt http-tests/Test/scalafmt - clean

References:
None - avoids reopening jars for every resource request
Motivation:
`fileUploadAll` called `File.deleteOnExit()` for each temporary upload file.
The JVM keeps every path passed to `deleteOnExit` in a global set for the
lifetime of the process, and the entry is not removed when the file itself
is deleted after the stream is consumed. A long-running server accepting
uploads therefore grows its heap by one entry per upload, forever.

Modification:
Put the temporary upload files in a directory of their own and register a
single shutdown hook that removes that directory recursively on exit.

Result:
The on-exit cleanup that the directive documents is unchanged, but it now
costs one shutdown hook per JVM instead of one permanent global entry per
uploaded file. The dedicated directory is created with the owner-only
permissions that `Files.createTempDirectory` applies.

Tests:
- sbt "http-tests/testOnly org.apache.pekko.http.scaladsl.server.directives.FileUploadDirectivesSpec" - pass, 1 new test asserting the temp files share one directory
- sbt http-tests/test - pass
- sbt +http/compile - pass
- sbt http/mimaReportBinaryIssues - pass
- sbt http/scalafmt http-tests/Test/scalafmt - clean

References:
None - removes an unbounded deleteOnExit registration per upload
Motivation:
Reading jar resource metadata through the JDK's jar file cache means the
jar file stays open for the lifetime of the process, which prevents the jar
from being replaced while the server runs (on Windows an open file cannot
be replaced). That should be a choice rather than something the directives
decide.

Modification:
Add a `pekko.http.routing.use-jar-file-cache` setting, on by default, and
pass it from `getFromResource` into `ResourceFile`. With the setting off,
the connection that reads the entry metadata owns its jar file and closes
it again, and the entity stream is opened through a connection with caches
disabled as well, so that nothing keeps the jar open between requests.
`ResourceFile.apply(url)` keeps its previous meaning and uses the cache.

Result:
The default is the cached behaviour, and deployments that need to replace
jar files at runtime can turn the cache off. Note that the previous
implementation could not offer that at all: it opened its own `ZipFile` for
the metadata but still streamed the content through `URL.openStream`, which
uses the JDK caches.

Tests:
- sbt "http-tests/testOnly org.apache.pekko.http.scaladsl.server.directives.FileAndResourceDirectivesSpec" - pass, 1 new test serving a jar resource with the cache disabled
- sbt http-tests/test - pass (TimeoutDirectivesSpec flaked in the full run, passes on its own)
- sbt +http/mimaReportBinaryIssues - pass
- sbt http/scalafmt http-tests/Test/scalafmt - clean

References:
None - follow-up to the jar resource change on this branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant