profiling: request carrier-scoped ddprof context storage on JDK 21+#11826
Open
rkennke wants to merge 3 commits into
Open
profiling: request carrier-scoped ddprof context storage on JDK 21+#11826rkennke wants to merge 3 commits into
rkennke wants to merge 3 commits into
Conversation
…PROF-15271) The ddprof profiler exposes its OTEL context ThreadContext as a DirectByteBuffer over the carrier thread's native record. When that storage is keyed by the virtual thread (the pre-fix default), a mounted vthread pins to its first carrier: writes land on the wrong carrier after migration, and become a use-after-free once that carrier's OS thread exits. The java-profiler fix (PROF-15271) adds carrier-scoped storage via jdk.internal.misc.CarrierThreadLocal, selected by the ddprof.context.storage.mode system property. Before the profiler is loaded, on JDK 21+ (profiler enabled, non-Windows): - export java.base/jdk.internal.misc to the classloader that loads com.datadoghq.profiler.* so CarrierThreadLocal is reachable, and - set ddprof.context.storage.mode=carrier (only if unset), which requests carrier scoping and makes the profiler fail fast if it cannot honor it. Both actions are inert against ddprof builds that predate carrier support (the property is ignored; the export goes unused), so this is safe to ship ahead of the ddprof version bump — it activates automatically when that lands. Operators opt out with -Dddprof.context.storage.mode=thread (or =auto); we only set the property when unset, so an explicit choice always wins. No separate dd.* config key: the profiler's own system property is the single kill-switch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
🎯 Code Coverage (details) 🔗 Commit SHA: 7c7fc81 | Docs | Datadog PR Page | Give us feedback! |
Contributor
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
…dogTracer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ROF-15271) The java-profiler side renamed the selector to ddprof.debug.context.storage.mode (ddprof.debug.*, signalling an internal knob). Match the property name we export and set here so the carrier request still takes effect. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
On JDK 21+, before the ddprof profiler is loaded, this:
java.base/jdk.internal.miscto the classloader that loadscom.datadoghq.profiler.*, andddprof.context.storage.modesystem property tocarrier(only when unset).Together these request carrier-scoped OTEL context storage in the profiler, backed by
jdk.internal.misc.CarrierThreadLocal.Implemented as a small
prepareDatadogProfilerContextStorage(Instrumentation)helper inAgent, invoked inInstallDatadogTracerCallbackright beforeinstallDatadogTracer(which triggers the profiler load, so the export/property are in place beforeJavaProfileris constructed).Motivation
The profiler exposes its OTEL context
ThreadContextas aDirectByteBufferover the carrier thread's native record — the record the (carrier-bound) sampler reads. When that storage is keyed by the virtual thread (the current default), a mounted vthread pins to whichever carrier it first ran on:ThreadsSMRSupport::free_listunder a Loom workload).The java-profiler fix (PROF-15271) adds carrier-scoped storage via
CarrierThreadLocal, so a mounted vthread always resolves to its current carrier's live record.CarrierThreadLocalis in a non-exported package, hence the export; and the profiler defaults toauto(graceful), so to get the strict carrier mode (which fails fast if it cannot be honored) dd-trace-java must request it explicitly.Additional Notes
1.44.0): the property is ignored and the export goes unused. They activate automatically when a ddprof release containing the fix is bumped inlibs.versions.toml— that version bump is the real activation gate and where CARRIER + fail-fast should be validated on JDK 21+.mode=carrier, the profiler'screate()throws ifCarrierThreadLocalis inaccessible; the existinginitJavaProfiler()try/catch turns that intoreasonNotLoaded→UnsupportedOperationException, so ddprof disables itself cleanly rather than crashing the app.-Dddprof.context.storage.mode=thread(or=auto); we only set the property when unset, so an explicit choice always wins. Nodd.*passthrough was added (a thin one is a trivial follow-up if env-var/remote-config control is later wanted).isProfilingEnabled() && isDatadogProfilerEnabled() && !isWindows() && isJavaVersionAtLeast(21).JDK9ModuleAccess(internal-api-9) is only referenced under the 21+ guard, so it never loads on Java 8. UsesSystemProperties.get/set(not rawSystem.*).jdk.internal.miscviaredefineModulefollows existing precedent (AdvancedAgentChecksdoes the same for CDS detection).Follow-up: bump
ddprofinlibs.versions.tomlto the release containing the fix (separate PR), plus a JDK 21+ test asserting carrier mode is active once bumped.Contributor Checklist
type:/comp:labelsclose/fixlinking keywords usedJira ticket: PROF-15271
🤖 Generated with Claude Code