Skip to content

fix(trace-exporter): mark OTLP client-computed stats#2245

Open
bengl wants to merge 1 commit into
mainfrom
bengl/otlp-client-computed-stats
Open

fix(trace-exporter): mark OTLP client-computed stats#2245
bengl wants to merge 1 commit into
mainfrom
bengl/otlp-client-computed-stats

Conversation

@bengl

@bengl bengl commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Marks OTLP trace exports as client-computed when tracer metadata already says stats were computed client-side.

This sets both:

  • the OTLP Resource attribute _dd.stats_computed=true
  • the direct Agent request header datadog-client-computed-stats: yes

when either metadata.client_computed_stats or libdatadog's native OTLP stats worker is enabled.

Motivation

dd-trace-js computes OTLP span metrics in JS and passes that state through set_client_computed_stats(). The OTLP trace exporter only looked at otlp_stats_enabled, so Resources sent by JS-backed stats did not carry _dd.stats_computed=true and system-tests rejected the payload.

Additional Notes

The native OTLP stats worker path remains unchanged because otlp_stats_enabled still sets the same Resource attribute and header.

How to test the change?

cargo test -p libdd-data-pipeline --test test_trace_exporter_otlp_export -- --nocapture

Result: 2 passed.

@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

📚 Documentation Check Results

⚠️ 1206 documentation warning(s) found

📦 libdd-data-pipeline - 1206 warning(s)


Updated: 2026-07-16 16:39:18 UTC | Commit: 76e5a6c | missing-docs job results

@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

🔒 Cargo Deny Results

⚠️ 3 issue(s) found, showing only errors (advisories, bans, sources)

📦 libdd-data-pipeline - 3 error(s)

Show output
error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:75:1
   │
75 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-data-pipeline v7.0.0 (*)
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-data-pipeline v7.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:218:1
    │
218 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by @lopopolo) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-data-pipeline v7.0.0
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-data-pipeline v7.0.0 (*)
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0
      │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   └── libdd-trace-stats v6.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-data-pipeline v7.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── libdd-data-pipeline v7.0.0 (*)
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:287:1
    │
287 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── tracing-appender v0.2.3
          └── libdd-log v1.0.0
              └── (dev) libdd-data-pipeline v7.0.0

advisories FAILED, bans ok, sources ok

Updated: 2026-07-16 16:40:58 UTC | Commit: 76e5a6c | dependency-check job results

@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Jul 16, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 74.73% (+0.01%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 6356aee | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jul 16, 2026

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-07-16 16:49:19

Comparing candidate commit 6356aee in PR branch bengl/otlp-client-computed-stats with baseline commit 95d0c13 in branch main.

Found 2 performance improvements and 0 performance regressions! Performance is the same for 22 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:trace_buffer/4_senders/no_delay

  • 🟩 execution_time [-171.484µs; -140.695µs] or [-6.981%; -5.728%]
  • 🟩 throughput [+90567.102op/s; +111073.141op/s] or [+6.175%; +7.573%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 6356aee 1784219749 bengl/otlp-client-computed-stats
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
trace_buffer/1_senders/10us_delay execution_time 56.352ms 56.610ms ± 0.146ms 56.595ms ± 0.108ms 56.710ms 56.874ms 56.967ms 57.065ms 0.83% 0.531 -0.234 0.26% 0.010ms 1 200
trace_buffer/1_senders/10us_delay throughput 15771.624op/s 15898.249op/s ± 40.990op/s 15902.488op/s ± 30.502op/s 15932.228op/s 15956.818op/s 15968.258op/s 15971.088op/s 0.43% -0.519 -0.253 0.26% 2.898op/s 1 200
trace_buffer/1_senders/1us_delay execution_time 49.978ms 50.086ms ± 0.031ms 50.082ms ± 0.008ms 50.089ms 50.144ms 50.190ms 50.203ms 0.24% 1.236 3.636 0.06% 0.002ms 1 200
trace_buffer/1_senders/1us_delay throughput 17927.292op/s 17969.194op/s ± 11.188op/s 17970.578op/s ± 2.808op/s 17973.654op/s 17983.987op/s 17990.537op/s 18007.765op/s 0.21% -1.228 3.625 0.06% 0.791op/s 1 200
trace_buffer/1_senders/no_delay execution_time 331.501µs 339.316µs ± 4.420µs 338.488µs ± 1.485µs 340.288µs 344.474µs 347.309µs 387.130µs 14.37% 6.771 67.882 1.30% 0.313µs 1 200
trace_buffer/1_senders/no_delay throughput 2324801.994op/s 2652808.972op/s ± 31929.351op/s 2658879.914op/s ± 11716.513op/s 2668679.403op/s 2681297.046op/s 2691178.891op/s 2714923.937op/s 2.11% -5.899 55.245 1.20% 2257.746op/s 1 200
trace_buffer/2_senders/10us_delay execution_time 56.831ms 57.262ms ± 0.183ms 57.253ms ± 0.109ms 57.364ms 57.577ms 57.696ms 57.756ms 0.88% 0.308 -0.100 0.32% 0.013ms 1 200
trace_buffer/2_senders/10us_delay throughput 31165.522op/s 31434.530op/s ± 100.407op/s 31439.492op/s ± 59.667op/s 31498.480op/s 31586.536op/s 31645.497op/s 31672.809op/s 0.74% -0.291 -0.113 0.32% 7.100op/s 1 200
trace_buffer/2_senders/1us_delay execution_time 50.060ms 50.121ms ± 0.031ms 50.117ms ± 0.016ms 50.133ms 50.179ms 50.212ms 50.233ms 0.23% 0.931 1.066 0.06% 0.002ms 1 200
trace_buffer/2_senders/1us_delay throughput 35833.240op/s 35913.368op/s ± 22.168op/s 35916.190op/s ± 11.687op/s 35928.257op/s 35943.607op/s 35950.081op/s 35956.651op/s 0.11% -0.926 1.055 0.06% 1.567op/s 1 200
trace_buffer/2_senders/no_delay execution_time 826.807µs 877.902µs ± 20.356µs 877.613µs ± 13.503µs 892.935µs 909.349µs 928.901µs 940.995µs 7.22% 0.019 0.195 2.31% 1.439µs 1 200
trace_buffer/2_senders/no_delay throughput 1912869.788op/s 2051441.294op/s ± 47629.351op/s 2051017.748op/s ± 31548.887op/s 2080766.298op/s 2134998.400op/s 2171114.552op/s 2177048.952op/s 6.14% 0.132 0.178 2.32% 3367.904op/s 1 200
trace_buffer/4_senders/10us_delay execution_time 57.222ms 57.523ms ± 0.132ms 57.513ms ± 0.083ms 57.597ms 57.769ms 57.895ms 57.946ms 0.75% 0.593 0.664 0.23% 0.009ms 1 200
trace_buffer/4_senders/10us_delay throughput 62127.181op/s 62583.926op/s ± 143.501op/s 62594.726op/s ± 90.822op/s 62684.452op/s 62776.771op/s 62891.469op/s 62912.903op/s 0.51% -0.577 0.637 0.23% 10.147op/s 1 200
trace_buffer/4_senders/1us_delay execution_time 50.058ms 50.176ms ± 0.042ms 50.164ms ± 0.019ms 50.201ms 50.261ms 50.294ms 50.328ms 0.33% 0.968 0.934 0.08% 0.003ms 1 200
trace_buffer/4_senders/1us_delay throughput 71530.615op/s 71747.103op/s ± 59.434op/s 71764.500op/s ± 27.892op/s 71783.997op/s 71818.041op/s 71830.505op/s 71916.733op/s 0.21% -0.963 0.925 0.08% 4.203op/s 1 200
trace_buffer/4_senders/no_delay execution_time 1.985ms 2.300ms ± 0.090ms 2.300ms ± 0.060ms 2.366ms 2.430ms 2.468ms 2.502ms 8.77% -0.483 0.503 3.88% 0.006ms 1 200
trace_buffer/4_senders/no_delay throughput 1438744.192op/s 1567454.289op/s ± 62556.966op/s 1564896.647op/s ± 40289.483op/s 1603305.196op/s 1678114.545op/s 1749006.742op/s 1813569.010op/s 15.89% 0.768 1.166 3.98% 4423.446op/s 1 200
trace_buffer/8_senders/10us_delay execution_time 57.675ms 58.050ms ± 0.279ms 57.934ms ± 0.143ms 58.274ms 58.621ms 58.738ms 58.771ms 1.44% 0.814 -0.527 0.48% 0.020ms 1 200
trace_buffer/8_senders/10us_delay throughput 122508.692op/s 124033.411op/s ± 593.471op/s 124278.450op/s ± 308.575op/s 124527.418op/s 124672.344op/s 124723.865op/s 124837.033op/s 0.45% -0.803 -0.553 0.48% 41.965op/s 1 200
trace_buffer/8_senders/1us_delay execution_time 50.204ms 50.309ms ± 0.065ms 50.298ms ± 0.043ms 50.348ms 50.428ms 50.487ms 50.540ms 0.48% 0.866 0.634 0.13% 0.005ms 1 200
trace_buffer/8_senders/1us_delay throughput 142460.921op/s 143114.526op/s ± 183.887op/s 143147.420op/s ± 123.800op/s 143256.022op/s 143345.409op/s 143396.221op/s 143415.836op/s 0.19% -0.858 0.612 0.13% 13.003op/s 1 200
trace_buffer/8_senders/no_delay execution_time 5.933ms 6.483ms ± 0.095ms 6.493ms ± 0.059ms 6.548ms 6.611ms 6.634ms 6.665ms 2.66% -1.383 5.026 1.46% 0.007ms 1 200
trace_buffer/8_senders/no_delay throughput 1080226.145op/s 1110768.596op/s ± 16600.541op/s 1108906.390op/s ± 10093.223op/s 1119561.772op/s 1137525.086op/s 1158372.109op/s 1213621.417op/s 9.44% 1.629 6.694 1.49% 1173.835op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
trace_buffer/1_senders/10us_delay execution_time [56.590ms; 56.631ms] or [-0.036%; +0.036%] None None None
trace_buffer/1_senders/10us_delay throughput [15892.568op/s; 15903.929op/s] or [-0.036%; +0.036%] None None None
trace_buffer/1_senders/1us_delay execution_time [50.081ms; 50.090ms] or [-0.009%; +0.009%] None None None
trace_buffer/1_senders/1us_delay throughput [17967.644op/s; 17970.745op/s] or [-0.009%; +0.009%] None None None
trace_buffer/1_senders/no_delay execution_time [338.703µs; 339.928µs] or [-0.181%; +0.181%] None None None
trace_buffer/1_senders/no_delay throughput [2648383.871op/s; 2657234.073op/s] or [-0.167%; +0.167%] None None None
trace_buffer/2_senders/10us_delay execution_time [57.237ms; 57.288ms] or [-0.044%; +0.044%] None None None
trace_buffer/2_senders/10us_delay throughput [31420.615op/s; 31448.446op/s] or [-0.044%; +0.044%] None None None
trace_buffer/2_senders/1us_delay execution_time [50.116ms; 50.125ms] or [-0.009%; +0.009%] None None None
trace_buffer/2_senders/1us_delay throughput [35910.296op/s; 35916.440op/s] or [-0.009%; +0.009%] None None None
trace_buffer/2_senders/no_delay execution_time [875.081µs; 880.723µs] or [-0.321%; +0.321%] None None None
trace_buffer/2_senders/no_delay throughput [2044840.324op/s; 2058042.264op/s] or [-0.322%; +0.322%] None None None
trace_buffer/4_senders/10us_delay execution_time [57.505ms; 57.541ms] or [-0.032%; +0.032%] None None None
trace_buffer/4_senders/10us_delay throughput [62564.038op/s; 62603.814op/s] or [-0.032%; +0.032%] None None None
trace_buffer/4_senders/1us_delay execution_time [50.171ms; 50.182ms] or [-0.011%; +0.011%] None None None
trace_buffer/4_senders/1us_delay throughput [71738.866op/s; 71755.340op/s] or [-0.011%; +0.011%] None None None
trace_buffer/4_senders/no_delay execution_time [2.288ms; 2.313ms] or [-0.540%; +0.540%] None None None
trace_buffer/4_senders/no_delay throughput [1558784.495op/s; 1576124.083op/s] or [-0.553%; +0.553%] None None None
trace_buffer/8_senders/10us_delay execution_time [58.012ms; 58.089ms] or [-0.067%; +0.067%] None None None
trace_buffer/8_senders/10us_delay throughput [123951.162op/s; 124115.660op/s] or [-0.066%; +0.066%] None None None
trace_buffer/8_senders/1us_delay execution_time [50.300ms; 50.318ms] or [-0.018%; +0.018%] None None None
trace_buffer/8_senders/1us_delay throughput [143089.041op/s; 143140.011op/s] or [-0.018%; +0.018%] None None None
trace_buffer/8_senders/no_delay execution_time [6.470ms; 6.497ms] or [-0.203%; +0.203%] None None None
trace_buffer/8_senders/no_delay throughput [1108467.921op/s; 1113069.271op/s] or [-0.207%; +0.207%] None None None

Baseline

Baseline benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 95d0c13 1784218428 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
trace_buffer/1_senders/10us_delay execution_time 56.288ms 56.541ms ± 0.126ms 56.529ms ± 0.076ms 56.606ms 56.752ms 56.911ms 56.962ms 0.77% 0.750 0.699 0.22% 0.009ms 1 200
trace_buffer/1_senders/10us_delay throughput 15799.929op/s 15917.867op/s ± 35.505op/s 15920.946op/s ± 21.497op/s 15942.397op/s 15971.338op/s 15978.412op/s 15989.178op/s 0.43% -0.736 0.669 0.22% 2.511op/s 1 200
trace_buffer/1_senders/1us_delay execution_time 49.973ms 50.081ms ± 0.036ms 50.080ms ± 0.014ms 50.092ms 50.146ms 50.204ms 50.209ms 0.26% 1.007 2.795 0.07% 0.003ms 1 200
trace_buffer/1_senders/1us_delay throughput 17925.042op/s 17970.882op/s ± 12.895op/s 17971.325op/s ± 4.991op/s 17976.841op/s 17988.634op/s 17992.834op/s 18009.589op/s 0.21% -0.999 2.780 0.07% 0.912op/s 1 200
trace_buffer/1_senders/no_delay execution_time 333.594µs 342.265µs ± 5.708µs 341.567µs ± 2.094µs 343.750µs 347.381µs 356.842µs 405.816µs 18.81% 7.152 75.435 1.66% 0.404µs 1 200
trace_buffer/1_senders/no_delay throughput 2217751.757op/s 2630193.281op/s ± 39565.071op/s 2634918.176op/s ± 16053.021op/s 2650020.296op/s 2668137.689op/s 2675796.596op/s 2697893.568op/s 2.39% -5.948 57.768 1.50% 2797.673op/s 1 200
trace_buffer/2_senders/10us_delay execution_time 56.703ms 57.201ms ± 0.228ms 57.172ms ± 0.166ms 57.356ms 57.617ms 57.704ms 57.878ms 1.24% 0.354 -0.410 0.40% 0.016ms 1 200
trace_buffer/2_senders/10us_delay throughput 31099.756op/s 31468.576op/s ± 124.992op/s 31484.008op/s ± 91.443op/s 31564.080op/s 31641.973op/s 31707.543op/s 31744.314op/s 0.83% -0.336 -0.427 0.40% 8.838op/s 1 200
trace_buffer/2_senders/1us_delay execution_time 50.059ms 50.115ms ± 0.031ms 50.111ms ± 0.017ms 50.126ms 50.179ms 50.205ms 50.247ms 0.27% 1.340 2.535 0.06% 0.002ms 1 200
trace_buffer/2_senders/1us_delay throughput 35822.751op/s 35917.517op/s ± 22.119op/s 35920.491op/s ± 11.858op/s 35933.246op/s 35944.287op/s 35950.131op/s 35957.245op/s 0.10% -1.335 2.515 0.06% 1.564op/s 1 200
trace_buffer/2_senders/no_delay execution_time 848.550µs 898.413µs ± 19.950µs 898.806µs ± 10.714µs 909.465µs 923.789µs 939.765µs 1035.810µs 15.24% 1.350 10.187 2.22% 1.411µs 1 200
trace_buffer/2_senders/no_delay throughput 1737770.627op/s 2004492.918op/s ± 43508.254op/s 2002655.954op/s ± 23684.086op/s 2026068.411op/s 2080800.888op/s 2104535.738op/s 2121266.244op/s 5.92% -0.764 6.251 2.17% 3076.498op/s 1 200
trace_buffer/4_senders/10us_delay execution_time 57.294ms 57.577ms ± 0.174ms 57.525ms ± 0.079ms 57.632ms 57.962ms 58.177ms 58.216ms 1.20% 1.497 2.329 0.30% 0.012ms 1 200
trace_buffer/4_senders/10us_delay throughput 61838.335op/s 62526.031op/s ± 188.387op/s 62581.554op/s ± 85.827op/s 62636.679op/s 62734.369op/s 62827.340op/s 62834.231op/s 0.40% -1.478 2.263 0.30% 13.321op/s 1 200
trace_buffer/4_senders/1us_delay execution_time 50.086ms 50.172ms ± 0.039ms 50.164ms ± 0.018ms 50.187ms 50.264ms 50.281ms 50.297ms 0.26% 1.154 1.106 0.08% 0.003ms 1 200
trace_buffer/4_senders/1us_delay throughput 71575.398op/s 71753.379op/s ± 55.562op/s 71764.572op/s ± 26.119op/s 71787.307op/s 71818.934op/s 71829.441op/s 71876.413op/s 0.16% -1.150 1.097 0.08% 3.929op/s 1 200
trace_buffer/4_senders/no_delay execution_time 2.257ms 2.456ms ± 0.066ms 2.460ms ± 0.044ms 2.503ms 2.556ms 2.570ms 2.682ms 9.01% -0.183 0.211 2.67% 0.005ms 1 200
trace_buffer/4_senders/no_delay throughput 1342383.055op/s 1466634.168op/s ± 39494.006op/s 1463318.092op/s ± 25540.965op/s 1490309.950op/s 1537885.494op/s 1558207.795op/s 1594735.752op/s 8.98% 0.355 0.232 2.69% 2792.648op/s 1 200
trace_buffer/8_senders/10us_delay execution_time 57.619ms 57.992ms ± 0.281ms 57.879ms ± 0.149ms 58.216ms 58.538ms 58.618ms 58.857ms 1.69% 0.841 -0.409 0.48% 0.020ms 1 200
trace_buffer/8_senders/10us_delay throughput 122330.180op/s 124157.090op/s ± 599.687op/s 124396.583op/s ± 322.139op/s 124630.671op/s 124816.466op/s 124922.265op/s 124957.736op/s 0.45% -0.828 -0.439 0.48% 42.404op/s 1 200
trace_buffer/8_senders/1us_delay execution_time 50.173ms 50.288ms ± 0.056ms 50.271ms ± 0.028ms 50.316ms 50.410ms 50.458ms 50.508ms 0.47% 1.349 2.238 0.11% 0.004ms 1 200
trace_buffer/8_senders/1us_delay throughput 142551.988op/s 143175.882op/s ± 158.967op/s 143224.391op/s ± 78.690op/s 143282.959op/s 143357.281op/s 143399.197op/s 143504.144op/s 0.20% -1.341 2.209 0.11% 11.241op/s 1 200
trace_buffer/8_senders/no_delay execution_time 5.897ms 6.591ms ± 0.122ms 6.602ms ± 0.085ms 6.685ms 6.757ms 6.774ms 6.828ms 3.43% -1.200 4.197 1.84% 0.009ms 1 200
trace_buffer/8_senders/no_delay throughput 1054532.410op/s 1092716.709op/s ± 20663.279op/s 1090659.850op/s ± 14116.642op/s 1104946.310op/s 1123636.503op/s 1136455.730op/s 1220939.244op/s 11.95% 1.499 6.261 1.89% 1461.114op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
trace_buffer/1_senders/10us_delay execution_time [56.523ms; 56.558ms] or [-0.031%; +0.031%] None None None
trace_buffer/1_senders/10us_delay throughput [15912.946op/s; 15922.788op/s] or [-0.031%; +0.031%] None None None
trace_buffer/1_senders/1us_delay execution_time [50.076ms; 50.086ms] or [-0.010%; +0.010%] None None None
trace_buffer/1_senders/1us_delay throughput [17969.095op/s; 17972.669op/s] or [-0.010%; +0.010%] None None None
trace_buffer/1_senders/no_delay execution_time [341.474µs; 343.056µs] or [-0.231%; +0.231%] None None None
trace_buffer/1_senders/no_delay throughput [2624709.943op/s; 2635676.619op/s] or [-0.208%; +0.208%] None None None
trace_buffer/2_senders/10us_delay execution_time [57.169ms; 57.232ms] or [-0.055%; +0.055%] None None None
trace_buffer/2_senders/10us_delay throughput [31451.253op/s; 31485.899op/s] or [-0.055%; +0.055%] None None None
trace_buffer/2_senders/1us_delay execution_time [50.111ms; 50.119ms] or [-0.009%; +0.009%] None None None
trace_buffer/2_senders/1us_delay throughput [35914.452op/s; 35920.583op/s] or [-0.009%; +0.009%] None None None
trace_buffer/2_senders/no_delay execution_time [895.648µs; 901.178µs] or [-0.308%; +0.308%] None None None
trace_buffer/2_senders/no_delay throughput [1998463.093op/s; 2010522.744op/s] or [-0.301%; +0.301%] None None None
trace_buffer/4_senders/10us_delay execution_time [57.552ms; 57.601ms] or [-0.042%; +0.042%] None None None
trace_buffer/4_senders/10us_delay throughput [62499.923op/s; 62552.140op/s] or [-0.042%; +0.042%] None None None
trace_buffer/4_senders/1us_delay execution_time [50.166ms; 50.177ms] or [-0.011%; +0.011%] None None None
trace_buffer/4_senders/1us_delay throughput [71745.678op/s; 71761.079op/s] or [-0.011%; +0.011%] None None None
trace_buffer/4_senders/no_delay execution_time [2.447ms; 2.465ms] or [-0.371%; +0.371%] None None None
trace_buffer/4_senders/no_delay throughput [1461160.679op/s; 1472107.657op/s] or [-0.373%; +0.373%] None None None
trace_buffer/8_senders/10us_delay execution_time [57.953ms; 58.031ms] or [-0.067%; +0.067%] None None None
trace_buffer/8_senders/10us_delay throughput [124073.979op/s; 124240.201op/s] or [-0.067%; +0.067%] None None None
trace_buffer/8_senders/1us_delay execution_time [50.280ms; 50.296ms] or [-0.015%; +0.015%] None None None
trace_buffer/8_senders/1us_delay throughput [143153.850op/s; 143197.913op/s] or [-0.015%; +0.015%] None None None
trace_buffer/8_senders/no_delay execution_time [6.575ms; 6.608ms] or [-0.256%; +0.256%] None None None
trace_buffer/8_senders/no_delay throughput [1089852.977op/s; 1095580.441op/s] or [-0.262%; +0.262%] None None None

@bengl
bengl marked this pull request as ready for review July 16, 2026 15:44
@bengl
bengl requested review from a team as code owners July 16, 2026 15:44
@bengl
bengl requested review from vpellan and removed request for a team July 16, 2026 15:44
}) && attrs.iter().any(|attr| {
attr.get("key").and_then(Value::as_str) == Some("_dd.stats_computed")
&& attr.pointer("/value/stringValue").and_then(Value::as_str)
== Some("true")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, why can't the new test be implemented in term of json_body_includes? I'm not sure how it works exactly, but naively it looks like the previous payload could be extended for _dd.stats_computed

@bengl
bengl force-pushed the bengl/otlp-client-computed-stats branch from a5515d2 to 1281505 Compare July 16, 2026 15:58
@dd-octo-sts

dd-octo-sts Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 86.33 MB 86.33 MB +0% (+56 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.88 MB 7.88 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 97.54 MB 97.54 MB +0% (+136 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.62 MB 10.62 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 25.58 MB 25.58 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 89.18 KB 89.18 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 185.48 MB 185.46 MB --.01% (-24.00 KB) 💪
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 958.37 MB 958.37 MB +0% (+128 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.37 MB 8.37 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 89.18 KB 89.18 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 24.76 MB 24.76 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 49.28 MB 49.28 MB +0% (+90 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 22.23 MB 22.23 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 90.58 KB 90.58 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 189.97 MB 189.95 MB -0% (-16.00 KB) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 946.26 MB 946.26 MB +0% (+96 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.47 MB 6.47 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 90.58 KB 90.58 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 26.59 MB 26.59 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 46.90 MB 46.90 MB +0% (+60 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 77.08 MB 77.08 MB +0% (+24 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.83 MB 8.83 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 92.50 MB 92.50 MB -0% (-48 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.74 MB 10.74 MB 0% (0 B) 👌

When tracers compute span metrics outside libdatadog, OTLP trace
exports still need to mark the resource and direct Agent request so the
Agent does not compute the same stats again. Use the tracer metadata flag
in addition to libdatadog's native OTLP stats worker state.
@bengl
bengl force-pushed the bengl/otlp-client-computed-stats branch from 1281505 to 6356aee Compare July 16, 2026 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants