Skip to content

feat(storage): OpenTelemetry client metrics configuration and options - #14484

Open
nidhiii-27 wants to merge 29 commits into
mainfrom
feat/storage-client-metrics-config
Open

nidhiii-27 wants to merge 29 commits into
mainfrom
feat/storage-client-metrics-config

Conversation

@nidhiii-27

@nidhiii-27 nidhiii-27 commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Description

This PR implements Phase 2 (Configuration & Options API) of the OpenTelemetry Client Metrics feature in the Java Cloud Storage SDK (google-cloud-storage), building on top of PR #14398 (feat/storage-client-metrics-infra).

Key Changes

  1. Public Options API (StorageOptions, GrpcStorageOptions, HttpStorageOptions):

    • Added configuration methods for OpenTelemetry metrics:
      • isEnableOtelMetrics() (default: false)
      • isEnableOtelDebugMetrics() (default: false)
      • getMeterProvider() returning io.opentelemetry.api.metrics.MeterProvider (from opentelemetry-api, avoiding leaking SDK implementation types in public API surface)
      • getMetricInterval() (default: 60 seconds; clamped to 60 seconds when using default Cloud Monitoring exporter)
    • Added builder setters annotated with @BetaApi:
      • setEnableOtelMetrics(boolean)
      • setEnableOtelDebugMetrics(boolean)
      • setMeterProvider(MeterProvider)
      • setMetricInterval(Duration) with precondition check requiring positive, non-zero durations (and >= 60 seconds unless a custom MeterProvider or OpenTelemetry is configured).
  2. Feature Gates & Environment Variables (StorageMetricsConfig):

    • Environment variable gates:
      • GCP_STORAGE_JAVA_ENABLE_OTEL_METRICS
      • GCP_STORAGE_JAVA_ENABLE_OTEL_DEBUG_METRICS
    • Priority resolution: programmatic builder configuration > environment variable > default (false).
    • Case-insensitive boolean parsing matching cross-language SDK standard (1/t/true and 0/f/false).
  3. Serialization Resiliency for Distributed Runtimes:

    • For distributed worker environments (Apache Spark, Apache Beam, Google Cloud Dataflow) where StorageOptions is serialized across worker nodes:
      • Defined meterProvider as transient (non-serializable).
      • Logged a one-time warning in readObject() when deserializing an instance configured with a custom MeterProvider, indicating fallback to the default provider.
      • In getMetricInterval(), clamped interval to minimum 60 seconds if meterProvider == null && !customOpenTelemetryConfigured to ensure compatibility with Cloud Monitoring.
  4. Testing:

    • Added comprehensive tests in StorageOptionsTest:
      • Environment variable gating precedence.
      • Builder validation asserting IllegalArgumentException on null, zero, negative, or invalid (< 60s without custom provider) metric intervals.
      • Roundtrip serialization/deserialization verification (ByteArrayOutputStream / ObjectInputStream) ensuring transient meterProvider handling, fallback defaults, and interval clamping post-deserialization.

[Generated-by: AI]

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces OpenTelemetry client metrics configuration across GrpcStorageOptions, HttpStorageOptions, and StorageOptions, supported by a new StorageMetricsConfig class and comprehensive unit tests. The review feedback highlights critical issues regarding backward compatibility during deserialization, where the newly added metricInterval field could be null and cause a NullPointerException, and recommends declaring the static resolvers in StorageMetricsConfig as volatile to ensure thread safety during parallel test execution.

@nidhiii-27
nidhiii-27 added this pull request to stack #14485 September 23, 2026 10:36

@cpriti-os cpriti-os left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes mainly for serialization behaviour and the enablement configuration (see inline comments).

@nidhiii-27
nidhiii-27 requested a review from cpriti-os October 8, 2026 07:29

@cpriti-os cpriti-os left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing all the previous comments!

@cpriti-os cpriti-os left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once the getMeterProvider() default behavior below is addressed.

@nidhiii-27
nidhiii-27 requested a review from Dhriti07 October 19, 2026 08:13
@nidhiii-27
nidhiii-27 requested a review from Dhriti07 October 19, 2026 13:42
@Dhriti07
Dhriti07 force-pushed the feat/storage-client-metrics-config branch from c21ff47 to f1d51f3 Compare October 20, 2026 05:37
nidhiii-27 added a commit that referenced this pull request Oct 20, 2026
@nidhiii-27
nidhiii-27 requested a review from Dhriti07 October 20, 2026 08:17

@Dhriti07 Dhriti07 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: We repeat the same test cases for both HttpStorageOptions and GrpcStorageOptions across most of StorageOptionsTest. Could we deduplicate these via a small helper or parameterization to reduce duplication across code. This prevents copy-paste drift between HTTP and gRPC and makes future test maintenance easier.

@nidhiii-27
nidhiii-27 requested a review from Dhriti07 October 20, 2026 09:09
…nfiguration

- Add configuration options to StorageOptions, GrpcStorageOptions, and HttpStorageOptions (enableOtelMetrics, enableOtelDebugMetrics, meterProvider, metricInterval).
- Add development gate checking system properties and environment variables in StorageMetricsConfig.
- Implement package-private instrument registry in StorageClientMetrics.
- Update OpenTelemetryBootstrappingUtils to register client views with custom histogram boundaries and create client meter provider.
- Add comprehensive unit tests in StorageOptionsTest, StorageClientMetricsTest, and OpenTelemetryBootstrappingUtilsTest.
Apply spotify fmt-maven-plugin formatting to StorageMetricsConfig, StorageClientMetrics, StorageOptionsTest, and StorageClientMetricsTest.

[Generated-by: AI]
Use simple class name Duration instead of fully qualified java.time.Duration in GrpcStorageOptions and OpenTelemetryBootstrappingUtils.

[Generated-by: AI]
…trappingUtils

Apply spotify fmt-maven-plugin formatting to OpenTelemetryBootstrappingUtils.

[Generated-by: AI]
… metrics

Refactor addClientHistogramView in OpenTelemetryBootstrappingUtils to remove unit selector and custom view description, preserving instrument default descriptions.

Remove deferred Phase 2 connection metrics (dnsLookupDuration, tcpConnectDuration, tlsHandshakeDuration) from StorageClientMetrics, CLIENT_LATENCY_HISTOGRAMS, and StorageClientMetricsTest.

Ensure clean Duration imports.

[Generated-by: AI]
…rProvider

Default api parameter to "grpc" instead of "storage" in createClientMeterProvider, allow specifying "json" / "http" transport protocols for Cloud Monitoring monitored resource storage.googleapis.com/Client, and add unit test coverage in OpenTelemetryBootstrappingUtilsTest.

[Generated-by: AI]
Update createClientMeterProvider to use prefix custom.googleapis.com with default monitored resource mapping.
Add KEY_URL_TEMPLATE attribute key constant in StorageClientMetrics.

[Generated-by: AI]
… options

- In GrpcStorageOptions and HttpStorageOptions:
  - Add null check in getMetricInterval() defaulting to 60 seconds
  - Use getMetricInterval() in equals() and hashCode()
  - Update Javadoc for setMeterProvider to document nullable/default behavior
- In StorageOptions:
  - Update Javadoc for setMeterProvider on Builder
- In StorageMetricsConfig:
  - Make sysPropResolver and envResolver volatile for thread safety
- In StorageOptionsTest:
  - Add unit test verifying deserialization with null metricInterval defaults to 60s and equals/hashCode match
  - Verify explicit client options take precedence over env vars and sysprops

[Generated-by: AI]
Format StorageOptions, GrpcStorageOptions, HttpStorageOptions, and StorageOptionsTest using com.spotify.fmt:fmt-maven-plugin:2.25:format under JDK 17 to resolve CI lint check failure.

[Generated-by: AI]
…iew comments

Clamp metric export interval to at least 60 seconds when meterProvider is null and no custom OpenTelemetry is configured, preventing invalid sub-60s intervals during deserialization when falling back to default Cloud Monitoring exporter.

[Generated-by: AI]
…ig and tests

Deduplicate flag resolution in StorageMetricsConfig by extracting resolveFlag helper, separate multi-behavior tests per go/tott/726, and add input diagnostic messages in looped assertions.

[Generated-by: AI]
@Dhriti07
Dhriti07 force-pushed the feat/storage-client-metrics-config branch from 4c7531d to e7a3588 Compare October 22, 2026 05:06
@nidhiii-27
nidhiii-27 removed this pull request from stack #14485 October 22, 2026 05:45
nidhiii-27 added a commit that referenced this pull request Oct 26, 2026
…iews (#14398)

This PR implements Phase 1 (Core Metrics Registry & Views) of
OpenTelemetry Client Metrics in the Java Cloud Storage SDK
(`google-cloud-storage`).

Following review feedback and architectural alignment across Storage
SDKs, this PR focuses strictly on the package-private instrument
registry and OpenTelemetry view/histogram bootstrapping, decoupling core
infrastructure from public configuration options (handled in follow-up
PR #14484).

## Summary of Changes

- **Instrument Registry (`StorageClientMetrics`)**:
- Implemented package-private registry defining standard and debug
OpenTelemetry instruments (call/request durations, operation and retry
counters, transfer sizes, TTFB, active requests, stall duration, etc.).
- Configured standardized attribute keys and UCUM unit strings (e.g.,
`s`, `By`, `{operation}`, `{attempt}`, `{retry}`).

- **OpenTelemetry Views & Exporter Bootstrapping
(`OpenTelemetryBootstrappingUtils`)**:
- Registered custom explicit-bucket histogram boundaries for latency
(2ms resolution) and size distributions (128KiB to 16GiB).
  - Configured metric name view translation (`.` to `/`).
- Added `createClientMeterProvider` to bootstrap Google Cloud Monitoring
export under `custom.googleapis.com` with client attributes
(`instance_id`, `api`).

- **Testing**:
- Added unit test suites (`StorageClientMetricsTest`,
`OpenTelemetryBootstrappingUtilsTest`) verifying metric registrations,
custom view bucket distributions, and exporter bootstrapping using
`InMemoryMetricReader`.

---
[Generated-by: AI]

---------

Co-authored-by: Dhriti07 <56169283+Dhriti07@users.noreply.github.com>
Base automatically changed from feat/storage-client-metrics-infra to main October 26, 2026 17:28

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants