feat(exporters): replace opencensus with opentelemetry by hakman · Pull Request #1297 · kubernetes/node-problem-detector

hakman · 2026-07-05T16:02:39Z

Based on @daveoy's #1141, adding remaining fixes.
Closes #1008
/cc @SergeyKanzhelev @wangzhen127

kubernetes-prow · 2026-07-05T16:02:43Z

@hakman: GitHub didn't allow me to request PR reviews from the following users: daveoy.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

Details

In response to this:

Based on #1141, adding remaining fixes.
Closes #1008
/cc @SergeyKanzhelev @wangzhen127 @daveoy

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

kubernetes-prow · 2026-07-05T16:02:50Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hakman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [hakman]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

hakman · 2026-07-05T17:37:49Z

/test pull-npd-e2e-test

* feat(exporters): replace opencensus with opentelemetry * fix(stackdriver_exporter): use metrics util constants * chore: use global meter and init * feat(exporters): add resource info, use global meter * chore: update deps * fix: use a gauge directly, narrow scope info * cleanup: deps * fix: try initOnce pattern for global problem metrics * fix(problemmetrics): init is being called too late * fix(gauge): remove _ratio suffix * fix: clean up after rebase * fix: cleanup after rebase * fix: cleanup after rebase * fix: cleanup after rebase * fix: remove nodename * fix(test): remove instance id check * fix(test): add attr values * fix(test): unexported resources * fix(test): initialize global problem metrics manager * fix(lint): make the linter happy

…rics export

…iEndpoint

disk_percent_used declared only device_name but recorded four labels (device_name, fs_type, mount_option, state). With per-metric label validation, Record now rejects the undeclared labels and the metric is dropped from the export (caught by the metriconly e2e). Record only device_name, preserving the schema NPD has exported since the metric was introduced: the OpenCensus view aggregated by device_name and dropped the other tags, so on the wire this metric has always had a single label. It also maps to the Google-owned compute.googleapis.com/guest/disk/percent_used descriptor, which can reject writes carrying undeclared label keys. Add a regression test pinning the schema.

DigitalVeer

Thanks for this! I work with NPD at GKE and I hope I can add my two cents here.

DigitalVeer · 2026-07-17T23:39:14Z

+	// Create Prometheus exporter with options to prevent automatic suffixing
+	promExporter, err := otelprometheus.New(
+		otelprometheus.WithRegisterer(reg),                                   // Use a dedicated registry
+		otelprometheus.WithTranslationStrategy(otlptranslator.NoTranslation), // Don't add suffixes or escape


I recorded the same metric and took a diff on the output between this PR & master.

# any plain scraper w/o escaping curl -s localhost:20257/metrics | grep uptime # host_uptime 12345.6 (same on both branches) # when using Prometheus 3.x curl -s -H 'Accept: text/plain; version=1.0.0; escaping=allow-utf-8' \ localhost:20257/metrics | grep uptime # PR head: {"host/uptime"} 12345.6 <- renamed # master: host_uptime 12345.6 <- unchanged

We should account for users scraping with Prometheus 3.x.

Given that Prometheus 3.x supports utf-8 and has escaping=allow-utf-8 by default, we should make sure the metric sent out is the same before and after this change.

Good catch. Fixed.

DigitalVeer · 2026-07-20T09:17:38Z

+		globalMeterProvider = sdkmetric.NewMeterProvider(opts...)
+
+		// Set as global meter provider
+		otel.SetMeterProvider(globalMeterProvider)


We should pass option.WithTelemetryDisabled() in monitoringClientOptions.

By default, google.golang.org/api adds an otelgrpc handler to each gRPC client. Thus the exporter makes rpc.client.* metrics for its own calls. These metric names are not in NPDMetricToSDMetric.

Cloud Monitoring rejects unknown metrics on export cycles and each failed export makes more rpc.client.* metrics so this can cause a failure loop.

Sounds good, done.

linux-foundation-easycla · 2026-07-22T06:09:48Z

The committers listed above are authorized under a signed CLA.

✅ login: daveoy / name: Dave Young (3781507, 3477a29, 5585dc8, 568b612, 869be6f, 9756747, c6afce7, d670640, e0f0a05, f2bba3e)
✅ login: hakman / name: Ciprian Hacman (181a014, 2b98aa4, 30593ca, 7631035, 8c7b22b, 976d129, 98c2be2, a8951ef, ad515aa, e856a85)

kubernetes-prow Bot requested review from SergeyKanzhelev and wangzhen127 July 5, 2026 16:02

kubernetes-prow Bot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jul 5, 2026

kubernetes-prow Bot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jul 5, 2026

hakman force-pushed the pr-1141-fixes branch 5 times, most recently from a537ec6 to ee90453 Compare July 5, 2026 16:52

hakman force-pushed the pr-1141-fixes branch from 375c801 to f117dbf Compare July 5, 2026 20:18

kubernetes-prow Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 9, 2026

hakman force-pushed the pr-1141-fixes branch from f117dbf to e1f9ca1 Compare July 11, 2026 16:37

kubernetes-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 11, 2026

hakman force-pushed the pr-1141-fixes branch 3 times, most recently from 8284a46 to a12139a Compare July 11, 2026 21:34

daveoy and others added 10 commits July 12, 2026 00:43

fix(test): allow for summary and histogram types to be emit

d670640

fix(otel): remove otel_scope labels

9756747

fix(stackdriver_exporter): filter resource attributes for gcp

5585dc8

fix(metrics): retain early nil return with no name

3477a29

chore(exporters): remove deprecated prometheus opts, use otel translator

568b612

feat(metric): use metricdata for unit tests

e0f0a05

fix(metrics/test): test our New() funcs

f2bba3e

feat(stackdriver_exporter/test): add cloudmock and use it to test met…

c6afce7

…rics export

chore(lint): make the linter happy by not returning some errs

869be6f

hakman added 8 commits July 12, 2026 00:43

fix(stackdriver): restore gce_instance resource, instance_name and ap…

a8951ef

…iEndpoint

fix(prometheus): serve metrics from a dedicated registry

e856a85

fix(metrics): error on unsupported aggregation and undeclared labels

ad515aa

refactor(metrics): dedupe int64/float64 metric implementations

181a014

chore: remove dead code from metrics, otel and cloudmock

98c2be2

refactor(metrics): simplify otelMetric internals

30593ca

fix(prometheus): drop target_info from scrape output

2b98aa4

hakman force-pushed the pr-1141-fixes branch from a12139a to 7631035 Compare July 11, 2026 21:45

hakman added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Jul 11, 2026

DigitalVeer reviewed Jul 20, 2026

View reviewed changes

kubernetes-prow Bot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. and removed cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jul 22, 2026

hakman added 2 commits July 22, 2026 09:10

fix(prometheus): preserve legacy metric names

976d129

fix(stackdriver): disable client telemetry

8c7b22b

hakman force-pushed the pr-1141-fixes branch from ea07947 to 8c7b22b Compare July 22, 2026 06:10

kubernetes-prow Bot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Jul 22, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(exporters): replace opencensus with opentelemetry#1297

feat(exporters): replace opencensus with opentelemetry#1297
hakman wants to merge 20 commits into
kubernetes:masterfrom
hakman:pr-1141-fixes

hakman commented Jul 5, 2026 •

edited

Loading

Uh oh!

kubernetes-prow Bot commented Jul 5, 2026

Uh oh!

kubernetes-prow Bot commented Jul 5, 2026

Uh oh!

hakman commented Jul 5, 2026

Uh oh!

DigitalVeer left a comment

Uh oh!

DigitalVeer Jul 17, 2026

Uh oh!

hakman Jul 22, 2026

Uh oh!

DigitalVeer Jul 20, 2026

Uh oh!

hakman Jul 22, 2026

Uh oh!

linux-foundation-easycla Bot commented Jul 22, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

hakman commented Jul 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kubernetes-prow Bot commented Jul 5, 2026

Uh oh!

kubernetes-prow Bot commented Jul 5, 2026

Uh oh!

hakman commented Jul 5, 2026

Uh oh!

DigitalVeer left a comment

Choose a reason for hiding this comment

Uh oh!

DigitalVeer Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

hakman Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

DigitalVeer Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

hakman Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

linux-foundation-easycla Bot commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hakman commented Jul 5, 2026 •

edited

Loading

linux-foundation-easycla Bot commented Jul 22, 2026 •

edited

Loading