Skip to content

Add MLflow OTLP telemetry exporter, docs, and example - #2112

Merged
rapids-bot[bot] merged 3 commits into
NVIDIA:developfrom
EnesYilmazcode:feat/mlflow-otlp-exporter
Jul 14, 2026
Merged

Add MLflow OTLP telemetry exporter, docs, and example#2112
rapids-bot[bot] merged 3 commits into
NVIDIA:developfrom
EnesYilmazcode:feat/mlflow-otlp-exporter

Conversation

@EnesYilmazcode

@EnesYilmazcode EnesYilmazcode commented Jul 12, 2026

Copy link
Copy Markdown
Contributor
  • Add mlflow telemetry exporter (OTLP to an MLflow tracking server) in nvidia_nat_opentelemetry
  • Document in Observe Workflows (tab + table) with a new MLflow observe guide
  • Add simple_calculator_observability config-mlflow.yml and README section
  • Add unit tests for the routing header and config defaults

Description

Adds an mlflow OTLP telemetry exporter so workflows can send OpenTelemetry traces to an MLflow tracking server. MLflow 3.6+ ingests OTLP/HTTP spans at <tracking-server>/v1/traces and routes them to an experiment via the x-mlflow-experiment-id header (both verified against MLflow source, mlflow/tracing/utils/otlp.py). The exporter mirrors the existing langfuse exporter and reuses OTLPSpanAdapterExporter, following the pattern of the merged Arize AX (#1898) and Weave (#1827) exporters.

Closes #2005

Changes

  • mlflow exporter (MLflowTelemetryExporter + factory) in packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py
  • Example config examples/observability/simple_calculator_observability/configs/config-mlflow.yml + README section and config-table row
  • Docs: docs/source/run-workflows/observe/observe-workflow-with-mlflow.md, plus the platform-table row and provider tab in observe.md
  • Unit test packages/nvidia_nat_opentelemetry/tests/observability/test_mlflow_telemetry_exporter.py

Validation

  • pytest packages/nvidia_nat_opentelemetry/tests/observability/test_mlflow_telemetry_exporter.py passes (2 passed): routing header + config defaults
  • Registration: nat info components -t tracing lists mlflow; the registered factory builds a real OTLPSpanAdapterExporter whose OTLP/HTTP transport carries endpoint http://localhost:5000/v1/traces and header x-mlflow-experiment-id: 0
  • Live: a real OTLP span exported through the mlflow exporter transport persists in a local mlflow server --backend-store-uri sqlite:///mlflow.db (MLflow 3.14.0); search_traces(experiment_ids=["0"]) returns it with the same trace ID, state OK, span name and attributes intact
  • YAML binding: config-mlflow.yml loads through nat.runtime.loader.load_config and parses to MLflowTelemetryExporter (env-var defaults are quoted so ${MLFLOW_EXPERIMENT_ID:-0} stays a string after interpolation)
  • pre-commit (yapf, ruff, markdown-link-check) clean on all changed files; ci/scripts/copyright.py --verify-apache-v2 passes; vale clean on the touched docs
  • Full nat run of the simple_calculator workflow (needs an LLM key; the OTLP-to-MLflow transport is proven by the live check above)

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
    • Any contribution which contains commits that are not Signed-Off will not be accepted.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

Summary by CodeRabbit

  • New Features

    • Added MLflow as a supported tracing and logging backend.
    • Enabled MLflow OTLP/HTTP telemetry exporting with configurable endpoint and experiment routing.
    • Expanded the observability “simple calculator” example to support MLflow.
  • Documentation

    • Added a guide for observing MLflow-tracked workflows with OpenTelemetry.
    • Updated exporter and example documentation to include MLflow setup, configuration, and viewing traces in the MLflow UI.
  • Tests

    • Added unit coverage for MLflow exporter header behavior and default settings.

Signed-off-by: Enes Yilmaz <enesyilmaz5157@gmail.com>
@EnesYilmazcode
EnesYilmazcode requested a review from a team as a code owner July 12, 2026 01:04
@copy-pr-bot

copy-pr-bot Bot commented Jul 12, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jul 12, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: fd93e752-2607-49d6-80d9-4bf37b27e7fa

📥 Commits

Reviewing files that changed from the base of the PR and between edc1d95 and e248911.

📒 Files selected for processing (1)
  • ci/.nim_models_used.json

Walkthrough

Adds MLflow OTLP tracing support, a configured calculator observability example, tests for exporter defaults and headers, documentation covering setup and MLflow UI verification, and an updated model configuration count.

Changes

MLflow tracing integration

Layer / File(s) Summary
MLflow OTLP exporter
packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py, packages/nvidia_nat_opentelemetry/tests/observability/test_mlflow_telemetry_exporter.py
Adds MLflow experiment routing headers, exporter configuration and registration, plus tests for header generation and default fields.
Example workflow configuration
examples/observability/simple_calculator_observability/configs/config-mlflow.yml, examples/observability/simple_calculator_observability/README.md
Adds MLflow telemetry settings, calculator workflow wiring, setup instructions, environment overrides, and configuration documentation.
MLflow integration documentation
docs/source/run-workflows/observe/observe-workflow-with-mlflow.md, docs/source/run-workflows/observe/observe.md
Documents MLflow server setup and trace verification, and adds MLflow to the exporter table and provider tabs.

Model configuration metadata

Layer / File(s) Summary
LLM configuration count
ci/.nim_models_used.json
Increments the recorded configuration count for meta/llama-3.1-70b-instruct.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Workflow
  participant MLflowTelemetryExporter
  participant MLflowServer
  Workflow->>MLflowTelemetryExporter: Emit telemetry spans
  MLflowTelemetryExporter->>MLflowServer: Send OTLP traces with experiment header
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise, imperative, and accurately summarizes the MLflow exporter, docs, and example additions.
Linked Issues check ✅ Passed The PR implements MLflow tracing integration and the requested documentation for issue #2005.
Out of Scope Changes check ✅ Passed The changes are scoped to MLflow tracing support, documentation, and the required config-count update.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/source/run-workflows/observe/observe.md`:
- Around line 231-237: Remove the extra closing MyST fence from the MLflow
tab-item block, leaving the nested include and only the tab-item’s closing :::,
matching the sibling entries in the observe workflow documentation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 38be1d23-5ed9-4e6d-9075-78d9e92a11c7

📥 Commits

Reviewing files that changed from the base of the PR and between e8692d0 and 61b85e9.

📒 Files selected for processing (6)
  • docs/source/run-workflows/observe/observe-workflow-with-mlflow.md
  • docs/source/run-workflows/observe/observe.md
  • examples/observability/simple_calculator_observability/README.md
  • examples/observability/simple_calculator_observability/configs/config-mlflow.yml
  • packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py
  • packages/nvidia_nat_opentelemetry/tests/observability/test_mlflow_telemetry_exporter.py

Comment thread docs/source/run-workflows/observe/observe.md
Signed-off-by: Enes Yilmaz <enesyilmaz5157@gmail.com>
@EnesYilmazcode EnesYilmazcode changed the title feat(observability): MLflow OTLP telemetry exporter, docs, and example Add MLflow OTLP telemetry exporter, docs, and example Jul 12, 2026
@willkill07 willkill07 added feature request New feature or request non-breaking Non-breaking change labels Jul 14, 2026
@willkill07

Copy link
Copy Markdown
Member

/ok to test edc1d95

@willkill07

Copy link
Copy Markdown
Member

@EnesYilmazcode I've approved the PR. Once CI passes I will merge.

If CI does happen to fail, please address the failures.

The new config-mlflow.yml example uses nvidia/nemotron-3-nano-30b-a3b,
which increments its config count from 43 to 44. Regenerate
ci/.nim_models_used.json via model_health_check.py so the
generate-models-list pre-commit hook passes in CI.

Signed-off-by: Enes Yilmaz <enesyilmaz5157@gmail.com>
@willkill07

Copy link
Copy Markdown
Member

/ok to test e248911

@willkill07

Copy link
Copy Markdown
Member

/merge

@rapids-bot
rapids-bot Bot merged commit 5850f4d into NVIDIA:develop Jul 14, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature request New feature or request non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integration with MLFlow for Tracing

2 participants