Add MLflow OTLP telemetry exporter, docs, and example - #2112
Conversation
Signed-off-by: Enes Yilmaz <enesyilmaz5157@gmail.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
WalkthroughAdds MLflow OTLP tracing support, a configured calculator observability example, tests for exporter defaults and headers, documentation covering setup and MLflow UI verification, and an updated model configuration count. ChangesMLflow tracing integration
Model configuration metadata
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Workflow
participant MLflowTelemetryExporter
participant MLflowServer
Workflow->>MLflowTelemetryExporter: Emit telemetry spans
MLflowTelemetryExporter->>MLflowServer: Send OTLP traces with experiment header
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/run-workflows/observe/observe.md`:
- Around line 231-237: Remove the extra closing MyST fence from the MLflow
tab-item block, leaving the nested include and only the tab-item’s closing :::,
matching the sibling entries in the observe workflow documentation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 38be1d23-5ed9-4e6d-9075-78d9e92a11c7
📒 Files selected for processing (6)
docs/source/run-workflows/observe/observe-workflow-with-mlflow.mddocs/source/run-workflows/observe/observe.mdexamples/observability/simple_calculator_observability/README.mdexamples/observability/simple_calculator_observability/configs/config-mlflow.ymlpackages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.pypackages/nvidia_nat_opentelemetry/tests/observability/test_mlflow_telemetry_exporter.py
Signed-off-by: Enes Yilmaz <enesyilmaz5157@gmail.com>
|
/ok to test edc1d95 |
|
@EnesYilmazcode I've approved the PR. Once CI passes I will merge. If CI does happen to fail, please address the failures. |
The new config-mlflow.yml example uses nvidia/nemotron-3-nano-30b-a3b, which increments its config count from 43 to 44. Regenerate ci/.nim_models_used.json via model_health_check.py so the generate-models-list pre-commit hook passes in CI. Signed-off-by: Enes Yilmaz <enesyilmaz5157@gmail.com>
|
/ok to test e248911 |
|
/merge |
mlflowtelemetry exporter (OTLP to an MLflow tracking server) innvidia_nat_opentelemetrysimple_calculator_observabilityconfig-mlflow.ymland README sectionDescription
Adds an
mlflowOTLP telemetry exporter so workflows can send OpenTelemetry traces to an MLflow tracking server. MLflow 3.6+ ingests OTLP/HTTP spans at<tracking-server>/v1/tracesand routes them to an experiment via thex-mlflow-experiment-idheader (both verified against MLflow source,mlflow/tracing/utils/otlp.py). The exporter mirrors the existinglangfuseexporter and reusesOTLPSpanAdapterExporter, following the pattern of the merged Arize AX (#1898) and Weave (#1827) exporters.Closes #2005
Changes
mlflowexporter (MLflowTelemetryExporter+ factory) inpackages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.pyexamples/observability/simple_calculator_observability/configs/config-mlflow.yml+ README section and config-table rowdocs/source/run-workflows/observe/observe-workflow-with-mlflow.md, plus the platform-table row and provider tab inobserve.mdpackages/nvidia_nat_opentelemetry/tests/observability/test_mlflow_telemetry_exporter.pyValidation
pytest packages/nvidia_nat_opentelemetry/tests/observability/test_mlflow_telemetry_exporter.pypasses (2 passed): routing header + config defaultsnat info components -t tracinglistsmlflow; the registered factory builds a realOTLPSpanAdapterExporterwhose OTLP/HTTP transport carries endpointhttp://localhost:5000/v1/tracesand headerx-mlflow-experiment-id: 0mlflowexporter transport persists in a localmlflow server --backend-store-uri sqlite:///mlflow.db(MLflow 3.14.0);search_traces(experiment_ids=["0"])returns it with the same trace ID, stateOK, span name and attributes intactconfig-mlflow.ymlloads throughnat.runtime.loader.load_configand parses toMLflowTelemetryExporter(env-var defaults are quoted so${MLFLOW_EXPERIMENT_ID:-0}stays a string after interpolation)pre-commit(yapf, ruff, markdown-link-check) clean on all changed files;ci/scripts/copyright.py --verify-apache-v2passes;valeclean on the touched docsnat runof the simple_calculator workflow (needs an LLM key; the OTLP-to-MLflow transport is proven by the live check above)By Submitting this PR I confirm:
Summary by CodeRabbit
New Features
Documentation
Tests