diff --git a/README.md b/README.md index 0a2f88a..fc75964 100644 --- a/README.md +++ b/README.md @@ -163,6 +163,41 @@ set. | `{PREFIX}SENTRY_PROFILE_LIFECYCLE` | `"trace"` | Profile lifecycle mode: `"trace"` or `"manual"`. | | `{PREFIX}SENTRY_ENABLE_LOGS` | `true` | Forward log records to Sentry. | +#### OpenTelemetry (`{PREFIX}OTEL_`) + +Settings class: `OTelSettings`. OpenTelemetry is only initialised when `ENABLED=true` **and** the +standard `OTEL_EXPORTER_OTLP_ENDPOINT` is set. Each signal is then independently toggleable — +traces and metrics default on, logs off (their volume/cost profile differs). Telemetry is exported +via OTLP/gRPC, e.g. to the internal OTel gateway backing the +[Grafana](https://grafana.aignostics.ai/) stack (Tempo/Loki/Prometheus). + +| Variable | Default | Description | +|---|---|---| +| `{PREFIX}OTEL_ENABLED` | `false` | Master switch for OpenTelemetry export via OTLP. | +| `{PREFIX}OTEL_TRACES_ENABLED` | `true` | Export traces (once `ENABLED`). | +| `{PREFIX}OTEL_METRICS_ENABLED` | `true` | Export metrics (once `ENABLED`). | +| `{PREFIX}OTEL_LOGS_ENABLED` | `false` | Bridge loguru records into OTLP log export (once `ENABLED`). | + +Endpoint, service name, and all other exporter behaviour come from the **standard, unprefixed** +[OpenTelemetry environment variables](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/) +the SDK reads itself — not project-prefixed settings: + +| Variable | Default | Description | +|---|---|---| +| `OTEL_EXPORTER_OTLP_ENDPOINT` | unset | OTLP/gRPC collector endpoint. **Required** — export is skipped if unset. | +| `OTEL_SERVICE_NAME` | project name | Service name attached to all telemetry. Defaults to the `FoundryContext` name. | +| `OTEL_EXPORTER_OTLP_CERTIFICATE` | OS CA bundle, else certifi's | CA file for the exporter's TLS. The Foundry Cloud Run Dockerfile installs the fleet's internal CA into the OS trust store, so this defaults to that bundle (falls back to certifi's public-roots-only bundle if it isn't present, e.g. running locally); set explicitly to override. | +| `OTEL_RESOURCE_ATTRIBUTES` | unset | Extra resource attributes, comma-separated `key=value` pairs. | +| `OTEL_SEMCONV_STABILITY_OPT_IN` | `http` | Opts HTTP instrumentation into the stable semantic conventions (low-cardinality route-template span names) instead of the old, experimental ones. | + +Process-level tracing/metrics/logs are set up by `boot()`. `boot()` also applies default +auto-instrumentors (HTTPX, SQLAlchemy) when traces are enabled — override via +`boot(otel_instrumentors=[...])`, or pass `[]` to opt out. Request-level FastAPI spans are +instrumented automatically too: `init_api()` applies `instrument_fastapi()` to the app it builds +(and to every versioned sub-app) — no explicit call needed. A project that constructs its +`FastAPI` instance some other way can call `instrument_fastapi(app)` directly, once, after +construction. + #### Database (`{PREFIX}DB_`) Settings class: `DatabaseSettings`. Database configuration is only activated when `{PREFIX}DB_URL` diff --git a/docs/decisions/0006-cloud-run-otel-telemetry-export.md b/docs/decisions/0006-cloud-run-otel-telemetry-export.md new file mode 100644 index 0000000..c4b6046 --- /dev/null +++ b/docs/decisions/0006-cloud-run-otel-telemetry-export.md @@ -0,0 +1,252 @@ +# 6. Cloud Run OpenTelemetry telemetry export path + +Date: 2026-07-08 + +Owner: Fabian Spieß · Informed: Platform Engineering, Johannes "Hannes" Zorn, Arne Baumann + +## Status + +Proposed + +Tracked as [OP-3108](https://aignx.atlassian.net/browse/OP-3108) + +Implementation PRs: + +- [gitops#3459](https://github.com/aignostics/gitops/pull/3459) — `aignx-otel-gateway` instance on `shared-tools`, Contour `HTTPProxy`, `k8sAttributes.enabled` toggle. +- [foundry-python-core#96](https://github.com/aignostics/foundry-python-core/pull/96) — `otel_initialize()` in `aignostics_foundry_core.otel`, wired into `boot()`; `instrument_fastapi()` applied automatically by `init_api()`. +- [foundry-python#385](https://github.com/aignostics/foundry-python/pull/385) — unconditional Direct VPC egress in the generated Cloud Run manifest/deploy workflow; installs the fleet's internal CA into the OS trust store via the generated Dockerfile. +- [foundry-infrastructure#10](https://github.com/aignostics/foundry-infrastructure/pull/10) — unconditional `vpc_network`/`vpc_subnetwork` placeholders in `project.hcl`, companion fix for foundry-python#385. + +## Context + +Foundry services running on Cloud Run (e.g. `pviz`) need to send traces, metrics, and logs +into the shared observability stack (Tempo, Loki, Prometheus/Thanos), the same way GKE-hosted +services already do. Today every OTel collector is cluster-internal — nothing is reachable +from Cloud Run, which sits on the shared VPC but outside any GKE cluster's pod network. + +Existing GKE-hosted services already use a single OTLP gRPC endpoint, one OTel push +gateway collector, fanning out to traces/metrics/logs internally and the goal is to give +Foundry Cloud Run services the same experience: one endpoint, no per-signal config, no code +beyond pointing the SDK at it. + +Out of scope per that ticket: +changing telemetry export for existing in-cluster services, and per-signal endpoints. + +This decision spans four repositories: `gitops` (the collector instance, ingress, and cert +wiring), this repo, `foundry-python-core` (how a Foundry service actually emits telemetry), +and the two Copier templates that scaffold every Foundry service, `foundry-python` and +`foundry-infrastructure` (Direct VPC egress reachability, see Network reachability below). + + +**Not up for discussion here**: the `collector-push-gateway` chart itself — a Deployment-mode +OTel push gateway collector with an optional two-tier loadbalancing backend, k8s metadata +enrichment, and fan-out to Tempo/Loki/Prometheus. This is in PEng's standard collector +pattern, already deployed elsewhere in the fleet. That architecture is a given. What this ADR +does cover: creating a new instance of it for Cloud Run, how that instance is exposed +(adding an ingress, since none of the standing instances need one) and specific enhancements +applied to this instance to make it suitable for a non-k8s sources, e.g. disabling `k8sattributes` +enrichment. + +**Also out of scope**: authentication. Neither this new gateway nor any existing collector in +the fleet authenticates callers — the entire telemetry stack chain today trusts network +location alone (reachable only from inside the VPC/cluster pod network). Adding +authentication — e.g. Contour's `HTTPProxy.spec.virtualhost.jwtProviders` (Envoy's `jwt_authn` +filter, which could verify Google-issued Cloud Run service-account identity tokens) +is a valid improvement worth considering, but it would apply to the +whole chain, not just this one gateway, and is out of scope here rather than solved as a +side effect of this decision. + +### Which collector instance Cloud Run talks to + +A new, vpc-only instance of the standard OTel push gateway collector: `collector-push-gateway` +(same chart, unchanged architecture) deployed in `shared-tools`, in loadbalancing mode, +reachable only over the shared VPC via Direct VPC egress, never +publicly. + +### Reachability — Direct VPC connection + +Being in a GCP project attached to the shared VPC does not, by itself, give Cloud Run a path +to reach anything on that VPC. Google's own docs confirm Shared VPC service-project membership +only makes subnets *available* but Cloud Run must be explicitly configured with Direct VPC +egress (a specific network + subnet) to actually route traffic there instead of the public +internet. + +What's already in place: 44 Foundry service-projects have `enable_cloudrun_vpc_access = true` +(`gcp-service-projects` Terraform module), each granting the Cloud Run service agent +`roles/compute.networkUser` on a dedicated per-project/per-env subnet. This is IAM only — it +permits Direct VPC egress, it doesn't configure it on any actual Cloud Run resource. + +What's incomplete on the `foundry-python` template side: +- The generated Cloud Run manifest (`service.template.yaml.jinja`) already carries the Direct + VPC egress annotations (`run.googleapis.com/network-interfaces`, + `vpc-access-egress: private-ranges-only`), but only rendered + if cloud sql or memorystore is enabled, not for general reachability. +- Even where it does render, the subnet values feeding it (`vpc_network`/`vpc_subnetwork` in + the service's own `infrastructure//project.hcl`) are manual placeholders + (`PLACEHOLDER_VPC_NETWORK`/`PLACEHOLDER_VPC_SUBNETWORK`) filled in by hand after onboarding. + +Net effect: reachability isn't a solved problem for any Foundry Cloud Run service today, +Cloud-SQL/Memorystore-enabled or not. This is the largest open risk for this ADR's rollout. + +Resolved direction: Direct VPC egress will be unconditional, every Cloud Run service gets +it regardless of whether it uses Cloud SQL/Memorystore. + +### Alternatives considered — Sidecar collector vs. direct SDK export from Cloud Run + +This is a separate axis from the instance question above: does each Cloud Run service run its +own local collector hop, or export straight to the shared instance? + +**Option A — Sidecar OTel Collector per Cloud Run service** +Each Cloud Run revision runs its own collector as a second container, forwarding OTLP upstream +to the shared instance. Foundry services are built into an image and deployed via the GCP API +from a generated Knative `Service` manifest (`deployment/cloudrun/service.template.yaml.jinja` +in the `foundry-python` Copier template, applied via the `deploy-cloudrun` GitHub Action), and +Cloud Run's multi-container/sidecar support is exposed directly through that manifest — +mechanically adding a second container is straightforward. The real benefit would be scaling. +A sidecar scales automatically alongside each Cloud Run instance, so collector capacity always +tracks the service's own traffic with no separate scaling decision to make. +Metrics are the hard blocker: our stack ingests metrics exclusively via +Prometheus **scrape** (a `ServiceMonitor` targeting a stable in-cluster Service). There is no Thanos +Receive enabled anywhere in the fleet, and we don't run a real Prometheus Pushgateway +either. A sidecar bundled into an ephemeral, non-cluster Cloud Run instance has no stable, +discoverable target for Prometheus to scrape. Its metrics +would simply never be, or with a static target configuration in Prometheus potentially too late, +collected. This rules the option out. + +**Option B (chosen) — Direct SDK export to the shared instance** +Cloud Run services export via the OTel SDK's own OTLP exporter straight to the gateway, no local hop. +This also matches how this same repo already handles error tracking: Sentry is integrated as in-process SDK instrumentation. + +Accepted risk: with no local buffer, telemetry the SDK hasn't flushed yet (spans queued in +`BatchSpanProcessor`, metric points accumulated since the last periodic export) is lost if the +Cloud Run instance is torn down first — scale-to-zero, revision replacement, or a crash. Cloud +Run sends `SIGTERM` with a grace period (10s default, extendable) before `SIGKILL`; registering +a shutdown hook that force-flushes the tracer/meter providers catches most of this, but it's a +mitigation, not a guarantee (an abrupt kill or a grace period shorter than the flush can still +drop data). + +Because every service points at this one gateway endpoint rather than a vendor-specific one, +the export destination is switchable later (e.g. to Grafana Cloud or another vendor) by +reconfiguring only the gateway's own exporters. + +### Alternatives considered — Gateway Ingress + +**Option A — `nginx-internal` Ingress + `GRPCS` backend-protocol annotation** +`nginx-internal` already runs on `shared-tools`. A plain `Ingress` with +`nginx.ingress.kubernetes.io/backend-protocol: "GRPCS"` (plus `proxy-ssl-verify: "off"`, since +the collector's cert isn't issued for the public hostname) would work. +Rejected: ingress-nginx is a deprecated ingress controller company-wide — the org has already +standardized on Contour as the forward-looking ingress technology, so this isn't only about +elegance for this one gateway. + +**Option B (chosen) — Add `contour-internal` to `shared-tools`, use a Contour `HTTPProxy`** +Contour's `HTTPProxy` has a native per-route `services[].protocol: tls` field — no +backend-protocol annotation needed. Required adding `shared-tools` to the `contour-internal` +ApplicationSet (envoy-internal IngressClass), which didn't run there before. + + +### Alternatives considered — k8s metadata enrichment for a non-k8s source + +**Option A — Leave the `k8sattributes` processor always on** +No chart change needed. +Rejected: Cloud Run isn't a Kubernetes workload. The processor can't resolve pod identity for +it and would attach empty or misleading `k8s.pod.name`/`k8s.deployment.name`/`k8s.namespace.name` +labels to Cloud Run telemetry — actively degrading data quality in Tempo/Loki/Prometheus rather +than just being a no-op. + +**Option B (chosen) — Add a `k8sAttributes.enabled` toggle to the `collector-push-gateway` chart** +Defaults to `true` (no behavior change for any existing collector instance); set to `false` +only for the new `aignx-otel-gateway` instance on `shared-tools`. + +### Alternatives considered — how a Foundry Python service instruments itself + +**Option A — `opentelemetry-instrument` CLI wrapper (zero-code)** +Wrap the Cloud Run service's startup command with the `opentelemetry-instrument` launcher; +configure entirely through environment variables. +Rejected: the actual Cloud Run startup command is generated per-service by the +`foundry-python` Copier *template* (`deployment/cloudrun/service.template.yaml.jinja`), which +only affects newly generated or explicitly re-templated (`copier update`) services. +Additionally this kind of instrumentation only provides predefined instrumentation and helps in case of lack of codeownership. + +**Option B (chosen) — Programmatic init inside this repo's `boot()`** +Add `otel_initialize()` (`aignostics_foundry_core.otel`), following the pattern already +established for Sentry: a pydantic-settings-driven `enabled` flag, +called once from `boot()`, which every generated Foundry service already calls before +instantiating FastAPI/uvicorn. One `aignostics-foundry-core` version bump reaches every service +that upgrades its dependency — no per-service code or template changes for process-level +tracing/metrics. +Request-level FastAPI span instrumentation (`instrument_fastapi(app)`) needs the live `app` +object, which doesn't exist yet when `boot()` runs — but `init_api()` (the function that +actually constructs it) does, and always runs after `boot()`, since it's only reachable via +code that imports the project package, which is where `boot()` fires as an `__init__.py` side +effect. So `init_api()` applies it automatically instead of requiring template wiring. + +### Alternatives considered — per-signal export defaults + +**Option A — One `enabled` flag governing traces, metrics, and logs together** +Simplest mental model, matches the original `sentry_initialize()`-style single switch. +Rejected: log volume and cost characteristics differ meaningfully from traces/metrics (far +higher per-request volume in most services). + +**Option B (chosen) — Independent per-signal toggles under a master switch** +`enabled` remains the overall kill switch; `traces_enabled` and `metrics_enabled` default to +`True` once it's on (preserving the original combined behavior as the default), `logs_enabled` +defaults to `False`. A service can disable any single signal (e.g. metrics only) without +affecting the others. + +### GCP-native observability — not a considered alternative + +It's also worth noting this ADR doesn't exclude it, since instrumentation +is configured entirely through the standard `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable. +any individual service remains free to point its SDK at Google's own OTLP-compatible +endpoint instead of our solution. + +## Decision + +Deploy `collector-push-gateway` (loadbalancing mode) on `shared-tools` as `aignx-otel-gateway`, +exposed only internally via a Contour `HTTPProxy` +(reachable from Cloud Run over the shared VPC via Direct VPC egress, never publicly), backed by +an explicit `cert-manager.io/v1 Certificate` and with `k8sAttributes.enabled: false` for this +instance only. + +On the service side, add `otel_initialize()` to `aignostics_foundry_core.boot()`, mirroring the +existing `sentry_initialize()` pattern: a master `{PROJECT}_OTEL_ENABLED` switch, with traces +and metrics independently defaulting on (`traces_enabled`/`metrics_enabled`) once that's set and +`OTEL_EXPORTER_OTLP_ENDPOINT` is configured (standard, unprefixed OTel env vars — not +project-specific ones); logs bridged from loguru via a custom sink but gated behind a separate +`{PROJECT}_OTEL_LOGS_ENABLED` flag, off by default. Each signal can also be disabled +individually. No sidecar collector, no CLI-wrapper instrumentation. The only `foundry-python` +template change is Direct VPC egress becoming unconditional — FastAPI instrumentation itself +needs no template wiring, since `init_api()` in this repo applies it automatically. + +## Consequences + +**Easier**: +- Foundry service owners get parity with existing mono services: one OTLP endpoint, one env + var to point at it, no per-signal configuration. +- Every Foundry Python service gets tracing/metrics support via a single + `aignostics-foundry-core` version bump — no per-service code or template changes. +- The `k8sAttributes.enabled` toggle is reusable for any future non-k8s telemetry source, not + just Cloud Run. +- Adding `contour-internal` to `shared-tools` and using `HTTPProxy` avoids nginx + backend-protocol annotation workarounds and matches how the rest of the fleet already + exposes gRPC-speaking backends. +- Routing Cloud Run metrics through the collector, rather than any hypothetical direct push + straight to a metrics backend, keeps cardinality control and label normalization centralized + (the same `attributes/drop_high_cardinality`/`resource/drop_high_cardinality` processors and + `k8s.cluster.name`-style conventions every other source already goes through) instead of each + new source inventing its own. Same reasoning applies to traces/logs — one place owns what + gets attached to telemetry before it lands in Tempo/Loki/Prometheus, giving every consumer of + that data (dashboards, alerts, on-call) a common, predictable shape regardless of source. + +**Harder / risks**: +- `shared-tools` now runs two ingress controllers (`nginx-internal` and `envoy-internal`) + instead of one — a small increase in operational surface for that cluster. +- Reachability isn't guaranteed for any Foundry Cloud Run service today (see Network + reachability above) — closing that gap is out of scope for this ADR, tracked as follow-up. +- The gateway starts with conservative replica counts (2 push-gateway / 2 loadbalancer-backend) + since this is a new, currently-zero-traffic path — will need revisiting once real traffic + lands. +- Logs being opt-in means a service that only sets `{PROJECT}_OTEL_ENABLED=true` will not see + its logs in Loki via this path — a support/documentation risk if not made clear in onboarding + material (the companion runbook covers this). diff --git a/pyproject.toml b/pyproject.toml index 32a4a10..dea0072 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -55,6 +55,12 @@ dependencies = [ "fastapi>=0.110,<1", "httpx2>=2.2.0,<3", "loguru>=0.7,<1", + "opentelemetry-sdk>=1.27,<2", + "opentelemetry-exporter-otlp-proto-grpc>=1.27,<2", + "opentelemetry-instrumentation-fastapi>=0.48b0,<1", + "opentelemetry-instrumentation-httpx>=0.48b0,<1", + "opentelemetry-instrumentation-sqlalchemy>=0.48b0,<1", + "opentelemetry-resourcedetector-gcp>=1.12.0a0,<2", "PyJWT>=2.10,<3", "platformdirs>=4,<5", "psutil>=6", diff --git a/src/aignostics_foundry_core/api/core.py b/src/aignostics_foundry_core/api/core.py index aa6d48a..6d25ad1 100644 --- a/src/aignostics_foundry_core/api/core.py +++ b/src/aignostics_foundry_core/api/core.py @@ -449,6 +449,12 @@ def init_api( app. **fastapi_kwargs: Extra keyword arguments forwarded to ``FastAPI()``. + Instruments the returned app (and every versioned sub-app, since Starlette + dispatches a mounted sub-app's requests through its own middleware stack, + not the root's) for request-level OTel tracing via + :func:`~aignostics_foundry_core.otel.instrument_fastapi`. No-op if + OpenTelemetry FastAPI instrumentation isn't installed. + Returns: A configured ``FastAPI`` instance. """ @@ -456,6 +462,8 @@ def init_api( from fastapi.exceptions import RequestValidationError # noqa: PLC0415 from pydantic import ValidationError # noqa: PLC0415 + from aignostics_foundry_core.otel import instrument_fastapi # noqa: PLC0415 + api = FastAPI(root_path=root_path, lifespan=lifespan, **fastapi_kwargs) for exc_class, handler in exception_handler_registrations or []: @@ -484,6 +492,8 @@ def init_api( exc_class_or_status_code=ValidationError, handler=validation_exception_handler ) version_app.add_exception_handler(exc_class_or_status_code=Exception, handler=unhandled_exception_handler) + instrument_fastapi(version_app) api.mount(f"/{version_name}", version_app) + instrument_fastapi(api) return api diff --git a/src/aignostics_foundry_core/boot.py b/src/aignostics_foundry_core/boot.py index 191d000..e72e0fd 100644 --- a/src/aignostics_foundry_core/boot.py +++ b/src/aignostics_foundry_core/boot.py @@ -19,6 +19,7 @@ from aignostics_foundry_core.foundry import get_context from aignostics_foundry_core.log import logging_initialize +from aignostics_foundry_core.otel import otel_initialize from aignostics_foundry_core.process import get_process_info from aignostics_foundry_core.sentry import sentry_initialize @@ -37,6 +38,7 @@ from collections.abc import Callable from loguru import Record + from opentelemetry.instrumentation.instrumentor import BaseInstrumentor from sentry_sdk.integrations import Integration from aignostics_foundry_core.foundry import FoundryContext @@ -47,6 +49,7 @@ def boot( context: FoundryContext | None = None, sentry_integrations: list[Integration] | None = None, + otel_instrumentors: list[BaseInstrumentor] | None = None, log_filter: Callable[[Record], bool] | None = None, show_cmdline: bool = True, ) -> None: @@ -61,8 +64,10 @@ def boot( 2. Initialise loguru logging via :func:`~aignostics_foundry_core.log.logging_initialize`. 3. Amend the SSL trust chain with *truststore* and *certifi*. 4. Initialise Sentry via :func:`~aignostics_foundry_core.sentry.sentry_initialize`. - 5. Log a boot message with version, PID, and process information. - 6. Register an atexit shutdown message. + 5. Initialise OpenTelemetry via :func:`~aignostics_foundry_core.otel.otel_initialize`. + 6. Log a boot message with version, PID, process information, and whether + Sentry/OpenTelemetry actually initialised. + 7. Register an atexit shutdown message. Args: context: :class:`~aignostics_foundry_core.foundry.FoundryContext` providing @@ -72,6 +77,11 @@ def boot( is used. sentry_integrations: List of Sentry SDK integrations to register, or ``None`` to skip Sentry initialisation. + otel_instrumentors: List of OTel ``BaseInstrumentor`` instances to apply + (e.g. ``SQLAlchemyInstrumentor``), forwarded to + :func:`~aignostics_foundry_core.otel.otel_initialize`. ``None`` (the + default) uses that function's own best-practice defaults + (currently ``HTTPXClientInstrumentor``); pass ``[]`` to opt out. log_filter: Optional loguru filter callable forwarded to :func:`~aignostics_foundry_core.log.logging_initialize`. show_cmdline: Whether to include the process command line in the @@ -86,11 +96,17 @@ def boot( _parse_env_args(ctx.name) logging_initialize(filter_func=log_filter, context=ctx) _amend_ssl_trust_chain() - sentry_initialize( + sentry_initialized = sentry_initialize( integrations=sentry_integrations, context=ctx, ) - _log_boot_message(context=ctx, show_cmdline=show_cmdline) + otel_initialized = otel_initialize(context=ctx, instrumentors=otel_instrumentors) + _log_boot_message( + context=ctx, + show_cmdline=show_cmdline, + sentry_initialized=sentry_initialized, + otel_initialized=otel_initialized, + ) _register_shutdown_message(project_name=ctx.name, version=ctx.version) logger.trace("Boot sequence completed successfully.") @@ -158,6 +174,8 @@ def _amend_ssl_trust_chain() -> None: def _log_boot_message( context: FoundryContext, show_cmdline: bool = True, + sentry_initialized: bool = False, + otel_initialized: bool = False, ) -> None: """Log a boot message including version, PID, and parent process info. @@ -165,13 +183,18 @@ def _log_boot_message( context: Project context supplying name, version, library mode flag, and project root path. show_cmdline: Whether to append the process command line. + sentry_initialized: Whether :func:`~aignostics_foundry_core.sentry.sentry_initialize` + actually initialised Sentry (``False`` if disabled or misconfigured). + otel_initialized: Whether :func:`~aignostics_foundry_core.otel.otel_initialize` + actually initialised OpenTelemetry (``False`` if disabled or misconfigured). """ process_info = get_process_info(context=context) mode_suffix = ", library-mode" if context.is_library else "" message = ( f"⭐ Booting {context.name} v{context.version} " f"(project root {process_info.project_root}, pid {process_info.pid}), " - f"parent '{process_info.parent.name}' (pid {process_info.parent.pid}){mode_suffix}" + f"parent '{process_info.parent.name}' (pid {process_info.parent.pid}){mode_suffix}, " + f"sentry: {'on' if sentry_initialized else 'off'}, otel: {'on' if otel_initialized else 'off'}" ) if show_cmdline and process_info.cmdline: cmdline_str = " ".join(process_info.cmdline) diff --git a/src/aignostics_foundry_core/otel.py b/src/aignostics_foundry_core/otel.py new file mode 100644 index 0000000..a6bb623 --- /dev/null +++ b/src/aignostics_foundry_core/otel.py @@ -0,0 +1,640 @@ +"""OpenTelemetry integration for application telemetry. + +Mirrors the pattern used by :mod:`aignostics_foundry_core.sentry`: a +pydantic-settings-driven ``enabled`` flag, one initialisation entry-point +called from :func:`~aignostics_foundry_core.boot.boot`, and graceful +degradation when the optional SDK or an OTLP endpoint isn't configured. + +Each signal is independently toggleable: ``traces_enabled`` and +``metrics_enabled`` default to ``True`` once the master ``enabled`` switch is +on, while ``logs_enabled`` defaults to ``False`` (different volume/cost +characteristics, see :class:`OTelSettings`). + +Endpoint, service name, and all other exporter behaviour are read directly +from the standard OpenTelemetry environment variables (``OTEL_EXPORTER_OTLP_ENDPOINT``, +``OTEL_SERVICE_NAME``, ``OTEL_RESOURCE_ATTRIBUTES``, etc. — see +https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/) +rather than project-prefixed settings, so that services already familiar with +the OTel spec don't have to learn a second set of variable names. + +``boot()`` sets up whichever of the process-wide TracerProvider/MeterProvider/ +LoggerProvider are enabled, and FastAPI/ASGI-independent instrumentation, +including bridging loguru records into OTLP logs. Request-level FastAPI +tracing needs :func:`instrument_fastapi` applied to the app instance, which +can't happen here (``boot()`` runs before any FastAPI app is constructed) — +instead, :func:`~aignostics_foundry_core.api.core.init_api` calls it on the +app (and every versioned sub-app) it builds, since that function does +construct the instance and always runs after ``boot()``. + +Beyond FastAPI, :func:`otel_initialize` also applies a list of +``BaseInstrumentor`` instances — unlike ``instrument_fastapi``, these don't +need a live app object, so they can run here. Defaults to +:func:`default_otel_instrumentors` (``HTTPXClientInstrumentor``, since +without it an outbound ``httpx`` call to another Foundry service drops the +W3C ``traceparent`` header and starts a new, disconnected trace at the callee +instead of continuing the caller's; and ``SQLAlchemyInstrumentor``, since +every Foundry "service" gets a SQLAlchemy-backed database layer +unconditionally). Pass ``instrumentors=[]`` to opt out, or a project-built +list (mirroring how ``constants.py`` builds ``SENTRY_INTEGRATIONS``) to opt +into more. + +``OTEL_SEMCONV_STABILITY_OPT_IN`` also defaults to ``"http"``, opting HTTP +instrumentation into the stable semantic conventions instead of the old, +experimental ones both ``HTTPXClientInstrumentor`` and ``FastAPIInstrumentor`` +still default to — the old conventions name a span after the literal request +path (unbounded cardinality for any route with a path parameter), the stable +ones require a low-cardinality route template instead. + +``OTEL_EXPORTER_OTLP_CERTIFICATE`` also defaults to the OS's CA bundle, if +present (falling back to certifi's bundle otherwise), since the OTel +gateway's TLS-facing endpoints are signed by the fleet's internal +``aignx-ca-root-authority`` CA, not a publicly trusted root. The Foundry +Cloud Run Dockerfile installs that CA into the OS trust store via +``update-ca-certificates``, merging it with the distro's own roots into one +file — pointing at that file (rather than assembling a bundle here) keeps +trust independent of any one language/library, and still works for a +service that redirects ``OTEL_EXPORTER_OTLP_ENDPOINT`` at a publicly-signed +vendor. An explicit value for either env var always overrides the default. +""" + +from __future__ import annotations + +import atexit +import logging +import os +import pathlib +from importlib.util import find_spec +from typing import TYPE_CHECKING, Annotated, Any + +from loguru import logger +from pydantic import Field +from pydantic_settings import SettingsConfigDict + +from aignostics_foundry_core.foundry import get_context +from aignostics_foundry_core.settings import OpaqueSettings + +if TYPE_CHECKING: + from collections.abc import Callable + + import fastapi + from loguru import Message, Record + from opentelemetry.instrumentation.instrumentor import BaseInstrumentor + from opentelemetry.sdk._logs import LoggingHandler # pyright: ignore[reportPrivateImportUsage] + from opentelemetry.sdk.resources import Resource + + from aignostics_foundry_core.foundry import FoundryContext + +_OTEL_EXPORTER_OTLP_ENDPOINT = "OTEL_EXPORTER_OTLP_ENDPOINT" +_OTEL_EXPORTER_OTLP_CERTIFICATE = "OTEL_EXPORTER_OTLP_CERTIFICATE" +_OTEL_SERVICE_NAME = "OTEL_SERVICE_NAME" +_OTEL_SEMCONV_STABILITY_OPT_IN = "OTEL_SEMCONV_STABILITY_OPT_IN" + +# Opts HTTP instrumentation (HTTPXClientInstrumentor, FastAPIInstrumentor) into the stable +# semantic conventions instead of the old, experimental ones both still default to. The old +# conventions name a span after the literal request path (e.g. "GET /items/42" — unbounded +# cardinality for any route with a path parameter); the stable conventions require a +# low-cardinality route template instead (e.g. "GET /items/{item_id}"). "http" opts fully into +# the new conventions; "http/dup" would emit both old and new attributes side by side for a +# migration window, which nothing here needs. +_OTEL_SEMCONV_STABILITY_OPT_IN_DEFAULT = "http" + +# The fleet's OTel push-gateway instances (shared-tools, sandbox, ...) serve their +# HTTPProxy/ingress-facing TLS off certs issued by the internal aignx-ca-root-authority +# CA (GoogleCASClusterIssuer, CA pool aignx-ca-pool in aignx-host-project-kah) — not a +# publicly trusted root. In-cluster GKE consumers avoid this by using the collector's +# plaintext otlp/insecure:4319 receiver directly; Cloud Run and anything else reaching +# the gateway through its HTTPProxy over TLS needs this root to validate the connection. +# +# The Foundry Cloud Run Dockerfile installs that CA into the OS trust store via +# update-ca-certificates, which merges it with the distro's own roots into one bundle +# file at the path below — trusting it is then just a matter of pointing +# OTEL_EXPORTER_OTLP_CERTIFICATE there, no per-language cert handling needed (see +# _default_otlp_certificate_setdefault()). Falls back to certifi's bundle (public roots +# only, no internal CA) when that file isn't present, e.g. running outside the container. +_OS_CA_BUNDLE_PATH = "/etc/ssl/certs/ca-certificates.crt" + +# Logger-name prefixes whose records must never reach the OTLP log sink. opentelemetry's +# and grpc's own diagnostics arrive at loguru through log.py's root InterceptHandler; +# re-exporting them feeds OTel's export-failure logs straight back into the log export +# pipeline — an amplification loop that gets worse exactly when the backend is unhealthy. +_OTEL_LOG_SINK_EXCLUDED_LOGGER_PREFIXES = ("opentelemetry", "grpc") + +# Opaque per-instance identifiers that are high-cardinality by construction (a fresh +# random/unique value every time a container or process starts) and carry no debugging +# value on their own — unlike e.g. k8s.pod.name, nothing about them is human-readable or +# stable enough to group on. Left in, they turn into an unbounded label dimension on any +# backend that flattens resource attributes into labels (e.g. the OTel gateway's Prometheus +# exporter with resource_to_telemetry_conversion enabled). Dropped from the shared Resource +# in otel_initialize() before any provider is built, so this holds for traces/metrics/logs +# alike and for every backend, not just ones that happen to filter it back out downstream. +_HIGH_CARDINALITY_RESOURCE_ATTRS = frozenset({ + "service.instance.id", # auto-generated by opentelemetry-sdk's Resource.create() itself + "faas.instance", # set by GoogleCloudResourceDetector for Cloud Run +}) + +# Loguru level names are a superset of stdlib logging's (e.g. TRACE, SUCCESS). +# Map them to the closest stdlib numeric level so OTel's LoggingHandler (which +# expects a stdlib logging.LogRecord) records a sensible severity. +_LOGURU_TO_STDLIB_LEVEL = { + "TRACE": logging.DEBUG, + "DEBUG": logging.DEBUG, + "INFO": logging.INFO, + "SUCCESS": logging.INFO, + "WARNING": logging.WARNING, + "ERROR": logging.ERROR, + "CRITICAL": logging.CRITICAL, +} + + +class OTelSettings(OpaqueSettings): + """Configuration settings for OpenTelemetry integration. + + Reads environment variables using the prefix derived from the active + :class:`~aignostics_foundry_core.foundry.FoundryContext` (e.g. + ``MYPROJECT_OTEL_`` when the context's ``env_prefix`` is ``MYPROJECT_``). + + Only carries the project-scoped ``enabled`` switch — exporter endpoint, + service name, and sampling are configured via the standard unprefixed + ``OTEL_*`` environment variables the OpenTelemetry SDK reads itself. + """ + + model_config = SettingsConfigDict( + env_file_encoding="utf-8", + extra="ignore", + ) + + def __init__(self, **kwargs: Any) -> None: # noqa: ANN401 + """Initialise settings, deriving env_prefix from the active FoundryContext.""" + super().__init__(_env_prefix=f"{get_context().env_prefix}OTEL_", **kwargs) # pyright: ignore[reportCallIssue] + + enabled: Annotated[ + bool, + Field( + description="Master switch for OpenTelemetry export via OTLP. Individual signals are " + "further gated by 'traces_enabled'/'metrics_enabled'/'logs_enabled'.", + default=False, + ), + ] + + traces_enabled: Annotated[ + bool, + Field( + description="Export traces via OTLP. On by default once 'enabled' is set.", + default=True, + ), + ] + + metrics_enabled: Annotated[ + bool, + Field( + description="Export metrics via OTLP. On by default once 'enabled' is set.", + default=True, + ), + ] + + logs_enabled: Annotated[ + bool, + Field( + description=( + "Bridge loguru records into OTLP log export. Off by default even when 'enabled' " + "is set, since log volume/cost characteristics differ from traces/metrics." + ), + default=False, + ), + ] + + +def _default_otlp_certificate_setdefault() -> None: + """Default ``OTEL_EXPORTER_OTLP_CERTIFICATE`` to the OS CA bundle, or certifi's. + + A no-op if the env var is already set — an explicit value, e.g. pointing at + a different vendor's endpoint entirely, always wins. + + ``OTEL_EXPORTER_OTLP_CERTIFICATE`` *replaces* gRPC's trust roots rather than + adding to them, so this needs a bundle that trusts both the internal CA and + any public vendor a service might redirect the endpoint to. The Foundry + Cloud Run Dockerfile installs the internal CA into the OS trust store, + which merges it with the distro's own roots into one file at + :data:`_OS_CA_BUNDLE_PATH` — pointing there covers both, with no + per-language cert handling needed. Falls back to certifi's bundle (public + roots only, no internal CA) when that file isn't present, e.g. running + locally outside the container. + """ + if os.environ.get(_OTEL_EXPORTER_OTLP_CERTIFICATE): + return + + if pathlib.Path(_OS_CA_BUNDLE_PATH).is_file(): + os.environ[_OTEL_EXPORTER_OTLP_CERTIFICATE] = _OS_CA_BUNDLE_PATH + return + + if find_spec("certifi"): + import certifi # noqa: PLC0415 + + os.environ[_OTEL_EXPORTER_OTLP_CERTIFICATE] = certifi.where() + + +def default_otel_instrumentors() -> list[BaseInstrumentor]: + """Build the best-practice default list of OTel auto-instrumentors. + + Mirrors how a project's own ``constants.py`` builds ``SENTRY_INTEGRATIONS``: + try/except-import each optional instrumentation package, degrade + gracefully if it isn't installed. Unlike ``FastAPIInstrumentor``, these + instrumentors patch their target library globally and don't need a live + app instance, so :func:`otel_initialize` can apply them directly. + + ``HTTPXClientInstrumentor``: without it, an outbound ``httpx`` call to + another Foundry service doesn't get the W3C ``traceparent`` header + injected, so the callee's span starts a new, disconnected trace instead + of continuing the caller's — undermining the actual point of exporting + traces at all. + + ``SQLAlchemyInstrumentor``: every Foundry "service" project gets a + SQLAlchemy-backed database layer unconditionally (see + :mod:`aignostics_foundry_core.database`), so DB query spans are as much a + baseline expectation as HTTP ones. Calling ``.instrument()`` with no + ``engine`` argument hooks SQLAlchemy's event system globally — it + instruments any engine created afterward, which fits + :func:`~aignostics_foundry_core.database.init_engine` being called later, + after ``boot()`` (and therefore this) has already run. + + Returns: + list[BaseInstrumentor]: Instantiated (not yet applied) instrumentors + for every optional package found installed. Empty if none are. + """ + instrumentors: list[BaseInstrumentor] = [] + + # Each instrumentation package depends on its target library, so its own presence is + # a sufficient gate — no need to separately probe httpx/sqlalchemy. + if find_spec("opentelemetry.instrumentation.httpx"): + from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor # noqa: PLC0415 + + instrumentors.append(HTTPXClientInstrumentor()) + + if find_spec("opentelemetry.instrumentation.sqlalchemy"): + from opentelemetry.instrumentation.sqlalchemy import SQLAlchemyInstrumentor # noqa: PLC0415 + + instrumentors.append(SQLAlchemyInstrumentor()) + + return instrumentors + + +def _gcp_resource_detect() -> Resource: + """Detect GCP resource attributes (``cloud.*``, ``faas.*``, ``k8s.*``), if any. + + Off GCP (local dev, non-GCP CI), ``GoogleCloudResourceDetector`` returns an empty + ``Resource`` rather than raising, so this is always safe to merge in unconditionally. + + Returns: + Resource: The detected attributes, empty if not running on GCP. + """ + from opentelemetry.resourcedetector.gcp_resource_detector import ( # noqa: PLC0415 + GoogleCloudResourceDetector, + ) + + return GoogleCloudResourceDetector().detect() + + +def _drop_high_cardinality_resource_attrs(resource: Resource) -> Resource: + """Strip :data:`_HIGH_CARDINALITY_RESOURCE_ATTRS` from a fully-merged ``Resource``. + + ``Resource.merge()`` can only overwrite a key's value, not remove it — so the + auto-generated ``service.instance.id``/``faas.instance`` values can't be suppressed + by merging over them, only by rebuilding the resource without them, after every + other source (detectors, ``Resource.create()``'s own defaults) has already merged in. + + Args: + resource: The fully-merged ``Resource`` about to be handed to a provider. + + Returns: + Resource: A copy with the high-cardinality keys removed, same schema_url. + """ + from opentelemetry.sdk.resources import Resource # noqa: PLC0415 + + filtered = {k: v for k, v in resource.attributes.items() if k not in _HIGH_CARDINALITY_RESOURCE_ATTRS} + return Resource(filtered, resource.schema_url) + + +def otel_initialize( + *, + context: FoundryContext | None = None, + instrumentors: list[BaseInstrumentor] | None = None, +) -> bool: + """Initialize OpenTelemetry tracing, metrics, and/or logs. + + Sets up a process-wide ``TracerProvider``, ``MeterProvider``, and/or + ``LoggerProvider`` exporting via OTLP/gRPC, using ``OTEL_EXPORTER_OTLP_ENDPOINT`` + as configured target — independently for each signal, per + :attr:`OTelSettings.traces_enabled`/``metrics_enabled``/``logs_enabled``. + Does nothing (returns ``False``) unless explicitly enabled *and* an OTLP + endpoint is configured, so that services which don't opt in never pay the + cost of provider setup. + + Logs are bridged from loguru (which this project uses instead of stdlib + ``logging``) via a loguru sink that forwards each record into OTel's own + ``LoggingHandler`` — loguru has no first-party OTel integration, so this + is the simplest way to reuse the SDK's own record conversion and trace + correlation instead of duplicating it. + + Args: + context: :class:`~aignostics_foundry_core.foundry.FoundryContext` providing + the project name (used as the ``OTEL_SERVICE_NAME`` fallback) and + version. Falls back to the global context set via + :func:`~aignostics_foundry_core.foundry.set_context`. + instrumentors: ``BaseInstrumentor`` instances to apply once traces are + initialised (gated by :attr:`OTelSettings.traces_enabled`, same as + ``instrument_fastapi``'s purpose). ``None`` (the default) uses + :func:`default_otel_instrumentors`; pass ``[]`` to opt out entirely, + or a longer list to opt into more than the default. + + Returns: + bool: ``True`` if OpenTelemetry was initialised successfully, ``False`` otherwise. + """ + ctx = context or get_context() + + settings = OTelSettings( + _env_file=ctx.env_file, # pyright: ignore[reportCallIssue] + ) + + if not find_spec("opentelemetry.sdk") or not settings.enabled: + logger.trace("OpenTelemetry integration is disabled or opentelemetry.sdk not found, initialization skipped.") + return False + + if not os.environ.get(_OTEL_EXPORTER_OTLP_ENDPOINT): + logger.warning( + "OpenTelemetry integration is enabled but {} is not set, initialization skipped.", + _OTEL_EXPORTER_OTLP_ENDPOINT, + ) + return False + + if not (settings.traces_enabled or settings.metrics_enabled or settings.logs_enabled): + logger.warning( + "OpenTelemetry integration is enabled but every signal " + "(traces/metrics/logs) is disabled, nothing will be exported." + ) + return False + + os.environ.setdefault(_OTEL_SERVICE_NAME, ctx.name) + os.environ.setdefault(_OTEL_SEMCONV_STABILITY_OPT_IN, _OTEL_SEMCONV_STABILITY_OPT_IN_DEFAULT) + _default_otlp_certificate_setdefault() + + from opentelemetry.sdk.resources import Resource # noqa: PLC0415 + + # foundry_service duplicates service.name under an unambiguous label. On metrics + # (via the gateway's resource_to_telemetry_conversion), service.name lands as + # `service_name`, and Prometheus's own scrape-level `job`/`instance` labels (naming + # the collector pod, not this service) shadow the app's same-named resource + # attributes into `exported_job`/`exported_instance` — confusing unless you already + # know that convention. `foundry_service` sidesteps the whole collision. + # + # Merged with GoogleCloudResourceDetector so Cloud Run/GKE/GCE deployments also get + # the standard cloud.*/faas.*/k8s.* resource attributes GCP's own detector fills in + # (it no-ops with an empty Resource off-GCP, e.g. local dev or non-GCP CI). + resource = _drop_high_cardinality_resource_attrs( + _gcp_resource_detect().merge( + Resource.create({"service.version": ctx.version_full, "foundry_service": ctx.name}) + ) + ) + + if settings.traces_enabled and _otel_traces_initialize(resource): + _otel_instrumentors_apply(instrumentors if instrumentors is not None else default_otel_instrumentors()) + + if settings.metrics_enabled: + _otel_metrics_initialize(resource) + + if settings.logs_enabled: + _otel_logs_initialize(resource) + + logger.trace("OpenTelemetry integration initialized, exporting to {}.", os.environ[_OTEL_EXPORTER_OTLP_ENDPOINT]) + + return True + + +def _otel_traces_initialize(resource: Resource) -> bool: + """Set up the OTLP ``TracerProvider``. + + Split out of :func:`otel_initialize` since traces export is gated by + :attr:`OTelSettings.traces_enabled`. + + No-ops if a real (non-proxy) ``TracerProvider`` is already registered — + guards against ``boot()`` running twice in the same process (e.g. in tests, + or a subprocess that inherited the parent's OTel state), which would + otherwise register a second ``BatchSpanProcessor``/exporter pair on top of + the first. Registers an ``atexit`` shutdown so the final batch of spans + still buffered in the ``BatchSpanProcessor`` is flushed on process exit — + on Cloud Run in particular, the container can be frozen or killed shortly + after the last request without this. + + Args: + resource: The shared OTel ``Resource`` used by all providers. + + Returns: + bool: ``True`` if a provider was freshly registered, ``False`` if the + re-initialisation guard skipped setup. The caller uses this to avoid + re-applying instrumentors against an already-instrumented process. + """ + from opentelemetry import trace # noqa: PLC0415 + from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter # noqa: PLC0415 + from opentelemetry.sdk.trace import TracerProvider # noqa: PLC0415 + from opentelemetry.sdk.trace.export import BatchSpanProcessor # noqa: PLC0415 + + if isinstance(trace.get_tracer_provider(), TracerProvider): + logger.trace("OTel TracerProvider already registered, skipping re-initialization.") + return False + + tracer_provider = TracerProvider(resource=resource) + tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter())) + trace.set_tracer_provider(tracer_provider) + atexit.register(tracer_provider.shutdown) + return True + + +def _otel_instrumentors_apply(instrumentors: list[BaseInstrumentor]) -> None: + """Apply each instrumentor's global ``.instrument()``. + + Called after :func:`_otel_traces_initialize` so instrumentors that read + the global ``TracerProvider`` at instrument-time (rather than lazily per + call) pick up the one this module just configured. + + Args: + instrumentors: Instrumentors to apply, e.g. from + :func:`default_otel_instrumentors`. + """ + for instrumentor in instrumentors: + instrumentor.instrument() + logger.trace("Applied OTel instrumentor {}.", type(instrumentor).__name__) + + +def _otel_metrics_initialize(resource: Resource) -> None: + """Set up the OTLP ``MeterProvider``. + + Split out of :func:`otel_initialize` since metrics export is gated by + :attr:`OTelSettings.metrics_enabled`. + + No-ops if a real (non-proxy) ``MeterProvider`` is already registered, and + registers an ``atexit`` shutdown — see :func:`_otel_traces_initialize` for + why both matter. ``PeriodicExportingMetricReader`` only exports on its + timer; without a shutdown flush, whatever accumulated since the last tick + (up to the full export interval) is lost when the process exits. + + Args: + resource: The shared OTel ``Resource`` used by all providers. + """ + from opentelemetry import metrics # noqa: PLC0415 + from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter # noqa: PLC0415 + from opentelemetry.sdk.metrics import MeterProvider # noqa: PLC0415 + from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader # noqa: PLC0415 + + if isinstance(metrics.get_meter_provider(), MeterProvider): + logger.trace("OTel MeterProvider already registered, skipping re-initialization.") + return + + meter_provider = MeterProvider( + resource=resource, + metric_readers=[PeriodicExportingMetricReader(OTLPMetricExporter())], + ) + metrics.set_meter_provider(meter_provider) + atexit.register(meter_provider.shutdown) + + +def _otel_logs_initialize(resource: Resource) -> None: + """Set up the OTLP ``LoggerProvider`` and bridge loguru records into it. + + Split out of :func:`otel_initialize` since logs export is opt-in via + :attr:`OTelSettings.logs_enabled`, off by default even when traces/metrics + are enabled. + + No-ops if a real (non-proxy) ``LoggerProvider`` is already registered, and + registers an ``atexit`` shutdown — see :func:`_otel_traces_initialize`. + Logs are the highest-value signal to lose on an ungraceful exit (they're + often what explains *why* the process is exiting), so this flush matters + most here. + + Args: + resource: The shared OTel ``Resource`` used by the tracer/meter providers. + """ + # opentelemetry-python's logs API is still under a leading-underscore module path + # even in stable releases (see open-telemetry/opentelemetry-python#3565) — noqa/ + # pyright-ignore below are for that upstream naming, not our own code. + from opentelemetry import _logs as logs # noqa: PLC0415, PLC2701 + from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter # noqa: PLC0415, PLC2701 + from opentelemetry.sdk._logs import ( # noqa: PLC0415, PLC2701 + LoggerProvider, # pyright: ignore[reportPrivateImportUsage] + LoggingHandler, # pyright: ignore[reportPrivateImportUsage] + ) + from opentelemetry.sdk._logs.export import ( # noqa: PLC0415, PLC2701 + BatchLogRecordProcessor, # pyright: ignore[reportPrivateImportUsage] + ) + + if isinstance(logs.get_logger_provider(), LoggerProvider): + logger.trace("OTel LoggerProvider already registered, skipping re-initialization.") + return + + logger_provider = LoggerProvider(resource=resource) + logger_provider.add_log_record_processor(BatchLogRecordProcessor(OTLPLogExporter())) + logs.set_logger_provider(logger_provider) + atexit.register(logger_provider.shutdown) + logger.add( + _make_otel_log_sink(LoggingHandler(level=logging.NOTSET, logger_provider=logger_provider)), + filter=_otel_log_sink_filter, + ) + + +def _otel_log_sink_filter(record: Record) -> bool: + """Keep OTel/grpc self-diagnostics out of the OTLP log sink. + + log.py routes all stdlib logging into loguru via a root ``InterceptHandler``, + so opentelemetry's and grpc's own log records (including export failures) + reach loguru with their originating logger name preserved. Forwarding those + back into the OTLP log pipeline would re-export the exporter's own error + logs, amplifying volume precisely when the backend is unhealthy — this drops + them at the sink. + + Args: + record: The loguru record about to be handed to the OTel log sink. + + Returns: + bool: ``False`` for records originating from OpenTelemetry/grpc itself. + """ + return not str(record["name"] or "").startswith(_OTEL_LOG_SINK_EXCLUDED_LOGGER_PREFIXES) + + +def _make_otel_log_sink(handler: LoggingHandler) -> Callable[[Message], None]: + """Build a loguru sink that forwards records to an OTel ``LoggingHandler``. + + Loguru has no first-party OTel integration, so this converts each loguru + record into the stdlib ``logging.LogRecord`` shape ``LoggingHandler.emit`` + expects, reusing the SDK's own conversion/trace-correlation logic instead + of reimplementing it. + + Args: + handler: The OTel ``LoggingHandler`` to forward converted records to. + + Returns: + A callable suitable for passing to ``loguru.logger.add()``. + """ + + def sink(message: Message) -> None: + record = message.record + exc_info = None + if record["exception"] is not None: + exc = record["exception"] + exc_info = (exc.type, exc.value, exc.traceback) + handler.emit( + logging.LogRecord( + name=record["name"] or "", + level=_LOGURU_TO_STDLIB_LEVEL.get(record["level"].name, logging.INFO), + pathname=record["file"].path, + lineno=record["line"], + msg=record["message"], + args=None, + exc_info=exc_info, # pyright: ignore[reportArgumentType] + func=record["function"], + ) + ) + + return sink + + +def instrument_fastapi(app: fastapi.FastAPI) -> bool: + """Instrument a FastAPI app instance for request-level tracing. + + Called automatically by :func:`~aignostics_foundry_core.api.core.init_api` + for the app (and every versioned sub-app) it builds — most projects never + need to call this directly. Exposed for a project that constructs its + ``FastAPI`` instance some other way; in that case, call it once, after the + app is constructed and after :func:`otel_initialize` has run (so the + TracerProvider it sets up is already in place). + + No-ops if no real (non-proxy) ``TracerProvider`` is registered — mirrors the + guard :func:`_otel_traces_initialize` uses, so a service with tracing + disabled (or not yet initialized) doesn't pay for ``OpenTelemetryMiddleware`` + on every request for nothing. Without this, :func:`init_api` calling this + unconditionally would instrument every app regardless of whether OTel is + even enabled, unlike :func:`default_otel_instrumentors`'s HTTPX/SQLAlchemy + instrumentors, which are only applied once :func:`_otel_traces_initialize` + confirms a provider was actually registered. + + Args: + app: The FastAPI application instance to instrument. + + Returns: + bool: ``True`` if instrumentation was applied, ``False`` if the + ``opentelemetry-instrumentation-fastapi`` package isn't installed + or no real ``TracerProvider`` is registered. + """ + if not find_spec("opentelemetry.instrumentation.fastapi"): + logger.trace("opentelemetry-instrumentation-fastapi not found, FastAPI instrumentation skipped.") + return False + + from opentelemetry import trace # noqa: PLC0415 + from opentelemetry.sdk.trace import TracerProvider # noqa: PLC0415 + + if not isinstance(trace.get_tracer_provider(), TracerProvider): + logger.trace("No TracerProvider registered, skipping FastAPI instrumentation.") + return False + + from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor # noqa: PLC0415 + + FastAPIInstrumentor.instrument_app(app) + logger.trace("FastAPI instrumentation applied.") + return True diff --git a/tests/aignostics_foundry_core/api/core_test.py b/tests/aignostics_foundry_core/api/core_test.py index 5b187ca..e7cc820 100644 --- a/tests/aignostics_foundry_core/api/core_test.py +++ b/tests/aignostics_foundry_core/api/core_test.py @@ -368,3 +368,44 @@ def test_init_api_without_versions_unchanged() -> None: mount_routes = [r for r in app.routes if isinstance(r, Mount)] assert mount_routes == [] + + +@pytest.mark.unit +def test_init_api_instruments_root_app(monkeypatch: pytest.MonkeyPatch) -> None: + """init_api applies OTel FastAPI instrumentation to the root app it returns.""" + from unittest.mock import MagicMock + + from aignostics_foundry_core.api.core import init_api + + mock_instrument = MagicMock(return_value=True) + monkeypatch.setattr("aignostics_foundry_core.otel.instrument_fastapi", mock_instrument) + + app = init_api() + + mock_instrument.assert_called_once_with(app) + + +@pytest.mark.unit +def test_init_api_instruments_versioned_apps(monkeypatch: pytest.MonkeyPatch) -> None: + """init_api applies OTel FastAPI instrumentation to every versioned sub-app, not just root.""" + from typing import Any + from unittest.mock import MagicMock, call + + from fastapi import FastAPI + + import aignostics_foundry_core.api.core as core_module + from aignostics_foundry_core.api.core import init_api + + stub_v1 = FastAPI() + stub_v2 = FastAPI() + + def fake_get_versioned(versions: list[str], **_: Any) -> dict[str, FastAPI]: # noqa: ANN401 + return {VERSION_V1: stub_v1, VERSION_V2: stub_v2} + + monkeypatch.setattr(core_module, "get_versioned_api_instances", fake_get_versioned) + mock_instrument = MagicMock(return_value=True) + monkeypatch.setattr("aignostics_foundry_core.otel.instrument_fastapi", mock_instrument) + + app = init_api(versions=[VERSION_V1, VERSION_V2]) + + assert mock_instrument.call_args_list == [call(stub_v1), call(stub_v2), call(app)] diff --git a/tests/aignostics_foundry_core/boot_test.py b/tests/aignostics_foundry_core/boot_test.py index 852839f..0eb951a 100644 --- a/tests/aignostics_foundry_core/boot_test.py +++ b/tests/aignostics_foundry_core/boot_test.py @@ -113,3 +113,24 @@ def test_boot_explicit_context_overrides_global(monkeypatch: pytest.MonkeyPatch) call_ctx = mock_sentry.call_args.kwargs["context"] assert call_ctx.name == _OTHER_PROJECT + + +@pytest.mark.unit +def test_boot_forwards_otel_instrumentors_to_otel_initialize(monkeypatch: pytest.MonkeyPatch) -> None: + """An explicit otel_instrumentors list is forwarded to otel_initialize().""" + monkeypatch.setattr(boot_mod, "_boot_called", False) + monkeypatch.setattr(boot_mod, "logging_initialize", MagicMock()) + monkeypatch.setattr(boot_mod, "sentry_initialize", MagicMock(return_value=False)) + mock_otel = MagicMock(return_value=False) + monkeypatch.setattr(boot_mod, "otel_initialize", mock_otel) + monkeypatch.setattr(boot_mod, "truststore", None) + monkeypatch.setattr(boot_mod, "certifi", None) + + sentinel_instrumentors = [MagicMock()] + boot_mod.boot( + context=make_context(), + sentry_integrations=None, + otel_instrumentors=sentinel_instrumentors, # pyright: ignore[reportArgumentType] + ) + + assert mock_otel.call_args.kwargs["instrumentors"] is sentinel_instrumentors diff --git a/tests/aignostics_foundry_core/otel_test.py b/tests/aignostics_foundry_core/otel_test.py new file mode 100644 index 0000000..1d9eab7 --- /dev/null +++ b/tests/aignostics_foundry_core/otel_test.py @@ -0,0 +1,549 @@ +"""Tests for aignostics_foundry_core.otel.""" + +import logging +import os +import sys +from unittest.mock import ANY, MagicMock, patch + +import pytest +from opentelemetry.sdk.resources import Resource + +from aignostics_foundry_core.foundry import set_context +from aignostics_foundry_core.otel import ( + _OS_CA_BUNDLE_PATH, + OTelSettings, + _default_otlp_certificate_setdefault, + _make_otel_log_sink, + _otel_log_sink_filter, + _otel_logs_initialize, + _otel_metrics_initialize, + _otel_traces_initialize, + default_otel_instrumentors, + instrument_fastapi, + otel_initialize, +) +from tests.conftest import TEST_PROJECT_NAME, TEST_PROJECT_PREFIX, make_context + +_OTEL_PREFIX = f"{TEST_PROJECT_PREFIX}OTEL_" +_OTLP_ENDPOINT = "https://otel-gateway.example.com:4317" +_OTEL_EXPORTER_OTLP_ENDPOINT = "OTEL_EXPORTER_OTLP_ENDPOINT" +_OTEL_EXPORTER_OTLP_CERTIFICATE = "OTEL_EXPORTER_OTLP_CERTIFICATE" +_OTEL_SERVICE_NAME = "OTEL_SERVICE_NAME" +_TRACE_SET_TRACER_PROVIDER = "opentelemetry.trace.set_tracer_provider" +_METRICS_SET_METER_PROVIDER = "opentelemetry.metrics.set_meter_provider" +_LOGS_SET_LOGGER_PROVIDER = "opentelemetry._logs.set_logger_provider" +_OTEL_INSTRUMENTORS_APPLY = "aignostics_foundry_core.otel._otel_instrumentors_apply" + + +@pytest.fixture(autouse=True) +def _clean_otlp_certificate_env(monkeypatch: pytest.MonkeyPatch) -> None: + """Ensure OTEL_EXPORTER_OTLP_CERTIFICATE starts unset for every test in this module. + + otel_initialize() sets this directly via os.environ (not monkeypatch, since it must + outlive the call for the exporter constructors that follow) whenever it defaults it, + so without this fixture one test's default would leak into every later test in the + process, regardless of file, given pytest-randomly reorders tests. + """ + monkeypatch.delenv(_OTEL_EXPORTER_OTLP_CERTIFICATE, raising=False) + + +_LOGURU_LOGGER_ADD = "aignostics_foundry_core.otel.logger.add" + + +@pytest.mark.integration +class TestOtelInitialize: + """Behavioural tests for otel_initialize().""" + + def test_otel_initialize_returns_false_when_disabled(self, monkeypatch: pytest.MonkeyPatch) -> None: + """Returns False when TESTPROJECT_OTEL_ENABLED is not set (default False).""" + monkeypatch.delenv(f"{_OTEL_PREFIX}ENABLED", raising=False) + result = otel_initialize() + assert result is False + + def test_otel_initialize_returns_false_when_sdk_absent(self, monkeypatch: pytest.MonkeyPatch) -> None: + """Returns False when opentelemetry.sdk is not importable (find_spec returns None).""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + with patch("aignostics_foundry_core.otel.find_spec", return_value=None): + result = otel_initialize() + assert result is False + + def test_otel_initialize_returns_false_when_enabled_but_endpoint_unset( + self, monkeypatch: pytest.MonkeyPatch + ) -> None: + """Returns False when enabled but no OTLP endpoint is configured.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.delenv(_OTEL_EXPORTER_OTLP_ENDPOINT, raising=False) + result = otel_initialize() + assert result is False + + def test_otel_initialize_returns_true_and_sets_providers_when_enabled( + self, monkeypatch: pytest.MonkeyPatch + ) -> None: + """Returns True and installs tracer/meter providers when enabled with an endpoint.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + monkeypatch.delenv(_OTEL_SERVICE_NAME, raising=False) + with ( + patch(_TRACE_SET_TRACER_PROVIDER) as mock_set_tracer_provider, + patch(_METRICS_SET_METER_PROVIDER) as mock_set_meter_provider, + patch(_OTEL_INSTRUMENTORS_APPLY), + ): + result = otel_initialize() + assert result is True + mock_set_tracer_provider.assert_called_once() + mock_set_meter_provider.assert_called_once() + + def test_otel_initialize_sets_foundry_service_resource_attribute(self, monkeypatch: pytest.MonkeyPatch) -> None: + """The shared Resource carries foundry_service=, disambiguated from service.name.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + with ( + patch(_TRACE_SET_TRACER_PROVIDER), + patch(_METRICS_SET_METER_PROVIDER), + patch(_OTEL_INSTRUMENTORS_APPLY), + patch("opentelemetry.sdk.resources.Resource.create", wraps=Resource.create) as mock_create, + ): + otel_initialize() + mock_create.assert_called_once_with({"service.version": ANY, "foundry_service": TEST_PROJECT_NAME}) + + def test_otel_initialize_skips_traces_when_traces_disabled(self, monkeypatch: pytest.MonkeyPatch) -> None: + """Tracer provider is not installed when traces_enabled=false, metrics still are.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(f"{_OTEL_PREFIX}TRACES_ENABLED", "false") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + with ( + patch(_TRACE_SET_TRACER_PROVIDER) as mock_set_tracer_provider, + patch(_METRICS_SET_METER_PROVIDER) as mock_set_meter_provider, + ): + result = otel_initialize() + assert result is True + mock_set_tracer_provider.assert_not_called() + mock_set_meter_provider.assert_called_once() + + def test_otel_initialize_skips_metrics_when_metrics_disabled(self, monkeypatch: pytest.MonkeyPatch) -> None: + """Meter provider is not installed when metrics_enabled=false, traces still are.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(f"{_OTEL_PREFIX}METRICS_ENABLED", "false") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + with ( + patch(_TRACE_SET_TRACER_PROVIDER) as mock_set_tracer_provider, + patch(_METRICS_SET_METER_PROVIDER) as mock_set_meter_provider, + patch(_OTEL_INSTRUMENTORS_APPLY), + ): + result = otel_initialize() + assert result is True + mock_set_tracer_provider.assert_called_once() + mock_set_meter_provider.assert_not_called() + + def test_otel_initialize_skips_logs_setup_by_default(self, monkeypatch: pytest.MonkeyPatch) -> None: + """Logs bridge is not installed when only 'enabled' (not 'logs_enabled') is set.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.delenv(f"{_OTEL_PREFIX}LOGS_ENABLED", raising=False) + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + with ( + patch(_TRACE_SET_TRACER_PROVIDER), + patch(_METRICS_SET_METER_PROVIDER), + patch(_LOGS_SET_LOGGER_PROVIDER) as mock_set_logger_provider, + patch(_LOGURU_LOGGER_ADD) as mock_logger_add, + patch(_OTEL_INSTRUMENTORS_APPLY), + ): + result = otel_initialize() + assert result is True + mock_set_logger_provider.assert_not_called() + mock_logger_add.assert_not_called() + + def test_otel_initialize_installs_logs_bridge_when_logs_enabled(self, monkeypatch: pytest.MonkeyPatch) -> None: + """Logs bridge is installed when both 'enabled' and 'logs_enabled' are set.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(f"{_OTEL_PREFIX}LOGS_ENABLED", "true") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + with ( + patch(_TRACE_SET_TRACER_PROVIDER), + patch(_METRICS_SET_METER_PROVIDER), + patch(_LOGS_SET_LOGGER_PROVIDER) as mock_set_logger_provider, + patch(_LOGURU_LOGGER_ADD) as mock_logger_add, + patch(_OTEL_INSTRUMENTORS_APPLY), + ): + result = otel_initialize() + assert result is True + mock_set_logger_provider.assert_called_once() + mock_logger_add.assert_called_once() + + def test_otel_initialize_defaults_service_name_to_context_name(self, monkeypatch: pytest.MonkeyPatch) -> None: + """OTEL_SERVICE_NAME defaults to the context project name when unset.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + monkeypatch.delenv(_OTEL_SERVICE_NAME, raising=False) + ctx = make_context() + with ( + patch(_TRACE_SET_TRACER_PROVIDER), + patch(_METRICS_SET_METER_PROVIDER), + patch(_OTEL_INSTRUMENTORS_APPLY), + ): + otel_initialize(context=ctx) + assert os.environ[_OTEL_SERVICE_NAME] == TEST_PROJECT_NAME + + def test_otel_initialize_does_not_override_explicit_service_name(self, monkeypatch: pytest.MonkeyPatch) -> None: + """An explicitly set OTEL_SERVICE_NAME is left untouched.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + monkeypatch.setenv(_OTEL_SERVICE_NAME, "explicit-service") + with ( + patch(_TRACE_SET_TRACER_PROVIDER), + patch(_METRICS_SET_METER_PROVIDER), + patch(_OTEL_INSTRUMENTORS_APPLY), + ): + otel_initialize() + assert os.environ[_OTEL_SERVICE_NAME] == "explicit-service" + + def test_otel_initialize_returns_false_when_all_signals_disabled(self, monkeypatch: pytest.MonkeyPatch) -> None: + """Returns False (and installs nothing) when enabled but every signal is off.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(f"{_OTEL_PREFIX}TRACES_ENABLED", "false") + monkeypatch.setenv(f"{_OTEL_PREFIX}METRICS_ENABLED", "false") + monkeypatch.setenv(f"{_OTEL_PREFIX}LOGS_ENABLED", "false") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + with ( + patch(_TRACE_SET_TRACER_PROVIDER) as mock_set_tracer_provider, + patch(_METRICS_SET_METER_PROVIDER) as mock_set_meter_provider, + ): + result = otel_initialize() + assert result is False + mock_set_tracer_provider.assert_not_called() + mock_set_meter_provider.assert_not_called() + + +@pytest.mark.integration +class TestDefaultOtelInstrumentors: + """Behavioural tests for default_otel_instrumentors().""" + + def test_returns_httpx_instrumentor_when_package_installed(self) -> None: + """Includes an HTTPXClientInstrumentor when both httpx and its instrumentor are importable.""" + from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor + + instrumentors = default_otel_instrumentors() + assert any(isinstance(i, HTTPXClientInstrumentor) for i in instrumentors) + + def test_returns_sqlalchemy_instrumentor_when_package_installed(self) -> None: + """Includes a SQLAlchemyInstrumentor when both sqlalchemy and its instrumentor are importable.""" + from opentelemetry.instrumentation.sqlalchemy import SQLAlchemyInstrumentor + + instrumentors = default_otel_instrumentors() + assert any(isinstance(i, SQLAlchemyInstrumentor) for i in instrumentors) + + def test_returns_empty_list_when_optional_instrumentation_absent(self) -> None: + """Returns an empty list when no optional instrumentation package is importable.""" + with patch("aignostics_foundry_core.otel.find_spec", return_value=None): + assert default_otel_instrumentors() == [] + + +@pytest.mark.integration +class TestOtelInitializeInstrumentors: + """Behavioural tests for otel_initialize()'s instrumentors handling.""" + + def test_applies_default_instrumentors_when_none_given(self, monkeypatch: pytest.MonkeyPatch) -> None: + """default_otel_instrumentors() is used, and each is instrumented, when instrumentors=None.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + mock_instrumentor = MagicMock() + with ( + patch(_TRACE_SET_TRACER_PROVIDER), + patch(_METRICS_SET_METER_PROVIDER), + patch("aignostics_foundry_core.otel.default_otel_instrumentors", return_value=[mock_instrumentor]), + ): + result = otel_initialize() + assert result is True + mock_instrumentor.instrument.assert_called_once() + + def test_applies_given_instrumentors_instead_of_default(self, monkeypatch: pytest.MonkeyPatch) -> None: + """An explicit instrumentors list is used instead of default_otel_instrumentors().""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + mock_default = MagicMock() + mock_custom = MagicMock() + with ( + patch(_TRACE_SET_TRACER_PROVIDER), + patch(_METRICS_SET_METER_PROVIDER), + patch("aignostics_foundry_core.otel.default_otel_instrumentors", return_value=[mock_default]), + ): + result = otel_initialize(instrumentors=[mock_custom]) + assert result is True + mock_custom.instrument.assert_called_once() + mock_default.instrument.assert_not_called() + + def test_empty_instrumentors_list_opts_out(self, monkeypatch: pytest.MonkeyPatch) -> None: + """Passing instrumentors=[] applies nothing, without falling back to the default.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + mock_default = MagicMock() + with ( + patch(_TRACE_SET_TRACER_PROVIDER), + patch(_METRICS_SET_METER_PROVIDER), + patch("aignostics_foundry_core.otel.default_otel_instrumentors", return_value=[mock_default]), + ): + result = otel_initialize(instrumentors=[]) + assert result is True + mock_default.instrument.assert_not_called() + + def test_instrumentors_not_applied_when_traces_disabled(self, monkeypatch: pytest.MonkeyPatch) -> None: + """Instrumentors are skipped entirely when traces_enabled=false.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(f"{_OTEL_PREFIX}TRACES_ENABLED", "false") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + mock_instrumentor = MagicMock() + with ( + patch(_TRACE_SET_TRACER_PROVIDER), + patch(_METRICS_SET_METER_PROVIDER), + ): + result = otel_initialize(instrumentors=[mock_instrumentor]) + assert result is True + mock_instrumentor.instrument.assert_not_called() + + def test_instrumentors_not_applied_when_tracer_provider_already_registered( + self, monkeypatch: pytest.MonkeyPatch + ) -> None: + """The re-init guard (traces init returns False) means instrumentors aren't re-applied.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + mock_instrumentor = MagicMock() + with ( + patch("aignostics_foundry_core.otel._otel_traces_initialize", return_value=False), + patch(_METRICS_SET_METER_PROVIDER), + ): + result = otel_initialize(instrumentors=[mock_instrumentor]) + assert result is True + mock_instrumentor.instrument.assert_not_called() + + +@pytest.mark.unit +class TestOtelExporterCertificateDefault: + """Behavioural tests for the OTEL_EXPORTER_OTLP_CERTIFICATE default.""" + + def test_defaults_to_os_ca_bundle_when_present(self) -> None: + """Points at the OS CA bundle (installed by the Dockerfile) when it exists.""" + with patch("pathlib.Path.is_file", return_value=True): + _default_otlp_certificate_setdefault() + assert os.environ.get(_OTEL_EXPORTER_OTLP_CERTIFICATE) == _OS_CA_BUNDLE_PATH + + def test_falls_back_to_certifi_when_os_bundle_absent(self) -> None: + """Falls back to certifi's bundle when the OS CA bundle file isn't present.""" + import certifi + + with patch("pathlib.Path.is_file", return_value=False): + _default_otlp_certificate_setdefault() + assert os.environ.get(_OTEL_EXPORTER_OTLP_CERTIFICATE) == certifi.where() + + def test_does_not_override_explicit_value(self, monkeypatch: pytest.MonkeyPatch) -> None: + """An explicitly set OTEL_EXPORTER_OTLP_CERTIFICATE is left untouched.""" + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_CERTIFICATE, "/explicit/ca.pem") + _default_otlp_certificate_setdefault() + assert os.environ[_OTEL_EXPORTER_OTLP_CERTIFICATE] == "/explicit/ca.pem" + + def test_leaves_unset_when_neither_os_bundle_nor_certifi_present(self) -> None: + """Doesn't set the env var if neither the OS bundle nor certifi is available.""" + with ( + patch("pathlib.Path.is_file", return_value=False), + patch("aignostics_foundry_core.otel.find_spec", return_value=None), + ): + _default_otlp_certificate_setdefault() + assert _OTEL_EXPORTER_OTLP_CERTIFICATE not in os.environ + + def test_otel_initialize_sets_certificate_default_when_traces_enabled( + self, monkeypatch: pytest.MonkeyPatch + ) -> None: + """otel_initialize() defaults the certificate as part of its normal startup.""" + monkeypatch.setenv(f"{_OTEL_PREFIX}ENABLED", "true") + monkeypatch.setenv(_OTEL_EXPORTER_OTLP_ENDPOINT, _OTLP_ENDPOINT) + with ( + patch(_TRACE_SET_TRACER_PROVIDER), + patch(_METRICS_SET_METER_PROVIDER), + patch(_OTEL_INSTRUMENTORS_APPLY), + patch("pathlib.Path.is_file", return_value=True), + ): + otel_initialize() + assert os.environ.get(_OTEL_EXPORTER_OTLP_CERTIFICATE) == _OS_CA_BUNDLE_PATH + + +@pytest.mark.integration +class TestOtelProviderReinitGuards: + """Each per-signal init no-ops when a real provider is already registered.""" + + def test_traces_initialize_noops_when_already_registered(self) -> None: + """_otel_traces_initialize returns False when a real TracerProvider is registered.""" + from opentelemetry.sdk.trace import TracerProvider + + with patch("opentelemetry.trace.get_tracer_provider", return_value=TracerProvider()): + assert _otel_traces_initialize(MagicMock()) is False + + def test_metrics_initialize_noops_when_already_registered(self) -> None: + """_otel_metrics_initialize doesn't re-register when a real MeterProvider exists.""" + from opentelemetry.sdk.metrics import MeterProvider + + with ( + patch("opentelemetry.metrics.get_meter_provider", return_value=MeterProvider(metric_readers=[])), + patch(_METRICS_SET_METER_PROVIDER) as mock_set, + ): + _otel_metrics_initialize(MagicMock()) + mock_set.assert_not_called() + + def test_logs_initialize_noops_when_already_registered(self) -> None: + """_otel_logs_initialize doesn't re-register when a real LoggerProvider exists.""" + from opentelemetry.sdk._logs import LoggerProvider + + with ( + patch("opentelemetry._logs.get_logger_provider", return_value=LoggerProvider()), + patch(_LOGS_SET_LOGGER_PROVIDER) as mock_set, + patch(_LOGURU_LOGGER_ADD) as mock_add, + ): + _otel_logs_initialize(MagicMock()) + mock_set.assert_not_called() + mock_add.assert_not_called() + + +@pytest.mark.integration +class TestOtelSettings: + """Behavioural tests for OTelSettings.""" + + def test_otel_settings_default_disabled(self) -> None: + """OpenTelemetry is disabled by default (no env vars set).""" + settings = OTelSettings() # pyright: ignore[reportCallIssue] + assert settings.enabled is False + + def test_otel_settings_logs_default_disabled(self) -> None: + """Logs bridging is disabled by default, independent of 'enabled'.""" + settings = OTelSettings() # pyright: ignore[reportCallIssue] + assert settings.logs_enabled is False + + def test_otel_settings_traces_and_metrics_default_enabled(self) -> None: + """Traces and metrics default to enabled, independent of the master 'enabled' switch.""" + settings = OTelSettings() # pyright: ignore[reportCallIssue] + assert settings.traces_enabled is True + assert settings.metrics_enabled is True + + def test_otel_settings_uses_context_env_prefix(self, monkeypatch: pytest.MonkeyPatch) -> None: + """OTelSettings reads env vars from the prefix supplied by FoundryContext.""" + set_context(make_context()) + monkeypatch.setenv(f"{TEST_PROJECT_PREFIX}OTEL_ENABLED", "true") + settings = OTelSettings() # pyright: ignore[reportCallIssue] + assert settings.enabled is True + + +def _make_loguru_message( + *, + level_name: str = "INFO", + message: str = "hello", + exception: object = None, +) -> MagicMock: + """Build a minimal fake loguru Message with a `.record` dict for sink tests.""" + level = MagicMock() + level.name = level_name + file_ = MagicMock() + file_.path = "app.py" + msg = MagicMock() + msg.record = { + "name": "my_module", + "level": level, + "file": file_, + "line": 42, + "message": message, + "function": "my_func", + "exception": exception, + } + return msg + + +@pytest.mark.unit +class TestMakeOtelLogSink: + """Behavioural tests for _make_otel_log_sink().""" + + def test_sink_emits_stdlib_log_record_from_loguru_message(self) -> None: + """Sink converts a loguru message into a stdlib LogRecord and emits it via the handler.""" + handler = MagicMock() + sink = _make_otel_log_sink(handler) + sink(_make_loguru_message(level_name="WARNING", message="disk almost full")) + handler.emit.assert_called_once() + record = handler.emit.call_args.args[0] + assert isinstance(record, logging.LogRecord) + assert record.name == "my_module" + assert record.levelno == logging.WARNING + assert record.getMessage() == "disk almost full" + assert record.funcName == "my_func" + assert record.exc_info is None + + def test_sink_maps_loguru_only_levels_to_stdlib(self) -> None: + """TRACE and SUCCESS (loguru-only levels) map to the closest stdlib level.""" + handler = MagicMock() + sink = _make_otel_log_sink(handler) + + sink(_make_loguru_message(level_name="TRACE")) + assert handler.emit.call_args.args[0].levelno == logging.DEBUG + + sink(_make_loguru_message(level_name="SUCCESS")) + assert handler.emit.call_args.args[0].levelno == logging.INFO + + def test_sink_includes_exc_info_when_exception_present(self) -> None: + """An exception on the loguru record is forwarded as exc_info. + + Raises: + ValueError: deliberately, to obtain a real exc_info tuple. + """ + handler = MagicMock() + sink = _make_otel_log_sink(handler) + error_message = "boom" + try: + raise ValueError(error_message) # noqa: TRY301 + except ValueError: + exc_type, exc_value, exc_tb = sys.exc_info() + exception = MagicMock(type=exc_type, value=exc_value, traceback=exc_tb) + + sink(_make_loguru_message(exception=exception)) + record = handler.emit.call_args.args[0] + assert record.exc_info is not None + assert record.exc_info[1] is exc_value + + +@pytest.mark.unit +class TestOtelLogSinkFilter: + """Behavioural tests for _otel_log_sink_filter().""" + + @pytest.mark.parametrize("name", ["opentelemetry.sdk.trace.export", "grpc._channel"]) + def test_excludes_otel_and_grpc_records(self, name: str) -> None: + """Records originating from opentelemetry/grpc are dropped to avoid a feedback loop.""" + assert _otel_log_sink_filter({"name": name}) is False # pyright: ignore[reportArgumentType] + + @pytest.mark.parametrize("name", ["my_module", "aignostics_foundry_core.otel", None]) + def test_keeps_application_records(self, name: str | None) -> None: + """Application records (and nameless ones) are forwarded to the OTLP sink.""" + assert _otel_log_sink_filter({"name": name}) is True # pyright: ignore[reportArgumentType] + + +@pytest.mark.unit +class TestInstrumentFastapi: + """Behavioural tests for instrument_fastapi().""" + + def test_instrument_fastapi_returns_false_when_package_absent(self) -> None: + """Returns False when opentelemetry-instrumentation-fastapi is not importable.""" + with patch("aignostics_foundry_core.otel.find_spec", return_value=None): + result = instrument_fastapi(MagicMock()) + assert result is False + + def test_instrument_fastapi_returns_true_and_instruments_app(self) -> None: + """Returns True and calls FastAPIInstrumentor.instrument_app with the given app.""" + from opentelemetry.sdk.trace import TracerProvider + + mock_app = MagicMock() + with ( + patch("opentelemetry.trace.get_tracer_provider", return_value=TracerProvider()), + patch("opentelemetry.instrumentation.fastapi.FastAPIInstrumentor.instrument_app") as mock_instrument, + ): + result = instrument_fastapi(mock_app) + assert result is True + mock_instrument.assert_called_once_with(mock_app) + + def test_instrument_fastapi_noops_when_no_tracer_provider_registered(self) -> None: + """Returns False and skips instrumentation when no real TracerProvider is registered.""" + with patch("opentelemetry.instrumentation.fastapi.FastAPIInstrumentor.instrument_app") as mock_instrument: + result = instrument_fastapi(MagicMock()) + assert result is False + mock_instrument.assert_not_called() diff --git a/uv.lock b/uv.lock index 04d270e..f64eb2a 100644 --- a/uv.lock +++ b/uv.lock @@ -1,6 +1,11 @@ version = 1 revision = 3 requires-python = ">=3.11, <3.15" +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version < '3.13'", +] [manifest] overrides = [ @@ -21,6 +26,12 @@ dependencies = [ { name = "httpx2" }, { name = "loguru" }, { name = "nicegui" }, + { name = "opentelemetry-exporter-otlp-proto-grpc" }, + { name = "opentelemetry-instrumentation-fastapi" }, + { name = "opentelemetry-instrumentation-httpx" }, + { name = "opentelemetry-instrumentation-sqlalchemy" }, + { name = "opentelemetry-resourcedetector-gcp" }, + { name = "opentelemetry-sdk" }, { name = "platformdirs" }, { name = "psutil" }, { name = "pydantic" }, @@ -78,6 +89,12 @@ requires-dist = [ { name = "httpx2", specifier = ">=2.2.0,<3" }, { name = "loguru", specifier = ">=0.7,<1" }, { name = "nicegui", specifier = ">=3,<4" }, + { name = "opentelemetry-exporter-otlp-proto-grpc", specifier = ">=1.27,<2" }, + { name = "opentelemetry-instrumentation-fastapi", specifier = ">=0.48b0,<1" }, + { name = "opentelemetry-instrumentation-httpx", specifier = ">=0.48b0,<1" }, + { name = "opentelemetry-instrumentation-sqlalchemy", specifier = ">=0.48b0,<1" }, + { name = "opentelemetry-resourcedetector-gcp", specifier = ">=1.12.0a0,<2" }, + { name = "opentelemetry-sdk", specifier = ">=1.27,<2" }, { name = "platformdirs", specifier = ">=4,<5" }, { name = "psutil", specifier = ">=6" }, { name = "pydantic", specifier = ">=2,<3" }, @@ -336,6 +353,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/ed/c9/d7977eaacb9df673210491da99e6a247e93df98c715fc43fd136ce1d3d33/arrow-1.4.0-py3-none-any.whl", hash = "sha256:749f0769958ebdc79c173ff0b0670d59051a535fa26e8eba02953dc19eb43205", size = 68797, upload-time = "2025-10-18T17:46:45.663Z" }, ] +[[package]] +name = "asgiref" +version = "3.11.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/63/40/f03da1264ae8f7cfdbf9146542e5e7e8100a4c66ab48e791df9a03d3f6c0/asgiref-3.11.1.tar.gz", hash = "sha256:5f184dc43b7e763efe848065441eac62229c9f7b0475f41f80e207a114eda4ce", size = 38550, upload-time = "2026-02-03T13:30:14.33Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5c/0a/a72d10ed65068e115044937873362e6e32fab1b7dce0046aeb224682c989/asgiref-3.11.1-py3-none-any.whl", hash = "sha256:e8667a091e69529631969fd45dc268fa79b99c92c5fcdda727757e52146ec133", size = 24345, upload-time = "2026-02-03T13:30:13.039Z" }, +] + [[package]] name = "asyncpg" version = "0.31.0" @@ -1232,6 +1258,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/9a/9a/e35b4a917281c0b8419d4207f4334c8e8c5dbf4f3f5f9ada73958d937dcc/frozenlist-1.8.0-py3-none-any.whl", hash = "sha256:0c18a16eab41e82c295618a77502e17b195883241c563b00f0aa5106fc4eaa0d", size = 13409, upload-time = "2025-10-06T05:38:16.721Z" }, ] +[[package]] +name = "googleapis-common-protos" +version = "1.75.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "protobuf" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/b5/c8/f439cffde755cffa462bfbb156278fa6f9d09119719af9814b858fd4f81f/googleapis_common_protos-1.75.0.tar.gz", hash = "sha256:53a062ff3c32552fbd62c11fe23768b78e4ddf0494d5e5fd97d3f4689c75fbbd", size = 151035, upload-time = "2026-05-07T08:04:49.423Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e7/c8/e2645aa8ed02fd4c7a2f59d68783b65b1f3cbdfe39a6308e156509d1fee8/googleapis_common_protos-1.75.0-py3-none-any.whl", hash = "sha256:961ed60399c457ceb0ee8f285a84c870aabc9c6a832b9d37bb281b5bebde43ed", size = 300631, upload-time = "2026-05-07T08:03:30.345Z" }, +] + [[package]] name = "greenlet" version = "3.5.1" @@ -1289,6 +1327,57 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/57/f3/97dfcf4a6eb5077f8a672234216fb5923eb89f2cab7081cb10b2cf75b605/greenlet-3.5.1-cp314-cp314t-win_amd64.whl", hash = "sha256:45718441607f9325d948db98cbc691276059316d0358c188c246da4e1d4d23d2", size = 245246, upload-time = "2026-05-20T13:12:22.646Z" }, ] +[[package]] +name = "grpcio" +version = "1.82.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/b6/19/e29d3979b420b92d516ee97f0bff4fb88cbb3d724791318f50beb55db549/grpcio-1.82.0.tar.gz", hash = "sha256:bfe3247e0bb598585ce26565730970d21bccf3c9dc547dc6703a1a3c7888e8e1", size = 13184479, upload-time = "2026-07-06T04:22:54.421Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b7/ed/dfba8843c9e0cbf5b3ad33998fab400a6290e29432c5a2c94e684ebbfadf/grpcio-1.82.0-cp311-cp311-linux_armv7l.whl", hash = "sha256:64be5bea5f32b1b13c81d8f322bd49ab22d388b87b5c146050c4faf2769c2036", size = 6181469, upload-time = "2026-07-06T04:21:04.208Z" }, + { url = "https://files.pythonhosted.org/packages/ce/41/fe3cc2de8ccdf0b6a8a9939d731c8bdab37f80c96581237de359baa37bbe/grpcio-1.82.0-cp311-cp311-macosx_11_0_universal2.whl", hash = "sha256:03c0f3085e30e51aa4592a2a0b8ee518c9fb1593eb0368f57d7ccfdda2a1cff1", size = 11971108, upload-time = "2026-07-06T04:21:06.634Z" }, + { url = "https://files.pythonhosted.org/packages/27/49/896d665121bd0a806d62a5bacf93c0db2f7097b29bc52bfab76141589f68/grpcio-1.82.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:bc428d9825f1f83e61c65c8e6fb662384105022231938d6775e1b6cfb2bf517a", size = 6760133, upload-time = "2026-07-06T04:21:09.779Z" }, + { url = "https://files.pythonhosted.org/packages/78/1a/49739ab1be3f66106f54af46da599f2be9fdb79d4fc52e4d5a43a73cfc92/grpcio-1.82.0-cp311-cp311-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:4a9389c1b3568b1d6b8c1d85d43eea701814e59280dd955ea0a2d2d65b90cb24", size = 7484373, upload-time = "2026-07-06T04:21:11.942Z" }, + { url = "https://files.pythonhosted.org/packages/d4/a9/53cc88d792b318ec8ed924791cde109096a4174e6e2c115231ece5b69a9a/grpcio-1.82.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:848774370f0783f88e1fa9888f972fa61ab1ec985fffe2668f53bccfa51ef6cb", size = 6924263, upload-time = "2026-07-06T04:21:14.864Z" }, + { url = "https://files.pythonhosted.org/packages/f3/23/d4ea910fcfbd62ad4a7efde95f0fbfec9fe99595be9b58a1a6d67c57c4ff/grpcio-1.82.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:743390c02fb9b565351fc4429b3385c84d851626df11d91a537636b02eaa67df", size = 7531845, upload-time = "2026-07-06T04:21:17.674Z" }, + { url = "https://files.pythonhosted.org/packages/f0/a6/bc6503929c332ae9c74ccf74c57fb67c6b2d9e6ed27bfddb0888d812df52/grpcio-1.82.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:b53959e9af6618b10b46dba4ac82706c4ee90443e95a28004c014c2532d7e053", size = 8568207, upload-time = "2026-07-06T04:21:19.868Z" }, + { url = "https://files.pythonhosted.org/packages/d1/a5/7529b811efd1cb3181bc789d22ec9403b27aa3d3d8acb0b122987036143a/grpcio-1.82.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:8045b89d863b5a271e65413361f75dddd70eeab6bf79f2b0e0eaeeb0900d8d24", size = 7938767, upload-time = "2026-07-06T04:21:22.597Z" }, + { url = "https://files.pythonhosted.org/packages/5f/b3/1fa12730f95458fba4319b7935fc4c02a604404ea27f74756c51bf73a263/grpcio-1.82.0-cp311-cp311-win32.whl", hash = "sha256:24310aaf1f96a5327fc0d8e00fc98f0e9f90be8e09cc51becd2553d4b823d286", size = 4256371, upload-time = "2026-07-06T04:21:25.157Z" }, + { url = "https://files.pythonhosted.org/packages/60/10/71202ae4c1cade358a07436c010c145210bc2a730a731e5d297a2dac4f6e/grpcio-1.82.0-cp311-cp311-win_amd64.whl", hash = "sha256:ddcd5f8db890c5c822e4c0b89d641b1db6a0c7255fff02535cbe77c4dabf450f", size = 5009634, upload-time = "2026-07-06T04:21:27.282Z" }, + { url = "https://files.pythonhosted.org/packages/2f/e6/9ce68bc431224177b6f506cfb4896a742119c51255143069970955b6510b/grpcio-1.82.0-cp312-cp312-linux_armv7l.whl", hash = "sha256:03cce11532da292215cd8e5f444de91982e39dfb41a916e0e0f91a5c128588ea", size = 6144680, upload-time = "2026-07-06T04:21:29.808Z" }, + { url = "https://files.pythonhosted.org/packages/59/93/00ecf28e1be7a70b4b4d148d3a551b6c9a37024a669c3650a2c374c64026/grpcio-1.82.0-cp312-cp312-macosx_11_0_universal2.whl", hash = "sha256:01336fe31d1ea5d5154b0d803eb3584e69fb96e0e657acc87bd4d48d6abc9587", size = 11952213, upload-time = "2026-07-06T04:21:32.347Z" }, + { url = "https://files.pythonhosted.org/packages/38/05/9be6bf4cddb5b5f39c0748cdf5639525cd4116a157c8f3802c9217e54882/grpcio-1.82.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:fe56f1bea709ddb2531fbd510a2dd6040e993985507c3b2bf3404507aee989b1", size = 6710770, upload-time = "2026-07-06T04:21:35.382Z" }, + { url = "https://files.pythonhosted.org/packages/00/97/62c0969e993673ebef16d526db2504f38bc4c73cf2593c28746791ab7956/grpcio-1.82.0-cp312-cp312-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:77a5d8fe331376c7aaa4e06e903a43bd144de4f98c6e414e13cbc5d64e5aa61e", size = 7450671, upload-time = "2026-07-06T04:21:37.923Z" }, + { url = "https://files.pythonhosted.org/packages/67/1f/d83d9fbaaef2b6ebbf70a6e467000f13f92b4cf0b84c561c162b43128439/grpcio-1.82.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:9ce80ea66c803398fde9c5b1fd925920c9742da7f10f8b85acac0adac3e4d7e0", size = 6886855, upload-time = "2026-07-06T04:21:40.527Z" }, + { url = "https://files.pythonhosted.org/packages/c5/36/5ed2e28b8c5f00599c7d5e94d5f82c32cf73a6fa42d13c2900cb37c861ec/grpcio-1.82.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:4665dc8c9ec30acff455e2b15c47b825f18647c555483aa1ccf670adead8adcc", size = 7501322, upload-time = "2026-07-06T04:21:43.78Z" }, + { url = "https://files.pythonhosted.org/packages/89/4e/69113709e14e24289940d6aef067b797c73c8c96f580683af7d5f09b22b0/grpcio-1.82.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:4405d19ecfc7a8caa9043ff5a651e1871b54fc620f917de5dede7e6fb78e9e14", size = 8536896, upload-time = "2026-07-06T04:21:46.387Z" }, + { url = "https://files.pythonhosted.org/packages/30/f6/08bd6eb89a558f0f59dacaef747afda7c21e2c2ecf138ae2ec533dea8aeb/grpcio-1.82.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:30e580c19178609799660f52b1e1eb09248969be20dc44caaa4dcaa3e8450dd9", size = 7913890, upload-time = "2026-07-06T04:21:49.733Z" }, + { url = "https://files.pythonhosted.org/packages/d5/a1/04b467e2ffc93f8e50b660bc6f47c3c8e9ba7603104a741e3d5663a261d4/grpcio-1.82.0-cp312-cp312-win32.whl", hash = "sha256:dbbd83dbaa856b387a4eba74ec3add7f594f090d3b4d390d829dd5834816aae4", size = 4240969, upload-time = "2026-07-06T04:21:52.281Z" }, + { url = "https://files.pythonhosted.org/packages/25/3c/15d3601384d5937e8d9fbe6d8e7b8171008d649744dd7b6c864932937ec9/grpcio-1.82.0-cp312-cp312-win_amd64.whl", hash = "sha256:73176d270699d76a9dc3753f25e01c19fb98f830f826a506e917d83629f1b39b", size = 5001575, upload-time = "2026-07-06T04:21:54.839Z" }, + { url = "https://files.pythonhosted.org/packages/f0/c8/7aace81e7cd12c6ffccf0d47ac389a3f19e5280e9ab04d301d02855ecd25/grpcio-1.82.0-cp313-cp313-linux_armv7l.whl", hash = "sha256:841bcaa24882921d41cd7a82118f9a766edcf8807c2f486e7deeb0ff5ea36181", size = 6146066, upload-time = "2026-07-06T04:21:57.527Z" }, + { url = "https://files.pythonhosted.org/packages/10/17/b448f2265e4927188425c0b09fb943c39b50f7f53fbead644b44ccdf834b/grpcio-1.82.0-cp313-cp313-macosx_11_0_universal2.whl", hash = "sha256:d4bb7219c39c735e6573ae3c33aeafb2a4e1f1cc93a762f7899c283defaed365", size = 11948617, upload-time = "2026-07-06T04:22:00.066Z" }, + { url = "https://files.pythonhosted.org/packages/8e/a3/fba54e50fffc9f74b1ffd5eb4e47310e6650557ef136a165f1fe7df03a65/grpcio-1.82.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:254230d2cb773a87380858d0fc74653b4761bf2013c83fe1c5d77ed8f241fc9d", size = 6714591, upload-time = "2026-07-06T04:22:04.04Z" }, + { url = "https://files.pythonhosted.org/packages/29/6f/f0ec3f1a61a746dc7037a737ba4cbe60959645311ac8542bb1ad4e72ae8b/grpcio-1.82.0-cp313-cp313-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:16d4f215344057eb21038319b05a0c531306230c17b075fa8461a944581006dd", size = 7454986, upload-time = "2026-07-06T04:22:06.776Z" }, + { url = "https://files.pythonhosted.org/packages/b6/11/9d6ce94465d6a7f92c413430b3e77f0879b39427a217ffb3d23585df4bb3/grpcio-1.82.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:a7c283b459151a2e84df32b28c31562ab3e7f624bccffce176c5608565b0212f", size = 6888623, upload-time = "2026-07-06T04:22:09.469Z" }, + { url = "https://files.pythonhosted.org/packages/51/1a/b887adb7abb69081c4f24bc66c3612975e87caf5f7c6aa3916bb82d42e5f/grpcio-1.82.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:96a3be4d2a482ff004c036f6391f33553b36a7627138d1ca3ecc1e70d55d8af5", size = 7505067, upload-time = "2026-07-06T04:22:11.728Z" }, + { url = "https://files.pythonhosted.org/packages/60/94/90f11c0f227ac653cff769e05c7f279da945e134c9351a1c37d475714686/grpcio-1.82.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:bc481e6c7e6c8721797f2ab764092c843fd5cea06d4c7e9f4c3e3b7ecf331eb0", size = 8535382, upload-time = "2026-07-06T04:22:14.453Z" }, + { url = "https://files.pythonhosted.org/packages/d2/3d/de8569386685213135dfbaab58e221add11858485524cf7d5987efc6d70b/grpcio-1.82.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:680f633bc788652222cb90a568ec374edcd91e1fe7715a7c248ece8e1032c2bd", size = 7910708, upload-time = "2026-07-06T04:22:17.994Z" }, + { url = "https://files.pythonhosted.org/packages/42/ab/3950b32369763818ae89ef76d52e84bd034d3a0ab46bf315669c913b32e2/grpcio-1.82.0-cp313-cp313-win32.whl", hash = "sha256:640b4715b9c206e5176e46601aa1422c47c779f642a869dfd27062e8fde13dfb", size = 4240351, upload-time = "2026-07-06T04:22:20.626Z" }, + { url = "https://files.pythonhosted.org/packages/a7/5c/c7b9a48efe973883f5707a311f87772a4c307a22ec80d6f3c851846a0a02/grpcio-1.82.0-cp313-cp313-win_amd64.whl", hash = "sha256:8523e81bd23f83e607aed28fbd8244b2be4aeb34e084fcef1bd1867709085c2a", size = 5000985, upload-time = "2026-07-06T04:22:23.365Z" }, + { url = "https://files.pythonhosted.org/packages/bd/6e/abc5efe11b73eefa0183531032eef2a53f33aeced6484b0d9da559c12ed6/grpcio-1.82.0-cp314-cp314-linux_armv7l.whl", hash = "sha256:136969867c14fcc743fa39b12d7da133cae7da4f8e0e7767b6b8b7e75a8c24fd", size = 6146898, upload-time = "2026-07-06T04:22:26.082Z" }, + { url = "https://files.pythonhosted.org/packages/e0/e4/d01ee88ceac221436dce1584ae1f046bd44d0dffc5a92e95aa34857c4516/grpcio-1.82.0-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:a7e246cd216662f513ae79ea3afadde6afe8a659b48a4170bae9a896e69ac254", size = 11954897, upload-time = "2026-07-06T04:22:29.114Z" }, + { url = "https://files.pythonhosted.org/packages/99/b9/2a35540ec08afc08859c35afd1a8a51e2585f39b12cf4eee68dcb1fa973d/grpcio-1.82.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:85c2a4e9e8a30d73d41e547a5101e57a1ed2093ec4b2daf6847c344adba00523", size = 6723102, upload-time = "2026-07-06T04:22:32.146Z" }, + { url = "https://files.pythonhosted.org/packages/ec/ba/af71af59943b5d08879617a1d97495d45c53e43f3c6ef16f820c64e107f9/grpcio-1.82.0-cp314-cp314-manylinux2014_i686.manylinux_2_17_i686.whl", hash = "sha256:a3a7802328e0d20e6ac458ff1974a8096cfaa3ca84dea0903235f641f1d703c1", size = 7454545, upload-time = "2026-07-06T04:22:34.776Z" }, + { url = "https://files.pythonhosted.org/packages/b3/7e/258aae12b68910140725873657469ff166f8968bdb0674e12d6452b1812c/grpcio-1.82.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:9331e6b91b26cffd63fb70a545f3c2cac8dc1b434b4436f43915c23e1e22f265", size = 6889585, upload-time = "2026-07-06T04:22:37.411Z" }, + { url = "https://files.pythonhosted.org/packages/5e/04/cfada8930306b9101cfb405b33ecb3b72abce51d725f900f167ffbc3146b/grpcio-1.82.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:670ab93a0059e707de53b9e715304b9f566b6defee80e5490cdd15820ce032f6", size = 7514162, upload-time = "2026-07-06T04:22:40.076Z" }, + { url = "https://files.pythonhosted.org/packages/2e/57/955f95989338857a6d73ee838b4d9e8198d50972a70ca83ec22322396f4c/grpcio-1.82.0-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:fbfe2e321bb30b84799677d6e8469896e487b494e9e4fbc9b8cc4425fe054d44", size = 8536163, upload-time = "2026-07-06T04:22:42.801Z" }, + { url = "https://files.pythonhosted.org/packages/06/62/826da0b0f01c7ba3f7ea0185968f551290deb28968d98b1edf604c895db8/grpcio-1.82.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2472b72a55da8570e089f1ca22f34a350d86c109be2755f6ee1cca5ebbdd9b54", size = 7912573, upload-time = "2026-07-06T04:22:46.187Z" }, + { url = "https://files.pythonhosted.org/packages/4d/b4/d8824a66ac1ac7fbcdf63806f1a8725d95426654e859afa4606070c78740/grpcio-1.82.0-cp314-cp314-win32.whl", hash = "sha256:c9feb986151c2726789599b3672c1c9faa1d0824da921dc3f33a8cb1f93e2861", size = 4321824, upload-time = "2026-07-06T04:22:48.866Z" }, + { url = "https://files.pythonhosted.org/packages/da/08/02e581cc0bbb4c7b842f85e66a945b46d5bcc5927575c1086edfbc3360e7/grpcio-1.82.0-cp314-cp314-win_amd64.whl", hash = "sha256:b1eeb0480d86965263f8c8ae7ee5360c284d22176a196aab02fce7fba8d89dd8", size = 5141112, upload-time = "2026-07-06T04:22:51.443Z" }, +] + [[package]] name = "h11" version = "0.16.0" @@ -2057,6 +2146,190 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/7f/95/4df134a100b5a9a12378d5301b934366686ef6fbdaffcd21211d5654970e/nox-2026.4.10-py3-none-any.whl", hash = "sha256:082c117627590d9b90aa21f86df89b310b07c5842539524203bcb3c719f116c1", size = 75536, upload-time = "2026-04-10T17:42:40.664Z" }, ] +[[package]] +name = "opentelemetry-api" +version = "1.43.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ae/cc/e4c9584181f86494df0f6bdec1a4f3280c50db44704dc2a407e994fc87bb/opentelemetry_api-1.43.0.tar.gz", hash = "sha256:107d0d03857ea8fc7c5fcbbbd83f800c281f0d560553d61c1d675fccfd1761c1", size = 73476, upload-time = "2026-06-24T15:19:55.323Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/17/83/6dba32b85f31868400440dc7ad2ca1eab94cbbf3a7b0459ed39f8311a9e2/opentelemetry_api-1.43.0-py3-none-any.whl", hash = "sha256:20acf45e9b21851926835292e4045d290acade1edd2ff3de86d2f069687ba1fd", size = 61912, upload-time = "2026-06-24T15:19:35.434Z" }, +] + +[[package]] +name = "opentelemetry-exporter-otlp-proto-common" +version = "1.43.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "opentelemetry-proto" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/55/c1/e8098490ab15abf116dcaf9fa89ededcb35547c7d08d4b5a62f573dc1e63/opentelemetry_exporter_otlp_proto_common-1.43.0.tar.gz", hash = "sha256:c4e32ba6d6b13bdb2b8f6764c4fd28d00192826561aa04f6d14eedfce7ac076f", size = 20197, upload-time = "2026-06-24T15:20:00.247Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d0/b2/41ebc74ae1d5859901f1b69305de58724bf043381103d6ef413521cbc35a/opentelemetry_exporter_otlp_proto_common-1.43.0-py3-none-any.whl", hash = "sha256:123c3f9cc87218562490c63b36f497bf3a722faf174a515d1443f31ababa6264", size = 17048, upload-time = "2026-06-24T15:19:41.264Z" }, +] + +[[package]] +name = "opentelemetry-exporter-otlp-proto-grpc" +version = "1.43.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "googleapis-common-protos" }, + { name = "grpcio" }, + { name = "opentelemetry-api" }, + { name = "opentelemetry-exporter-otlp-proto-common" }, + { name = "opentelemetry-proto" }, + { name = "opentelemetry-sdk" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e2/1d/6336453716ca0a240d4417d19e6d5b77a5e7163e5670ec4f7ec4d3ede7bf/opentelemetry_exporter_otlp_proto_grpc-1.43.0.tar.gz", hash = "sha256:1b3e0627daa9bc21884d4a13946807c255eb558bfe5bdd543dffb6f4c9faee0d", size = 27213, upload-time = "2026-06-24T15:20:00.907Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/6d/74/2700b5d5c946bf2dba87073fce3dfc198c46bc92ea3d5693f54bc51c90b1/opentelemetry_exporter_otlp_proto_grpc-1.43.0-py3-none-any.whl", hash = "sha256:6a10d1feacffffda19acacbf277b736094b1e2f4dbb98c90ccb2c6e1962e2ec6", size = 19626, upload-time = "2026-06-24T15:19:42.233Z" }, +] + +[[package]] +name = "opentelemetry-instrumentation" +version = "0.64b0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "opentelemetry-api" }, + { name = "opentelemetry-semantic-conventions" }, + { name = "packaging" }, + { name = "wrapt" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/7e/97/02fe6e1c8b1ffac42d0b429c18080edb24e0e0d18c86612edf72b5752382/opentelemetry_instrumentation-0.64b0.tar.gz", hash = "sha256:b47d528dead6271d7743114417eb67fc915bd9258111c48dbf9a4951d2efa88d", size = 41935, upload-time = "2026-06-24T15:19:12.951Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d2/0c/cb9fe342de5299c7af24582eb7d788661cc53a1c4b904da92309caaa9417/opentelemetry_instrumentation-0.64b0-py3-none-any.whl", hash = "sha256:133ab7ffca796557aec059bf6be3190a34b6dea987f25be3d9409e230cbdad8b", size = 35880, upload-time = "2026-06-24T15:18:17.277Z" }, +] + +[[package]] +name = "opentelemetry-instrumentation-asgi" +version = "0.64b0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "asgiref" }, + { name = "opentelemetry-api" }, + { name = "opentelemetry-instrumentation" }, + { name = "opentelemetry-semantic-conventions" }, + { name = "opentelemetry-util-http" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/85/0c/71c696fccb86d37af383ea1604af4729fabad0af2fabaf203e4c79c1e859/opentelemetry_instrumentation_asgi-0.64b0.tar.gz", hash = "sha256:4dd3eee566a4303f8e6b9b84f2a0a7abc57a6640df768926c68a3868bf5b2090", size = 26136, upload-time = "2026-06-24T15:19:17.003Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/51/20/218b65a63d847a7ed28d1bea84c39234689160b74480b8702272e37f4240/opentelemetry_instrumentation_asgi-0.64b0-py3-none-any.whl", hash = "sha256:e0840b66e15303a9254b0540946010bd008aa0504f4d89b8e1b7fb63490a36f0", size = 15906, upload-time = "2026-06-24T15:18:23.107Z" }, +] + +[[package]] +name = "opentelemetry-instrumentation-fastapi" +version = "0.64b0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "opentelemetry-api" }, + { name = "opentelemetry-instrumentation" }, + { name = "opentelemetry-instrumentation-asgi" }, + { name = "opentelemetry-semantic-conventions" }, + { name = "opentelemetry-util-http" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/db/a1/52282e2cc5c08f4df13b087896d9907258fe2ff4f34035d3b21aa92684c3/opentelemetry_instrumentation_fastapi-0.64b0.tar.gz", hash = "sha256:05f75149929e433c1630de381688e650bf651c1e1cce7f9a7b649a807dac8a98", size = 26236, upload-time = "2026-06-24T15:19:27.219Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/72/2d/4f48dc2d6f289f2febc5b871940dbea9f3b04ab9186d23342581b49cb984/opentelemetry_instrumentation_fastapi-0.64b0-py3-none-any.whl", hash = "sha256:43cbbfb2d3079dc81104478a2950ae93ac6d0e90a5020fa3987a236f8f2bdef1", size = 13263, upload-time = "2026-06-24T15:18:38.553Z" }, +] + +[[package]] +name = "opentelemetry-instrumentation-httpx" +version = "0.64b0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "opentelemetry-api" }, + { name = "opentelemetry-instrumentation" }, + { name = "opentelemetry-semantic-conventions" }, + { name = "opentelemetry-util-http" }, + { name = "wrapt" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/0d/2a/2893a8781b93894f1e8014904c0342da7e4302de6597c2d5c0cb6c1a552e/opentelemetry_instrumentation_httpx-0.64b0.tar.gz", hash = "sha256:c2cfcd03d3665762860ebd0c28038c6e47fbb48d7942dec31dd75fc634d25c92", size = 23555, upload-time = "2026-06-24T15:19:29.107Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/06/29/a20309bd3f5a8051b61ca475e78623410c3b077e3deccae19aa4f8b5b9a2/opentelemetry_instrumentation_httpx-0.64b0-py3-none-any.whl", hash = "sha256:04829e5723941b5ceb0c88b44d63983e226b5c75b2b2e34a57739fdd0e060608", size = 16336, upload-time = "2026-06-24T15:18:41.412Z" }, +] + +[[package]] +name = "opentelemetry-instrumentation-sqlalchemy" +version = "0.64b0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "opentelemetry-api" }, + { name = "opentelemetry-instrumentation" }, + { name = "opentelemetry-semantic-conventions" }, + { name = "packaging" }, + { name = "wrapt" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/46/d7/86e333dede47a67a0f989f383da5d0840162ee21e0b252ca56cf61351c59/opentelemetry_instrumentation_sqlalchemy-0.64b0.tar.gz", hash = "sha256:dd737e0b0a72500961aa0dbb6fb2a10525f8dbb5299f9315ddf82d415cc64719", size = 18006, upload-time = "2026-06-24T15:19:40.225Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/49/90/5424365cb111d2383747b36ca3dbc558e86ef5ec6694efb53bfcb771f046/opentelemetry_instrumentation_sqlalchemy-0.64b0-py3-none-any.whl", hash = "sha256:687ee4c453020c7db009a7f3e390bb2b28eab4e61517e6d267e6e448c3eedcfb", size = 14410, upload-time = "2026-06-24T15:18:57.287Z" }, +] + +[[package]] +name = "opentelemetry-proto" +version = "1.43.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "protobuf" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e0/b9/d357faefb40bda1d4799913e6af611171ff22a2dedcb93576bc92242d056/opentelemetry_proto-1.43.0.tar.gz", hash = "sha256:224778df17e1f3fafeaaa21d874236ca5f6ffc2f86e0899298ec7351aac27924", size = 46481, upload-time = "2026-06-24T15:20:07.625Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ed/a7/3e5308cf548b8f72529c7db1afdb3a404211982376a12927fd7759f77bf3/opentelemetry_proto-1.43.0-py3-none-any.whl", hash = "sha256:c58f1f7ef84bc7dc2834016c0c37fe0081dde7ca9f6339be1970fbf9cdaaa90d", size = 72489, upload-time = "2026-06-24T15:19:51.164Z" }, +] + +[[package]] +name = "opentelemetry-resourcedetector-gcp" +version = "1.12.0a0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "opentelemetry-api" }, + { name = "opentelemetry-sdk" }, + { name = "requests" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/21/ae/b62c5e986c9c7f908a15682ea173bcfcdc00403c0c85243ccbd30eca7fc2/opentelemetry_resourcedetector_gcp-1.12.0a0.tar.gz", hash = "sha256:d5e3f78283a272eb92547e00bbeff45b7332a34ae791a70ab4eba81af9bc3baf", size = 18797, upload-time = "2026-04-28T20:59:43.195Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/df/84/9db2999adbc41505af3e6717e8d958746778cbfc9e07ed9c670bf9d1e6db/opentelemetry_resourcedetector_gcp-1.12.0a0-py3-none-any.whl", hash = "sha256:e803688d14e2969fe816077be81f7b034368314d485863f12ce49daba7c81919", size = 18798, upload-time = "2026-04-28T20:59:39.257Z" }, +] + +[[package]] +name = "opentelemetry-sdk" +version = "1.43.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "opentelemetry-api" }, + { name = "opentelemetry-semantic-conventions" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/3e/eb/5041074274ac0956b03637cc039d434569112468e875eddfcc9a0674ce06/opentelemetry_sdk-1.43.0.tar.gz", hash = "sha256:d8187c81c162df9913e4003dd6485f7390d9a24fc17026ec7387b8b8218b08e9", size = 254744, upload-time = "2026-06-24T15:20:08.467Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/49/e3/b17be23af124201c9f52eececd4cc8ddfed1597d37b4ee771895d325805c/opentelemetry_sdk-1.43.0-py3-none-any.whl", hash = "sha256:d1323a547c1ce69d6a069a17a44b7da82bb8b332051ecb074041f87642c86823", size = 178852, upload-time = "2026-06-24T15:19:52.169Z" }, +] + +[[package]] +name = "opentelemetry-semantic-conventions" +version = "0.64b0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "opentelemetry-api" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/5a/30/5f26df29509eccd86b99b481ac9ffa39da49ba9577cc69071c552ae30447/opentelemetry_semantic_conventions-0.64b0.tar.gz", hash = "sha256:72f76fb2d1582d9d033dd1fcd84532e961e6ff3d90d24ba6fabc72975a83864c", size = 148340, upload-time = "2026-06-24T15:20:09.267Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f2/ca/23ba87a221b574a7c5a99d48849d80bfe8b047624681357e2b002e566187/opentelemetry_semantic_conventions-0.64b0-py3-none-any.whl", hash = "sha256:ea77e85e354b8f604ddbe5f3d9135216f982fa4d77e5859ac30f6d8a50505aa6", size = 203713, upload-time = "2026-06-24T15:19:53.339Z" }, +] + +[[package]] +name = "opentelemetry-util-http" +version = "0.64b0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c3/1b/1029a805fd7242f7dfce91633b244c3b14a94d703232878f71e01ce862b1/opentelemetry_util_http-0.64b0.tar.gz", hash = "sha256:8a86a220dbfc56d736f47f1e5c4e7932a21fcf69052312e1bcf166444dc79322", size = 11102, upload-time = "2026-06-24T15:19:48.974Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1c/c7/5f8ec5b30546f2dc22cd5fc5759bce2ab5be6e89a2e710a405ac9ef64ed3/opentelemetry_util_http-0.64b0-py3-none-any.whl", hash = "sha256:c1e5350d25507c1afcd6076cf9ac062485a0a4f79cd9971366996fd3056bacdb", size = 8204, upload-time = "2026-06-24T15:19:09.02Z" }, +] + [[package]] name = "orjson" version = "3.11.9" @@ -2385,6 +2658,21 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/3a/ed/1cdcab6ba3d6ab7feca11fc14f0eeea80755bb53ef4e892079f31b10a25f/propcache-0.5.2-py3-none-any.whl", hash = "sha256:be1ddfcbb376e3de5d2e2db1d58d6d67463e6b4f9f040c000de8e300295465fe", size = 14036, upload-time = "2026-05-08T21:02:10.673Z" }, ] +[[package]] +name = "protobuf" +version = "7.35.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/da/01/9ef0afd7999eb9badb3a768b4aedd78c86d4c65cfaf1958ab276199e76b4/protobuf-7.35.1.tar.gz", hash = "sha256:ce115a26fe0c39a2c29973d914d327e516a6455464489fe3cd1e51a1b354f81a", size = 458717, upload-time = "2026-06-11T21:55:40.257Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/10/03/8aeeb7458d22546bf64b5250ca1daeb5ff757d900e8e4a7476c6f0db843e/protobuf-7.35.1-cp310-abi3-macosx_10_9_universal2.whl", hash = "sha256:24f857477359a85c0c235261b8ba905fd51b2562f4a64ca1df5473f29850cbf6", size = 433226, upload-time = "2026-06-11T21:55:31.719Z" }, + { url = "https://files.pythonhosted.org/packages/37/4b/dfb89eb0e652a1ff073c39a59fb5e3a83cfe9b57a2c83fa6d78270101767/protobuf-7.35.1-cp310-abi3-manylinux2014_aarch64.whl", hash = "sha256:11d6b0ec246892d85215b0a13ca6e0233cf5284b68f0ac02646427f4ff88a799", size = 328847, upload-time = "2026-06-11T21:55:34.035Z" }, + { url = "https://files.pythonhosted.org/packages/0f/58/dc12f2cd484951524af6e3382c785869b9b3fb5e52ee95ae23add53ee8f9/protobuf-7.35.1-cp310-abi3-manylinux2014_s390x.whl", hash = "sha256:b73f9489a4b8b1c9cb1f8ed951c736392592edb24b9d6819f36d2e10b171d5b4", size = 344030, upload-time = "2026-06-11T21:55:34.941Z" }, + { url = "https://files.pythonhosted.org/packages/e4/be/5b3cfe508bfab6761414ff944e3366eb13be4fd71efcd69450f89ba39f43/protobuf-7.35.1-cp310-abi3-manylinux2014_x86_64.whl", hash = "sha256:74758715c53d7158fb76caf4f0cfdacc5329a4b1bb994f865d6cf302d413a1c4", size = 327130, upload-time = "2026-06-11T21:55:35.921Z" }, + { url = "https://files.pythonhosted.org/packages/d8/bc/6d6c7ba8709c85f8f2c390b2b118d6fb08a783676a572271851bf45a7d22/protobuf-7.35.1-cp310-abi3-win32.whl", hash = "sha256:353652e4efd0bca5b5fc2656abf8307ef351f0cf938c9eba09f0e09c20a25c30", size = 428945, upload-time = "2026-06-11T21:55:37.034Z" }, + { url = "https://files.pythonhosted.org/packages/0a/19/8d0cb6f20a1ef7b18f1c8986ad5783f22f84cce39c6ce9a6e645ea55192e/protobuf-7.35.1-cp310-abi3-win_amd64.whl", hash = "sha256:230a75ddfc2de4806e56696ce9640c1cdfdb6543b7cfce98d42a4c0a0e7bdb87", size = 439996, upload-time = "2026-06-11T21:55:38.123Z" }, + { url = "https://files.pythonhosted.org/packages/19/c7/5f7c636ec43e0c545e28d1f1db71990108306f7bdcb89f069ba97e428e7f/protobuf-7.35.1-py3-none-any.whl", hash = "sha256:4bc97768d8fe4ad6743c8a19403e314511ed9f6d13205b687e52421c023ac1b9", size = 171659, upload-time = "2026-06-11T21:55:39.155Z" }, +] + [[package]] name = "psutil" version = "7.2.2"