Skip to content

feat: Per-integration instrumentation.name on span origin#576

Merged
Abhijeet Prasad (AbhiPrasad) merged 9 commits into
mainfrom
feat/instrumentation-name
Jul 16, 2026
Merged

feat: Per-integration instrumentation.name on span origin#576
Abhijeet Prasad (AbhiPrasad) merged 9 commits into
mainfrom
feat/instrumentation-name

Conversation

@starfolkai

@starfolkai starfolkai Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Every span an integration creates now carries a stable identifier in context.span_origin.instrumentation.nameopenai-auto, anthropic-auto, temporal-auto, and so on — matching the naming convention in the Braintrust instrumentation spec's span_origin example. Previously all integration spans were tagged with the shared braintrust-python-logger default, which made per-integration filtering on span_origin impossible.

Builds on #570 (which added the span_origin scaffolding).

API surface

start_span (and every provider-level start_span method) gains an internal: dict | None = None kwarg reserved for Braintrust SDK internals. Integrations pass internal={"instrumentation": "<provider>-auto"} to stamp the span origin; the internal name signals to external callers that they shouldn't touch it and its keys can change without notice.

from braintrust import start_span

with start_span(name="my-op", internal={"instrumentation": "my-integration-auto"}) as span:
    ...
  • Threaded through every span-creation surface: top-level start_span, Span.start_span, SpanImpl.start_span, Experiment.start_span, Dataset.start_span, Logger.start_span, and _NoopSpan.start_span.
  • When set, SpanImpl reads internal["instrumentation"] and stamps it into context.span_origin.instrumentation.name via merge_span_origin_context.
  • When unset, spans fall back to the channel default — braintrust-python-logger for direct logging, braintrust-python-otel for the OTel processor.
  • Does NOT propagate through the parent/child edge. Each start_span call independently decides its own value, so a user's @traced scorer nested inside a wrapped provider call keeps the default identifier — the integration's name only lands on spans the integration itself opens. This matches the spec's "the module that directly created the span" definition and mirrors how Sentry's origin and OTel's InstrumentationScope behave.

Suggested for external Braintrust integrations authored outside this repo:

from braintrust.logger import start_span as _bt_start_span

_INSTRUMENTATION = "my-provider-auto"

def start_span(*args, **kwargs):
    internal = dict(kwargs.get("internal") or {})
    internal.setdefault("instrumentation", _INSTRUMENTATION)
    kwargs["internal"] = internal
    return _bt_start_span(*args, **kwargs)

Every module-scope start_span(...) call then flows through the shadow with zero further edits.

Integration migration (23 integrations)

21 shadow-based (adk, agentscope, agno, anthropic, autogen, bedrock_runtime, claude_agent_sdk, cohere, crewai, dspy, google_genai, huggingface_hub, instructor, langchain, litellm, livekit_agents, llamaindex, mistral, openai, openai_agents, openrouter, pydantic_ai) — the shadow above at the top of each tracing.py/callbacks.py.

Explicit at call sites (2 integrations):

  • temporal/plugin.py: internal={"instrumentation": "temporal-auto"} at each logger.start_span(...) site — Logger method calls don't go through the module-level shadow.
  • openai_agents/tracing.py: internal={"instrumentation": _INSTRUMENTATION} on the three current_context.start_span(...) / self._logger.start_span(...) / parent.start_span(...) sites.

Tests

  • py/src/braintrust/test_otel.py — new tests covering merge_span_origin_context resolution and an end-to-end SpanImpl assertion that child spans do NOT inherit the parent's instrumentation name (child falls back to braintrust-python-logger).
  • Per-integration inline assertion: each of the 23 integrations' existing VCR tests now asserts span["context"]["span_origin"]["instrumentation"]["name"] == "<provider>-auto" in place, next to the real provider-facing coverage. If a shadow ever regresses, that specific integration's test suite fails with a clear signal.

Docs

  • .agents/skills/sdk-integrations/SKILL.md gains a Span origin section documenting the shadow pattern, the no-inheritance rule, and the <provider>-auto naming convention.
  • Same file relaxes the earlier strict "metadata MUST NOT capture anything outside the spec" guidance to an allowlist-per-provider model — spec-defined keys must be captured, provider-specific detail fields can be included deliberately, no dumping raw request/response objects.
  • Spec reference now links to the actual URL (https://github.com/braintrustdata/braintrust-spec/blob/main/docs/instrumentation-guide.md).

Validation

  • nox -s test_core: 647 passed, 66 skipped, 12 xfailed
  • nox -s test_types: 16 passed (pyright + mypy clean)
  • nox -s pylint: successful (also ran ruff format + ruff check --fix as part of pre-commit)
  • 30 provider (latest) sessions — every one whose Python 3.14 wheels are available. Zero failures. test_livekit_agents and test_crewai skipped due to pre-existing Python 3.14 wheel gaps in onnxruntime / pydantic-core, unrelated to this change.

Follow-ups

  • JS SDK's mirror PR (feat: Add span origin provenance braintrust-sdk-javascript#2226) is still open with only braintrust-js-logger / braintrust-otel-js hardcoded and no per-integration wiring. Aligning JS on the same <provider>-auto names is worth doing before this ships so cross-SDK dashboards work — the spec only gives openai-auto as an illustrative example, so a coordinated commit-to-a-convention is needed on both sides.
  • Naming convention isn't currently prescribed by the spec (docs/instrumentation-guide.md just shows openai-auto in a JSON blob). Worth a spec PR to make it normative once JS and Python agree.

Created by abhijeet

Slack thread

Every span an integration creates now carries a stable identifier in
context.span_origin.instrumentation.name (e.g. openai-auto,
anthropic-auto, temporal-auto), matching the naming convention in the
Braintrust instrumentation spec example. Previously all integration
spans were tagged with the shared braintrust-python-logger default,
which made per-integration filtering on span_origin impossible.

New public API on start_span:

  from braintrust import start_span

  with start_span(name="my-op", instrumentation="my-integration") as s:
      ...

The instrumentation kwarg is threaded through every span-creation
surface: top-level start_span, Span.start_span, SpanImpl.start_span,
Experiment.start_span, Dataset.start_span, Logger.start_span, and
_NoopSpan.start_span. When set, SpanImpl stamps the value into
context.span_origin.instrumentation.name via
merge_span_origin_context (which gained an
override_instrumentation_name parameter). When unset, spans fall back
to the channel default (braintrust-python-logger for direct logging,
braintrust-python-otel for the OTel processor).

The kwarg does NOT propagate through the parent/child edge. Each
start_span call independently decides its own value, so a user's
@Traced scorer nested inside a wrapped provider call keeps the
default identifier — the integration's name only lands on spans the
integration itself opens.

Migration for 23 integrations:

- 21 integrations (adk, agentscope, agno, anthropic, autogen,
  bedrock_runtime, claude_agent_sdk, cohere, crewai, dspy,
  google_genai, huggingface_hub, instructor, langchain, litellm,
  livekit_agents, llamaindex, mistral, openai, openai_agents,
  openrouter, pydantic_ai) use a two-line shadow at the top of their
  tracing/callbacks module:

    from braintrust.logger import start_span as _bt_start_span
    _INSTRUMENTATION = "<provider>-auto"
    def start_span(*args, **kwargs):
        kwargs.setdefault("instrumentation", _INSTRUMENTATION)
        return _bt_start_span(*args, **kwargs)

  Every existing start_span(...) call site in the module then flows
  through the shadow with zero further edits.

- temporal passes instrumentation="temporal-auto" explicitly at each
  logger.start_span(...) site (not covered by the module shadow
  because those calls go through a Logger instance, not the imported
  top-level function).

- openai_agents passes instrumentation=_INSTRUMENTATION explicitly on
  its three parent.start_span(...) / self._logger.start_span(...)
  sites for the same reason.

Tests:

- py/src/braintrust/test_otel.py — 4 new unit tests covering
  merge_span_origin_context resolution order (default, override,
  empty override falls back), and a SpanImpl-level assertion that
  child spans do NOT inherit the parent's instrumentation.

- py/src/braintrust/integrations/test_span_origin.py — new file with
  46 parametrized tests asserting (a) each integration declares
  _INSTRUMENTATION = "<provider>-auto", (b) a span opened through
  the integration's local start_span emits
  context.span_origin.instrumentation.name matching, plus static
  assertions for temporal and openai_agents that every
  logger.start_span / parent.start_span site carries the explicit
  kwarg.

Docs:

- .agents/skills/sdk-integrations/SKILL.md gains a Span origin
  section documenting the shadow pattern, the no-inheritance rule,
  and the <provider>-auto naming convention. Same file relaxes the
  earlier strict metadata guidance to an allowlist-per-provider
  model.

Validation:

- test_core: 647 passed, 66 skipped, 12 xfailed
- test_types: 16 passed (pyright + mypy clean)
- pylint: successful
- 30 provider (latest) sessions run — every session in the noxfile
  whose Python 3.14 wheels are available. Zero failures.
  test_livekit_agents and test_crewai skipped due to pre-existing
  Python 3.14 wheel gaps in onnxruntime / pydantic-core, unrelated
  to this change.

Note: the JS SDK's mirror PR (braintrust-sdk-javascript#2226) is
still open with only braintrust-js-logger / braintrust-otel-js
hardcoded and no per-integration wiring. Aligning JS on the same
<provider>-auto names is worth doing before this ships so cross-SDK
dashboards work.
@starfolkai
starfolkai Bot force-pushed the feat/instrumentation-name branch from 31c2290 to d09b84a Compare July 16, 2026 14:15
… VCR tests

Two related cleanups:

1. Simplify merge_span_origin_context API. Drop the
   override_instrumentation_name kwarg and just pass the single
   resolved name in through the existing instrumentation_name
   parameter. SpanImpl.__init__ resolves the fallback with `or`:

     self._instrumentation = instrumentation or "braintrust-python-logger"

   Direct callers no longer have to know about two parameters.

2. Replace the consolidated integrations/test_span_origin.py file
   with one inline assertion in each integration's existing VCR
   test. The consolidated file re-tested the same shadow shape 22
   times without ever exercising a real provider payload; the
   per-integration assertions live alongside the real
   provider-facing coverage and fail on the specific integration if
   the shadow ever regresses.

Adds `assert span["context"]["span_origin"]["instrumentation"]["name"] == "<provider>-auto"`
to one existing VCR test per integration (adk, agentscope, agno,
anthropic, autogen, bedrock_runtime, claude_agent_sdk, cohere,
crewai, dspy, google_genai, huggingface_hub, instructor, langchain,
litellm, livekit_agents, llamaindex, mistral, openai,
openai_agents, openrouter, pydantic_ai, temporal).
Move the `instrumentation` parameter behind an `internal: dict` kwarg
on every span-creation surface. The `internal` dict is opaque to
external callers by design — the name signals \"reserved for
Braintrust SDK internals\" and the docstring says the keys inside
may change without notice.

Before:
    start_span(name=\"my-op\", instrumentation=\"openai-auto\")

After:
    start_span(name=\"my-op\", internal={\"instrumentation\": \"openai-auto\"})

Integration shadow helpers become:

    def start_span(*args, **kwargs):
        internal = dict(kwargs.get(\"internal\") or {})
        internal.setdefault(\"instrumentation\", _INSTRUMENTATION)
        kwargs[\"internal\"] = internal
        return _bt_start_span(*args, **kwargs)

temporal and openai_agents (which open spans through Logger /
parent-Span methods rather than the module-level `start_span`) pass
`internal={\"instrumentation\": \"<provider>-auto\"}` explicitly.

SKILL.md updated accordingly.
Adds a `SpanInternalOptions(TypedDict, total=False)` in logger.py so
IDEs still get autocompletion / type-checking on the opaque
`internal={...}` kwarg. Currently declares one field:

    class SpanInternalOptions(TypedDict, total=False):
        instrumentation: str

Every span-creation surface (top-level `start_span`,
`Span.start_span`, `SpanImpl.__init__` / `.start_span`,
`Experiment.start_span`, `Dataset.start_span`, `Logger.start_span`,
`_NoopSpan.start_span`) now types the kwarg as
`internal: SpanInternalOptions | None = None`.

pyright + mypy pass.
Three integrations open child spans through Logger/Span method calls
that bypass the module-level `start_span` shadow, so those spans were
falling back to `braintrust-python-logger`:

- langchain/callbacks.py: `parent_span.start_span(...)` and
  `init_logger().start_span(...)` in `_start_span`
- llamaindex/tracing.py: `parent_bt_span.start_span(...)` in
  `on_start`
- google_genai/tracing.py: `logger._start_span_impl(...)` in the
  streaming context helper

Each site now passes `internal={"instrumentation": _INSTRUMENTATION}`
explicitly. Matches the pattern already in place for
temporal/plugin.py and openai_agents/tracing.py.

Caught by the per-integration span_origin assertions in each
integration's test file.
@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) merged commit c056762 into main Jul 16, 2026
82 checks passed
@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) deleted the feat/instrumentation-name branch July 16, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants