feat: Per-integration instrumentation.name on span origin#576
Merged
Conversation
Every span an integration creates now carries a stable identifier in
context.span_origin.instrumentation.name (e.g. openai-auto,
anthropic-auto, temporal-auto), matching the naming convention in the
Braintrust instrumentation spec example. Previously all integration
spans were tagged with the shared braintrust-python-logger default,
which made per-integration filtering on span_origin impossible.
New public API on start_span:
from braintrust import start_span
with start_span(name="my-op", instrumentation="my-integration") as s:
...
The instrumentation kwarg is threaded through every span-creation
surface: top-level start_span, Span.start_span, SpanImpl.start_span,
Experiment.start_span, Dataset.start_span, Logger.start_span, and
_NoopSpan.start_span. When set, SpanImpl stamps the value into
context.span_origin.instrumentation.name via
merge_span_origin_context (which gained an
override_instrumentation_name parameter). When unset, spans fall back
to the channel default (braintrust-python-logger for direct logging,
braintrust-python-otel for the OTel processor).
The kwarg does NOT propagate through the parent/child edge. Each
start_span call independently decides its own value, so a user's
@Traced scorer nested inside a wrapped provider call keeps the
default identifier — the integration's name only lands on spans the
integration itself opens.
Migration for 23 integrations:
- 21 integrations (adk, agentscope, agno, anthropic, autogen,
bedrock_runtime, claude_agent_sdk, cohere, crewai, dspy,
google_genai, huggingface_hub, instructor, langchain, litellm,
livekit_agents, llamaindex, mistral, openai, openai_agents,
openrouter, pydantic_ai) use a two-line shadow at the top of their
tracing/callbacks module:
from braintrust.logger import start_span as _bt_start_span
_INSTRUMENTATION = "<provider>-auto"
def start_span(*args, **kwargs):
kwargs.setdefault("instrumentation", _INSTRUMENTATION)
return _bt_start_span(*args, **kwargs)
Every existing start_span(...) call site in the module then flows
through the shadow with zero further edits.
- temporal passes instrumentation="temporal-auto" explicitly at each
logger.start_span(...) site (not covered by the module shadow
because those calls go through a Logger instance, not the imported
top-level function).
- openai_agents passes instrumentation=_INSTRUMENTATION explicitly on
its three parent.start_span(...) / self._logger.start_span(...)
sites for the same reason.
Tests:
- py/src/braintrust/test_otel.py — 4 new unit tests covering
merge_span_origin_context resolution order (default, override,
empty override falls back), and a SpanImpl-level assertion that
child spans do NOT inherit the parent's instrumentation.
- py/src/braintrust/integrations/test_span_origin.py — new file with
46 parametrized tests asserting (a) each integration declares
_INSTRUMENTATION = "<provider>-auto", (b) a span opened through
the integration's local start_span emits
context.span_origin.instrumentation.name matching, plus static
assertions for temporal and openai_agents that every
logger.start_span / parent.start_span site carries the explicit
kwarg.
Docs:
- .agents/skills/sdk-integrations/SKILL.md gains a Span origin
section documenting the shadow pattern, the no-inheritance rule,
and the <provider>-auto naming convention. Same file relaxes the
earlier strict metadata guidance to an allowlist-per-provider
model.
Validation:
- test_core: 647 passed, 66 skipped, 12 xfailed
- test_types: 16 passed (pyright + mypy clean)
- pylint: successful
- 30 provider (latest) sessions run — every session in the noxfile
whose Python 3.14 wheels are available. Zero failures.
test_livekit_agents and test_crewai skipped due to pre-existing
Python 3.14 wheel gaps in onnxruntime / pydantic-core, unrelated
to this change.
Note: the JS SDK's mirror PR (braintrust-sdk-javascript#2226) is
still open with only braintrust-js-logger / braintrust-otel-js
hardcoded and no per-integration wiring. Aligning JS on the same
<provider>-auto names is worth doing before this ships so cross-SDK
dashboards work.
starfolkai
Bot
force-pushed
the
feat/instrumentation-name
branch
from
July 16, 2026 14:15
31c2290 to
d09b84a
Compare
… VCR tests
Two related cleanups:
1. Simplify merge_span_origin_context API. Drop the
override_instrumentation_name kwarg and just pass the single
resolved name in through the existing instrumentation_name
parameter. SpanImpl.__init__ resolves the fallback with `or`:
self._instrumentation = instrumentation or "braintrust-python-logger"
Direct callers no longer have to know about two parameters.
2. Replace the consolidated integrations/test_span_origin.py file
with one inline assertion in each integration's existing VCR
test. The consolidated file re-tested the same shadow shape 22
times without ever exercising a real provider payload; the
per-integration assertions live alongside the real
provider-facing coverage and fail on the specific integration if
the shadow ever regresses.
Adds `assert span["context"]["span_origin"]["instrumentation"]["name"] == "<provider>-auto"`
to one existing VCR test per integration (adk, agentscope, agno,
anthropic, autogen, bedrock_runtime, claude_agent_sdk, cohere,
crewai, dspy, google_genai, huggingface_hub, instructor, langchain,
litellm, livekit_agents, llamaindex, mistral, openai,
openai_agents, openrouter, pydantic_ai, temporal).
Move the `instrumentation` parameter behind an `internal: dict` kwarg
on every span-creation surface. The `internal` dict is opaque to
external callers by design — the name signals \"reserved for
Braintrust SDK internals\" and the docstring says the keys inside
may change without notice.
Before:
start_span(name=\"my-op\", instrumentation=\"openai-auto\")
After:
start_span(name=\"my-op\", internal={\"instrumentation\": \"openai-auto\"})
Integration shadow helpers become:
def start_span(*args, **kwargs):
internal = dict(kwargs.get(\"internal\") or {})
internal.setdefault(\"instrumentation\", _INSTRUMENTATION)
kwargs[\"internal\"] = internal
return _bt_start_span(*args, **kwargs)
temporal and openai_agents (which open spans through Logger /
parent-Span methods rather than the module-level `start_span`) pass
`internal={\"instrumentation\": \"<provider>-auto\"}` explicitly.
SKILL.md updated accordingly.
Adds a `SpanInternalOptions(TypedDict, total=False)` in logger.py so
IDEs still get autocompletion / type-checking on the opaque
`internal={...}` kwarg. Currently declares one field:
class SpanInternalOptions(TypedDict, total=False):
instrumentation: str
Every span-creation surface (top-level `start_span`,
`Span.start_span`, `SpanImpl.__init__` / `.start_span`,
`Experiment.start_span`, `Dataset.start_span`, `Logger.start_span`,
`_NoopSpan.start_span`) now types the kwarg as
`internal: SpanInternalOptions | None = None`.
pyright + mypy pass.
Stephen Belanger (Qard)
approved these changes
Jul 16, 2026
Three integrations open child spans through Logger/Span method calls
that bypass the module-level `start_span` shadow, so those spans were
falling back to `braintrust-python-logger`:
- langchain/callbacks.py: `parent_span.start_span(...)` and
`init_logger().start_span(...)` in `_start_span`
- llamaindex/tracing.py: `parent_bt_span.start_span(...)` in
`on_start`
- google_genai/tracing.py: `logger._start_span_impl(...)` in the
streaming context helper
Each site now passes `internal={"instrumentation": _INSTRUMENTATION}`
explicitly. Matches the pattern already in place for
temporal/plugin.py and openai_agents/tracing.py.
Caught by the per-integration span_origin assertions in each
integration's test file.
Abhijeet Prasad (AbhiPrasad)
approved these changes
Jul 16, 2026
Abhijeet Prasad (AbhiPrasad)
enabled auto-merge (squash)
July 16, 2026 14:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every span an integration creates now carries a stable identifier in
context.span_origin.instrumentation.name—openai-auto,anthropic-auto,temporal-auto, and so on — matching the naming convention in the Braintrust instrumentation spec'sspan_originexample. Previously all integration spans were tagged with the sharedbraintrust-python-loggerdefault, which made per-integration filtering onspan_originimpossible.Builds on #570 (which added the
span_originscaffolding).API surface
start_span(and every provider-levelstart_spanmethod) gains aninternal: dict | None = Nonekwarg reserved for Braintrust SDK internals. Integrations passinternal={"instrumentation": "<provider>-auto"}to stamp the span origin; theinternalname signals to external callers that they shouldn't touch it and its keys can change without notice.start_span,Span.start_span,SpanImpl.start_span,Experiment.start_span,Dataset.start_span,Logger.start_span, and_NoopSpan.start_span.SpanImplreadsinternal["instrumentation"]and stamps it intocontext.span_origin.instrumentation.nameviamerge_span_origin_context.braintrust-python-loggerfor direct logging,braintrust-python-otelfor the OTel processor.start_spancall independently decides its own value, so a user's@tracedscorer nested inside a wrapped provider call keeps the default identifier — the integration's name only lands on spans the integration itself opens. This matches the spec's "the module that directly created the span" definition and mirrors how Sentry'soriginand OTel'sInstrumentationScopebehave.Suggested for external Braintrust integrations authored outside this repo:
Every module-scope
start_span(...)call then flows through the shadow with zero further edits.Integration migration (23 integrations)
21 shadow-based (
adk,agentscope,agno,anthropic,autogen,bedrock_runtime,claude_agent_sdk,cohere,crewai,dspy,google_genai,huggingface_hub,instructor,langchain,litellm,livekit_agents,llamaindex,mistral,openai,openai_agents,openrouter,pydantic_ai) — the shadow above at the top of eachtracing.py/callbacks.py.Explicit at call sites (2 integrations):
temporal/plugin.py:internal={"instrumentation": "temporal-auto"}at eachlogger.start_span(...)site — Logger method calls don't go through the module-level shadow.openai_agents/tracing.py:internal={"instrumentation": _INSTRUMENTATION}on the threecurrent_context.start_span(...)/self._logger.start_span(...)/parent.start_span(...)sites.Tests
py/src/braintrust/test_otel.py— new tests coveringmerge_span_origin_contextresolution and an end-to-endSpanImplassertion that child spans do NOT inherit the parent's instrumentation name (child falls back tobraintrust-python-logger).span["context"]["span_origin"]["instrumentation"]["name"] == "<provider>-auto"in place, next to the real provider-facing coverage. If a shadow ever regresses, that specific integration's test suite fails with a clear signal.Docs
.agents/skills/sdk-integrations/SKILL.mdgains a Span origin section documenting the shadow pattern, the no-inheritance rule, and the<provider>-autonaming convention.Validation
nox -s test_core: 647 passed, 66 skipped, 12 xfailednox -s test_types: 16 passed (pyright + mypy clean)nox -s pylint: successful (also ran ruff format + ruff check --fix as part of pre-commit)(latest)sessions — every one whose Python 3.14 wheels are available. Zero failures.test_livekit_agentsandtest_crewaiskipped due to pre-existing Python 3.14 wheel gaps inonnxruntime/pydantic-core, unrelated to this change.Follow-ups
braintrust-js-logger/braintrust-otel-jshardcoded and no per-integration wiring. Aligning JS on the same<provider>-autonames is worth doing before this ships so cross-SDK dashboards work — the spec only givesopenai-autoas an illustrative example, so a coordinated commit-to-a-convention is needed on both sides.docs/instrumentation-guide.mdjust showsopenai-autoin a JSON blob). Worth a spec PR to make it normative once JS and Python agree.Created by abhijeet
Slack thread