[mohammadatallah-scale]

Important

Pending Mohammad's self-review. This note is removed by a human, not automation.

Offline failure checks read attributes an agent declares on its spans, and AgentEx agents hand-build every key with nothing checking the values (OVE-1375). This adds a helper that writes and validates them from one spec, ported to scale-gp-beta in scaleapi/sgp-dev-python-staging#10.

• Spec: one validated object writes the root span's declarations as flat attribute keys, with the output schema as JSON text that is checked again when it renders.
• Per-span helpers: one call each for a required output, a verification result and an escalation that may name several failures, passed as a span's data.
• Spans: harness tool spans carry the tool keys, OpenAI hook results are JSON, model calls name their model, and a span cancelled mid-call reads failed.
• OpenTelemetry: otel_attributes turns any of these into span attributes, writing the verification rules as JSON text, and otel_span_id names a span to escalates.

Merge waits until public scale-agentex-python merges reopen after the vulnerability work, about end of November. Until then the Dogfood agent carries its own copy of the helper, as the end-to-end harness does.

Test plan

• The two-failure escalation, cancelled span and model id tests fail on the previous head and pass now.
• OpenTelemetry SDK spans written through the adapter pass the failure checks' escalation and verification_ran checks. Without it the SDK drops the verification rules and the check cannot judge, and the adapter tests fail against a pass-through adapter.
• Same results and error messages as the scale-gp-beta port over 160,000 generated inputs, including schema edits made after construction and random non-string keys.
• The failure checks' own span reader (OVE-1371 branch) reads Claude Code and Codex tool spans, passed or failed, as tool calls and decodes an object OpenAI result.
• gap: not exported to a live SGP spans API from an AgentEx agent, the span request tests cover the payload and the port's dev-sgp round trip covers the store.
• gap: a traced example whose spans pass the failure checks is not shown here, the end-to-end harness in OVE-1400 proves it.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no actionable issues remain from this review.

Summary

This PR adds validated attributes for offline failure checks and records tool names, model names, JSON results, and interrupted calls on spans.

• Agents can declare failure-check facts directly on their spans.
• Tool and model spans identify the calls they represent.
• OpenAI tool spans keep structured results in JSON text.
• Cancelled calls leave their spans marked as failed.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[FailureCheckSpec and per-span helpers] --> B[Attribute dictionary]
  B --> C[AgentEx span data]
  C --> D[SGP metadata]
  B --> E[otel_attributes]
  E --> F[OpenTelemetry span attributes]
  G[OpenTelemetry span] --> H[otel_span_id]
  H --> I[escalates]
  I --> B

Loading

_{Reviews (7) · Last reviewed commit: "feat(tracing): write failure-check attri..." · Reviewed by Greptile}

[mohammadatallah-scale]

P3 tests/lib/core/tracing/test_failure_attributes.py:154 - the "error names the attribute" contract is pinned only for step_budget, stop_reason and prompt_version. Relabelling escalation_channels as quote_arguments in _NORMALIZERS, or renaming the required_outputs, verification and VerificationRule labels, leaves all 118 tests green.
P3 failure_attributes.py:89 - json.dumps coerces non-string keys, so output_schema={1: "a", "1": "b"} is accepted and silently stored as {"1": "b"} instead of being rejected.

Refuted: __agent_version__ and __commit_sha__ README claims hold at this head (_add_source_to_span, code_revision), and the port's logic matches line for line apart from 3.9 syntax.

[mohammadatallah-scale]

P3 Follow-up: the attribute-naming pins landed in 815dad0, and each label swap above now fails 2 to 11 tests. The non-string key coercion is left open because fixing it changes behavior the scale-gp-beta port must match, so both helpers need it together.