You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Stop requesting Observability API permissions for AI Teammates - #510
Merge gate: merge together with microsoft/agent365-skills#86 once the observability service authorizes AI Teammate instances by their blueprint's registration for all tenants. It is validated end to end below; until it is rolled out everywhere, AI Teammates without the OtelWrite role are still rejected.
Summary
a365 setup all no longer requests Observability API (Agent365.Observability.OtelWrite) permissions for AI Teammates, completing what #501 did for blueprint agents. a365 setup permissions bot and a365 create-instance no longer request them for any agent, so no CLI flow asks for Observability admin consent.
This closes two #501 follow-ups: the AI Teammate default, and setup permissions bot still configuring the Observability API.
Why
AI Teammates export telemetry the same way as blueprint agents: an app-only token for the agent identity, sent to the S2S endpoint. AI Teammate instances usually aren't registered themselves, so the service needs to authorize them through their blueprint's registration. Once it does that everywhere, the OtelWrite delegated grant and application role, and the admin consent they need, are unnecessary for AI Teammates too. Requesting them is why AI Teammate setup still shows the "Enable Agent 365 Telemetry" consent and tells non-admins that a Global Administrator must grant the app role.
Probed in a test tenant with an app-only token (no scp, roles=[]) whose parent app is an AI Teammate blueprint:
Agent identity
Result today
AI Teammate instance, unregistered, no OtelWrite role
403insufficient_scope
Registered agent instance, no role (control)
200
With the service change on, the first row now returns 200; see End-to-end validation below.
End-to-end validation
Validated against production with the service-side blueprint authorization turned on for a test tenant. The agent is a deployed AI Teammate instance that has no OtelWrite role or delegated grant, and that is not registered itself (only its blueprint is). It exports with the S2S exporter from microsoft/Agent365-Samples#339 and microsoft/Agent365-nodejs#290.
Token the agent sent (decoded, IDs redacted):
Claim
Value
aud
Agent 365 Observability API
azp
the AI Teammate's agent identity
xms_par_app_azp
the AI Teammate blueprint (different from azp)
idtyp
app
scp
absent
roles
[]
Requests to /observabilityService/tenants/{tenantId}/otlp/agents/{agentId}/traces:
Service-side blueprint authorization
Result
Off
403: the instance itself is not registered
On
200: authorized through the blueprint's registration; 1 invoke_agent span ingested
On the 200 the service also checked that the token's agent identity matched the agent in the URL and in the span payload. No OtelWrite role, delegated scope, or admin consent was involved.
The same token also returned 200 against the pre-production endpoint with the same service setting.
Behavior
AI Teammate setup all: no Observability API in inheritable permissions, grants, PowerShell hand-offs, or consent URLs, and the dry run and setup output say so. customBlueprintPermissions can still add the delegated scope back.
setup permissions bot: Bot API and Power Platform only. It no longer reports an S2S failure when it has no app role to grant.
create-instance: no Observability grant for the agent identity. It still rejects the ambiguous AzureUSGovernment environment before granting anything.
Fix, both agent types: a non-admin run whose customBlueprintPermissions adds the Observability API back now includes it in the combined consent URL. Before, the opt-back-in was stamped on the blueprint but left out of the admin hand-off.
Re-running setup does not revoke permissions granted earlier.
Upgrade note
The CHANGELOG upgrade note now covers both agent types. Grant the delegated permission only for agents that still export through the delegated route, and the application permission only if app-only exports return 403 insufficient_scope. The guided-setup docs, the setup README, and the design doc match.
Testing
Full suite: 2356 passed, 12 skipped (pre-existing), 0 failed. The Release build has 0 warnings.
New and updated tests cover the AI Teammate plan and config dry run (Observability omitted), setup permissions bot (no Observability API or app roles, no S2S outcome), create-instance (Power Platform granted, no Observability scopes), the non-admin consent URL with an opted-back-in Observability API, and EnsureUnambiguousEnvironment.
AI Teammates export telemetry with an app-only token through the S2S
endpoint, which can authorize them by their blueprint's registration.
Setup no longer requests Agent365.Observability.OtelWrite for them, and
"setup permissions bot" and "create-instance" no longer request it for
any agent.
A non-admin run whose customBlueprintPermissions adds the Observability
API back now includes it in the combined consent URL.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5b848d24-70d4-4288-a66a-9ca6e1282045
The reason will be displayed to describe this comment to others. Learn more.
Main blocker is the guided-setup docs still steering AI Teammates to the delegated route, which this change stops granting by default. Other comments are smaller.
- create-instance skips Observability entries saved in resourceConsents by
earlier setups unless customBlueprintPermissions opts back in, with a
regression test for both cases.
- Guided-setup observability doc points AI Teammates at the app-only S2S
exporter and marks the hosting-package token cache as a legacy delegated
route that needs the delegated OtelWrite grant added manually.
- AI Teammate dry run says when customBlueprintPermissions requests the
Observability API.
- setup permissions bot drops the unreachable S2S failure branch; tests drop
the unused s2sSucceeds parameter.
- Clarify SkipObservabilityPermissions and fix design.md tree indentation.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5b848d24-70d4-4288-a66a-9ca6e1282045
Setup reads config keys case-insensitively, so a mixed-case Observability
opt-in was honored by setup but ignored by create-instance.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5b848d24-70d4-4288-a66a-9ca6e1282045
The top-level lookup remains case-sensitive; the serializer options only apply to keys inside the array. A config with CustomBlueprintPermissions is accepted by ConfigService.LoadAsync (ConfigService.cs:102,151), but returns false here. With a saved Observability consent, instance creation therefore skips the explicitly requested delegated grant. Match the outer key case-insensitively too, and extend the opt-in regression test to cover a PascalCase top-level key.
- Restore Added entries orphaned under Breaking Changes by #385 and merge duplicate Changed/Fixed/Removed sections.
- Add missing entry for the develop-mcp publish error-message fix (#503).
- Drop stale entries for setup admin (removed in #393), create-instance re-enable (removed again in #397), and OtelWrite granted to all identities (superseded by #501/#510); add a Removed entry for setup admin.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
documentationImprovements or additions to documentation
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
github-actions
Bot
added
the
documentation
Rick Brighenti (rbrighenti)
left a comment
Summary
a365 setup allno longer requests Observability API (Agent365.Observability.OtelWrite) permissions for AI Teammates, completing what #501 did for blueprint agents.a365 setup permissions botanda365 create-instanceno longer request them for any agent, so no CLI flow asks for Observability admin consent.This closes two #501 follow-ups: the AI Teammate default, and
setup permissions botstill configuring the Observability API.Why
AI Teammates export telemetry the same way as blueprint agents: an app-only token for the agent identity, sent to the S2S endpoint. AI Teammate instances usually aren't registered themselves, so the service needs to authorize them through their blueprint's registration. Once it does that everywhere, the OtelWrite delegated grant and application role, and the admin consent they need, are unnecessary for AI Teammates too. Requesting them is why AI Teammate setup still shows the "Enable Agent 365 Telemetry" consent and tells non-admins that a Global Administrator must grant the app role.
Probed in a test tenant with an app-only token (no
scp,roles=[]) whose parent app is an AI Teammate blueprint:insufficient_scopeWith the service change on, the first row now returns 200; see End-to-end validation below.
End-to-end validation
Validated against production with the service-side blueprint authorization turned on for a test tenant. The agent is a deployed AI Teammate instance that has no OtelWrite role or delegated grant, and that is not registered itself (only its blueprint is). It exports with the S2S exporter from microsoft/Agent365-Samples#339 and microsoft/Agent365-nodejs#290.
Token the agent sent (decoded, IDs redacted):
audazpxms_par_app_azpazp)idtypappscproles[]Requests to
/observabilityService/tenants/{tenantId}/otlp/agents/{agentId}/traces:invoke_agentspan ingestedOn the 200 the service also checked that the token's agent identity matched the agent in the URL and in the span payload. No OtelWrite role, delegated scope, or admin consent was involved.
The same token also returned 200 against the pre-production endpoint with the same service setting.
Behavior
setup all: no Observability API in inheritable permissions, grants, PowerShell hand-offs, or consent URLs, and the dry run and setup output say so.customBlueprintPermissionscan still add the delegated scope back.setup permissions bot: Bot API and Power Platform only. It no longer reports an S2S failure when it has no app role to grant.create-instance: no Observability grant for the agent identity. It still rejects the ambiguousAzureUSGovernmentenvironment before granting anything.customBlueprintPermissionsadds the Observability API back now includes it in the combined consent URL. Before, the opt-back-in was stamped on the blueprint but left out of the admin hand-off.Upgrade note
The CHANGELOG upgrade note now covers both agent types. Grant the delegated permission only for agents that still export through the delegated route, and the application permission only if app-only exports return 403
insufficient_scope. The guided-setup docs, the setup README, and the design doc match.Testing
setup permissions bot(no Observability API or app roles, no S2S outcome),create-instance(Power Platform granted, no Observability scopes), the non-admin consent URL with an opted-back-in Observability API, andEnsureUnambiguousEnvironment.