Skip to content

feat(foundry): azd enable app insights in projects by default - #10743

Merged
Zhijie Huang (hund030) merged 17 commits into
Azure:mainfrom
kseager:feat/foundry-agent-default-tracing
Nov 6, 2026
Merged

Zhijie Huang (hund030) merged 17 commits into
Azure:mainfrom
kseager:feat/foundry-agent-default-tracing

Conversation

@kseager

@kseager Kayla Seager (kseager) commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

Automatically configure Application Insights for newly created Foundry projects through the normal provisioning flow, with no new command or flag required.

Summary

  • New projects: monitoring is synthesized into the main infrastructure graph and is on by default. It creates a workspace-based Application Insights component, its Log Analytics workspace, and a project-scoped AppInsights connection (ApiKey, credentials resolved internally). This works for native embedded ARM, editable Bicep, and Terraform. There is no separate imperative monitoring deployment.
  • Existing (BYO) projects: nothing is created and nothing changes unless the user sets monitoring.applicationInsightsResourceId. That connects an existing App Insights component (including cross-resource-group) to the project, preserving any existing compatible connection. monitoring: {enabled: true} without a resource ID fails with a clear error. Automatic creation of App Insights for existing projects is out of scope.
  • Opt-out: monitoring.enabled: false always wins, and opting out does not delete existing monitoring.
  • azure.yaml stays authoritative until ejection. After ejection, the exported Bicep/parameters or Terraform/tfvars are authoritative.
  • Terraform: uses ephemeral reads and write-only credentials, so the instrumentation key is never persisted in state or generated files. Monitoring-enabled exports require Terraform 1.11+ and AzAPI 2.7+; other exports keep their existing requirements.
  • Safety: a What-If preflight and a create-graph check protect new-project creation, and a template-export retry handles transient failures. Monitoring failures fail provisioning under normal provisioning semantics.
  • Shared synthesis module: synthesis now lives in the nested Go module azure.ai.projects/synthesis, shared by the Projects and Agents extensions via require + replace. The Projects ci-test.ps1 also runs its tests.

Validation

  • Live e2e (westus3) passed on native, Bicep and Terraform: new project with default monitoring, opt-out, existing project with a cross-RG BYO component, and the existing-project enabled: true error. For new projects, prompt and hosted agent invocations produced correlated traces in Application Insights.
  • Terraform: the instrumentation key is absent from state and generated files.
  • Unit tests, vet and format pass for the synthesis, Projects and Agents modules. Dependency alignment passes, and both ci-build.ps1 scripts succeed. Synthesis coverage is 84%.
  • The branch was rebased onto main after the live run (which included the upstream BYO AKS/ACR changes). Offline tests and builds pass, but the live e2e was not re-run on the rebased commit.

Extra

Fixes #9355

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@microsoft-github-policy-service microsoft-github-policy-service Bot added the customer-reported identify a customer issue label Oct 27, 2026
@microsoft-github-policy-service

Copy link
Copy Markdown
Contributor

Thank you for your contribution Kayla Seager (@kseager)! We will review the pull request and get back to you soon.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Project-target validation and monitoring configuration handling have unresolved defects that can misdirect credentials or create unintended billed resources.

5 open issues
What changed in this PR

This PR adds optional Application Insights tracing across the Foundry Projects, Agents, and Connections extensions, separating monitoring setup from required agent infrastructure and deployment.

Changes:

  • Enable monitoring when authoring new agent projects, with configuration, opt-out guidance, and cost notices.
  • Add isolated workspace-backed Application Insights provisioning and predeploy connection setup.
  • Add setup-status outputs, tests, documentation, and scaffold assertions.
File Description
cli/​azd/​extensions/​azure.ai.projects/​schemas/​azure.ai.project.json Defines optional monitoring configuration.
cli/​azd/​extensions/​azure.ai.projects/​README.md Documents setup, outputs, costs, and limitations.
cli/​azd/​extensions/​azure.ai.projects/​internal/​synthesis/​templates/​monitoring.bicep Defines isolated monitoring resources.
cli/​azd/​extensions/​azure.ai.projects/​internal/​synthesis/​templates/​monitoring.arm.json Adds the compiled monitoring template.
cli/​azd/​extensions/​azure.ai.projects/​internal/​synthesis/​templates_embed.go Embeds monitoring templates.
cli/​azd/​extensions/​azure.ai.projects/​internal/​synthesis/​synthesizer.go Captures the monitoring configuration node.
cli/​azd/​extensions/​azure.ai.projects/​internal/​synthesis/​monitoring.go Parses and validates monitoring settings.
cli/​azd/​extensions/​azure.ai.projects/​internal/​synthesis/​monitoring_test.go Tests parsing and template isolation.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​monitoring.go Implements optional provisioning and setup state.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​monitoring_test.go Tests failure isolation and output validation.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​foundry_provisioning_provider.go Integrates monitoring into provider operations.
cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​foundry_provisioning_provider_test.go Updates the preview-message assertion.
cli/​azd/​extensions/​azure.ai.projects/​internal/​cmd/​testdata/​TestAIHelp-add.snap Records help for --monitoring.
cli/​azd/​extensions/​azure.ai.projects/​internal/​cmd/​project_transaction_test.go Tests preservation of existing monitoring settings.
cli/​azd/​extensions/​azure.ai.projects/​internal/​cmd/​project_add.go Adds monitoring authoring and rollback handling.
cli/​azd/​extensions/​azure.ai.connections/​README.md Documents optional connection behavior.
cli/​azd/​extensions/​azure.ai.connections/​internal/​cmd/​root.go Registers the agent predeploy handler.
cli/​azd/​extensions/​azure.ai.connections/​internal/​cmd/​monitoring.go Implements connection reuse, creation, and status reporting.
cli/​azd/​extensions/​azure.ai.connections/​internal/​cmd/​monitoring_test.go Tests warnings and connection preservation.
cli/​azd/​extensions/​azure.ai.connections/​extension.yaml Declares lifecycle-event support.
cli/​azd/​extensions/​azure.ai.agents/​tests/​cli-interactive-tester-scenarios/​tier1/​1.01-init-template-python.yaml Adds monitoring scaffold and cost-notice assertions.
cli/​azd/​extensions/​azure.ai.agents/​README.md Explains default tracing and verification limits.
cli/​azd/​extensions/​azure.ai.agents/​internal/​synthesis/​templates/​monitoring.bicep Mirrors the monitoring resource template.
cli/​azd/​extensions/​azure.ai.agents/​internal/​synthesis/​templates/​monitoring.arm.json Mirrors the compiled monitoring template.
cli/​azd/​extensions/​azure.ai.agents/​internal/​synthesis/​templates_embed.go Embeds mirrored monitoring templates.
cli/​azd/​extensions/​azure.ai.agents/​internal/​synthesis/​synthesizer.go Mirrors monitoring configuration capture.
cli/​azd/​extensions/​azure.ai.agents/​internal/​synthesis/​monitoring.go Mirrors monitoring configuration parsing.
cli/​azd/​extensions/​azure.ai.agents/​internal/​cmd/​project_workflow.go Requests monitoring for new projects.
cli/​azd/​extensions/​azure.ai.agents/​internal/​cmd/​project_workflow_test.go Updates expected project-authoring arguments.

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 27, 2026 23:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Monitoring state refresh, variable resolution, and deployment-name collisions need correction, and connector safety checks need direct test coverage.

1 open issue
3 previously missed issues

In code that hasn't changed since last review

Medium severity Add transport-backed tests for monitoring auth and policy enforcement

cli/​azd/​extensions/​azure.ai.connections/​internal/​cmd/​monitoring.go:241

The predeploy tests replace h.connect with a stub, so none exercises these authentication/network policy checks or verifies that rejected components cause no connection PUT. Add transport-backed tests through connectMonitoring for disabled local authentication, disabled public ingestion, equivalent inherited/Entra connection reuse, and credential-bearing create errors. These are safety guarantees of default tracing, but the current tests would remain green if their enforcement or error sanitization were removed.

Medium severity Initialize reuse-only provider subscription before credential validation

cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​monitoring.go:159

For a freshly initialized reuse-only provider, p.subID is empty, so ensureCredential fails the host's subscription validation. Consequently, azd env refresh clears valid monitoring identifiers and changes ready to not-configured. Initialize the subscription from projectID before requesting credentials, and add a regression test through the reuse-only initialization and State path.

Medium severity Preserve project hash when truncating deployment names

cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​monitoring.go:368

For long deployment names, this truncation removes the project-path hash, so distinct projects with the same environment name get the same monitoring deployment name. If they share a resource group, their deployment records collide and concurrent setup can fail. Trim from the beginning instead to preserve the identity-bearing suffix within the 64-character limit.

5 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 27, 2026 23:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

An unset subscription causes fresh-provider refreshes to clear valid monitoring identifiers and block connection setup, while authentication and network-policy guards lack transport-backed regression coverage.

No open issues 🎉

2 previously missed issues

In code that hasn't changed since last review

Medium severity Add transport-backed tests for monitoring connection safety guards

cli/​azd/​extensions/​azure.ai.connections/​internal/​cmd/​monitoring.go:250

The current predeploy tests replace connect with a stub, so they never exercise these authentication/network-policy guards in connectMonitoring. Add transport-backed cases for DisableLocalAuth=true and disabled public ingestion that assert no connection PUT is sent. These safety checks could currently be removed or misread from the ARM response without failing any regression test.

Medium severity Initialize subscription before credential lookup for reused projects

cli/​azd/​extensions/​azure.ai.projects/​internal/​provisioning/​monitoring.go:159

For a reuse-only existing project, a fresh provider reaches this call with p.subID empty, so LookupTenant rejects the credential lookup. As a result, azd env refresh clears valid monitoring identifiers and saves not-configured, blocking subsequent connection setup until provisioning runs again. Initialize the subscription from projectID before requesting credentials, as provisionMonitoring does, and add a fresh-provider State regression test.

1 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Reported live validation predates the rebase, so a maintainer must verify final-commit provisioning, credential non-persistence, and teardown for default-on billable infrastructure across ARM, Bicep, and Terraform.

No open issues 🎉

Review effort: Balanced

Existing-project creation is no longer supported, so the opt-in test now
renders the new-project graph and maps its resource references to the test locals.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
Copilot AI balanced review requested due to automatic review settings November 2, 2026 20:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

A maintainer should validate billable-resource ownership, create/retry/cleanup behavior, and Terraform credential-state safety on the rebased commit because the reported live validation predates it.

No open issues 🎉

Review effort: Balanced

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed for a critical compatibility regression that causes existing private-network projects to fail provisioning even without monitoring opt-in.

No open issues 🎉

Review effort: Balanced

@hund030 Zhijie Huang (hund030) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for responding to the architectural feedback. The design is much cleaner and better aligned with ACR now! I still have a few questions about the implementation. Could you help clarify?

Kayla Ames and others added 2 commits November 3, 2026 08:20
Projects with a network: block that omit monitoring: no longer fail
synthesis; explicitly enabling monitoring there is still rejected.
Fix stale existing-project wording in the README.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Live Azure validation predates the rebase, so a maintainer must confirm ownership, connection preservation, and credential safety on the final ARM, Bicep, and Terraform deployment paths.

No open issues 🎉

Review effort: Balanced

Kayla Ames and others added 2 commits November 3, 2026 08:30
…f failing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because an unpublished critical finding shows that project-ID tags can exceed Azure’s length limit and cause default-on provisioning to fail for valid inputs.

No open issues 🎉

Review effort: Balanced

Port upstream BYO AKS supporting-resource changes to the relocated
azure.ai.projects/synthesis module and adapt new tests to monitoring.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the Application Insights integration and shared synthesis changes.

No open issues 🎉

Review effort: Balanced

Resolve conflicts with phase telemetry: keep the top-level synthesis package and add phasetelemetry to project_deployment.go and the provisioning provider.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The project ID used in monitoring tags can exceed Azure’s 256-character limit, causing otherwise valid projects to fail provisioning with default monitoring enabled.

1 open issue

Review effort: Balanced

The raw project resource ID can exceed Azure's 256-character tag value limit and fail provisioning. Tag the workspace and component with a UUIDv5 of the lowercased project ID, using the same namespace in Bicep and Terraform.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Live ARM, Bicep, and Terraform checks predate the BYO AKS/ACR rebase, so credential non-persistence and resource-ownership behavior need maintainer validation on the rebased commit.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the eng/pipelines change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c1d0003f-40c8-481d-92a5-ecb8d7afb491

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Projects’ Bicep ejection ignores synthesis warnings, silently exporting monitoringMode: none for network-enabled projects without informing users that automatic tracing was skipped.

No open issues 🎉

Review effort: Balanced

@hund030

Copy link
Copy Markdown
Contributor

/check-enforcer override

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ext-agents azure.ai.agents extension ext-connections azure.ai.connections extension ext-projects azure.ai.projects extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

App Insights is not created by default for activity protocol agents

6 participants