Repository navigation
Add read-only preflight for existing AKS Foundry hosting - #10828
Yi Liu (yiliuTo) wants to merge 4 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Azure Pipelines: Successfully started running 3 pipeline(s). 24 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
🟡 Changes recommended
Critical preflight gaps let customized AKS deployments skip validation and can falsely confirm role-assignment authorization by accepting unevaluated conditions or checking the wrong scope.
5 open issues
Do not treat unevaluated conditional grants as confirmed authorization · New Detect AKS hosting from the compiled resource plan · New Check P07 authorization at the resolved grant scope and target · New Emit skipped-check reports through a persistent output channel · New Use existing network target scopes for write permission checks · New
What changed in this PR
Adds a read-only preflight to the Foundry provisioning provider’s existing-AKS deployment and ARM what-if paths, building on #10824’s supplied-resource support without changing deployment templates or teardown behavior.
Changes:
- Derives resources, grants, and operation scopes from compiled ARM templates and merged parameters.
- Adds AKS, supporting-resource, networking, and authorization checks with mandatory-failure gates.
- Adds compatibility tests, shared Bicep test utilities, Azure SDK dependencies, and documentation.
This pull request requires changes. Unresolved findings cover customized-template preflight bypasses, conditional and group-derived authorization, incorrect permission scopes, unsupported child-resource grants, evidence classification and reporting, warning persistence, production-path test coverage, unused code, and feature-stage documentation.
| File | Description |
|---|---|
docs/reference/feature-status.md |
Records existing-AKS hosting maturity and preflight support. |
docs/guides/creating-an-extension.md |
Links the read-only provisioning-provider example. |
docs/architecture/extension-framework.md |
Describes Foundry preflight integration. |
cli/azd/extensions/azure.ai.projects/README.md |
Documents checks, permissions, limitations, and remediation. |
cli/azd/extensions/azure.ai.projects/internal/testutil/bicep.go |
Centralizes local Bicep discovery and version validation. |
cli/azd/extensions/azure.ai.projects/internal/synthesis/templates_embed.go |
Clarifies preflight’s runtime Bicep usage. |
cli/azd/extensions/azure.ai.projects/internal/synthesis/schema_test.go |
Reuses shared Bicep test helpers. |
cli/azd/extensions/azure.ai.projects/internal/synthesis/hosting_test.go |
Updates Bicep-helper test calls. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/onboarding_naming.go |
Evaluates constant naming functions through Bicep. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/onboarding_naming_test.go |
Tests naming evaluation, failures, and cleanup. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/foundry_provisioning_provider.go |
Passes progress callbacks into preflight preparation. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/byo_template_plan.go |
Resolves compiled resources, nested deployments, and scopes. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/byo_template_plan_test.go |
Tests nested names, scopes, and references. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/byo_plan_bindings.go |
Maps resources, grants, and network operations into preflight. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/aks_hosting.go |
Runs preflight and reports results before binding resolution. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/aks_hosting_compatibility_test.go |
Covers binding contracts and customized-template compatibility. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/types.go |
Defines plans, evidence, reports, and blocking errors. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/resource_sdk.go |
Collects read-only Azure resource evidence. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/resource_rules.go |
Implements AKS, AgentPool, and Storage compatibility rules. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/resource_clients.go |
Shares ARM clients within a preflight pass. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/plan.go |
Parses resource IDs and provides test-used plan construction. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/pipeline.go |
Combines resource and authorization checks with deployment gating. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/parallel_reads.go |
Bounds concurrent evidence reads. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/operator_authorization.go |
Checks operator permissions and role-assignment authority. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/network_sdk.go |
Reads selected subnets and parent VNets. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/network_configuration.go |
Checks subnet delegation, capacity, containment, and overlap. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/managed_node_network.go |
Resolves managed node subnets through VMSS evidence. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/identity_authorization.go |
Evaluates supplied-identity grants and permissions. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/deployment_plan.go |
Models standard-network operations and permission scopes. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/authorization_test.go |
Tests authorization scopes, inheritance, caching, and failures. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/authorization_sdk.go |
Coordinates authorization evidence and scoped operation checks. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/authorization_rules.go |
Defines grant requirements, action matching, and caller claims. |
cli/azd/extensions/azure.ai.projects/internal/onboarding/authorization_evidence.go |
Caches authorization evidence within each pass. |
cli/azd/extensions/azure.ai.projects/go.sum |
Records added Azure SDK dependency checksums. |
cli/azd/extensions/azure.ai.projects/go.mod |
Adds Compute, Network, and Storage SDK dependencies. |
cli/azd/extensions/azure.ai.agents/README.md |
Links existing-AKS preflight documentation. |
cli/azd/extensions/azure.ai.agents/internal/synthesis/templates_embed.go |
Mirrors the Projects runtime-Bicep comment update. |
cli/azd/.vscode/cspell.yaml |
Adds file-scoped preflight terminology. |
Review effort: Max
| check.Missing = "operator object ID" | ||
| return check | ||
| } | ||
| check.Status, check.Observed, check.Cause = a.basicAction(ctx, scope, caller, roleAssignmentWrite, false) |
| plan, err := p.identifyBYOTemplatePlan(ctx, source) | ||
| if err != nil { | ||
| return err | ||
| } |
| if checkID == "P07" { | ||
| if grant.Missing != "" { | ||
| plan.RegistryGrantMissing = grant.Missing | ||
| } else { | ||
| plan.RegistryGrantPlanned = new(true) |
| if output.Len() > 0 { | ||
| progress(output.String()) |
| if node.env.scope.groupExists { | ||
| return node.env.scope.groupARMID | ||
| } | ||
| return node.env.scope.subscriptionARMID |
Preserve registry lifecycle validation, process feature gates, and parallel binding reads alongside read-only preflight. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
Critical preflight defects can bypass required authorization checks, block valid deployments, or falsely report sufficient permissions because identities, grants, and resource scopes are handled incorrectly.
17 open issues
Resolve guest principals in the resource tenant for RBAC checks · New Exempt future-resource grants only when the required grant is planned · New Resolve group membership when token claims omit group data · New Do not pass unsupported conditional role assignments · New Accept valid child-resource IDs for Blob container grant scopes · New Do not fall back to an unrelated cluster when selection is unresolved · New Skip unsupported child-resource grant scopes instead of blocking · New Use the existing subnet scope for join authorization · New Check deployment-write permission for nested network modules · New Inspect the compiled plan before skipping preflight · New Check P07 authorization at the resolved grant scope and target Detect AKS hosting from the compiled resource plan Do not treat unevaluated conditional grants as confirmed authorization Use existing network target scopes for write permission checks Emit skipped-check reports through a persistent output channel Remove or exercise the unused cluster deployment check · New Document the required opt-in for infrastructure provisioning · New
Review effort: Max
| token, tokenErr := a.credential.GetToken(ctx, policy.TokenRequestOptions{ | ||
| Scopes: []string{strings.TrimRight(audience, "/") + "/.default"}, | ||
| }) | ||
| caller, identityErr = identityFromToken(token.Token) |
| future := grant.checkID == "G02" && !plan.Workload.Supplied && facts.absent(plan.Workload) || | ||
| grant.checkID == "G03" && !plan.Storage.Supplied && facts.absent(plan.Storage) | ||
| if future { | ||
| report = append(report, resourceResult(check, NotApplicable, | ||
| "dependency does not yet exist; the same ARM graph will create the dependency and grant")) |
| return Unverified, "operator role assignments could not be read", err | ||
| } | ||
| provenAssignments := map[*armauthorization.RoleAssignment]bool{} | ||
| if caller.groupOverage { |
| if unconditionalOnly && (authString(p.Condition) != "" || authString(p.ConditionVersion) != "" || | ||
| authString(p.DelegatedManagedIdentityResourceID) != "") { | ||
| continue |
| segments := 9 | ||
| if strings.EqualFold(resourceType, "Microsoft.Network/virtualNetworks/subnets") { | ||
| segments = 11 | ||
| } |
| joinScope := resource.ID | ||
| if !resource.Supplied { | ||
| joinScope = vnet |
| joinScope = vnet | ||
| if resource.LifecycleMissing == "" { | ||
| plan.AddOperation("P03", "Microsoft.Network/virtualNetworks/subnets/write", vnet) | ||
| } |
| return nil, err | ||
| } | ||
| if err := p.prepareBYO(ctx, src); err != nil { | ||
| if err := p.prepareBYO(ctx, src, progress); err != nil { |
| // CheckClusterDeployment checks deployment-write permission in the AKS resource group without checking supporting resources. | ||
| func CheckClusterDeployment( | ||
| ctx context.Context, clusterID string, credential azcore.TokenCredential, options *arm.ClientOptions, | ||
| ) (Report, error) { | ||
| cluster, err := ParseID(clusterID, "Microsoft.ContainerService/managedClusters") | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| a := &scopedAuthorizer{credential: credential, options: onboardingARMOptions(options)} | ||
| a.reads = newAuthorizationEvidence(a.credential, a.options) | ||
| caller, tokenErr, identityErr := a.operatorIdentity(ctx) | ||
| report := a.checkOperations(ctx, []operation{ | ||
| {"P09", "Microsoft.Resources/deployments/write", cluster.Parent.String()}, | ||
| }, caller, tokenErr, identityErr) | ||
| return report, report.Err() | ||
| } |
| | Azure Functions | Stable | | ||
| | Azure Kubernetes Service (AKS) | Beta | | ||
| | Azure AI | Beta | | ||
| | [Microsoft Foundry on existing AKS](../../cli/azd/extensions/azure.ai.projects/README.md#existing-aks-hosting) (default or supplied supporting resources, read-only preflight) | In development (infrastructure enabled by default; Agents init/adoption disabled by default) | |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Active AKS hosting can bypass preflight when its parameter is disabled or absent, and P07 can falsely approve role assignments despite assignment restrictions or insufficient permissions at the actual scope.
16 open issues
Inspect the compiled plan before skipping preflight Check deployment-write permission for nested network modules Use the existing subnet scope for join authorization Skip unsupported child-resource grant scopes instead of blocking Accept valid child-resource IDs for Blob container grant scopes Do not pass unsupported conditional role assignments Resolve group membership when token claims omit group data Exempt future-resource grants only when the required grant is planned Resolve guest principals in the resource tenant for RBAC checks Check P07 authorization at the resolved grant scope and target Detect AKS hosting from the compiled resource plan Do not treat unevaluated conditional grants as confirmed authorization Use existing network target scopes for write permission checks Emit skipped-check reports through a persistent output channel Document the required opt-in for infrastructure provisioning Remove or exercise the unused cluster deployment check
1 resolved since last review
Review effort: Max
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Templates with hard-coded AKS hosting or a differently named parameter can bypass AKS, resource, and authorization preflight checks in both deployment and preview.
16 open issues
Inspect the compiled plan before skipping preflight Check deployment-write permission for nested network modules Use the existing subnet scope for join authorization Skip unsupported child-resource grant scopes instead of blocking Accept valid child-resource IDs for Blob container grant scopes Do not pass unsupported conditional role assignments Resolve group membership when token claims omit group data Exempt future-resource grants only when the required grant is planned Resolve guest principals in the resource tenant for RBAC checks Check P07 authorization at the resolved grant scope and target Detect AKS hosting from the compiled resource plan Do not treat unevaluated conditional grants as confirmed authorization Use existing network target scopes for write permission checks Emit skipped-check reports through a persistent output channel Document the required opt-in for infrastructure provisioning Remove or exercise the unused cluster deployment check
Review effort: Max
azure.ai.agents PR buildNote This is an unsigned development build. Install it only if you trust this PR. Install the extension: azd ext install "https://azuresdkartifacts.z5.web.core.windows.net/azd/extensions/pr/10828/azure-ai-agents.zip"
|



Summary
Validation
azure.ai.projectsandazure.ai.agents.go test ./... -count=1. The color-sensitive command tests requireNO_COLORto be unset at process startup.Infra|BYOAKS|BYOAks.Related: #10824.