diff --git a/run-samples.sh b/run-samples.sh index c101365..15aee08 100755 --- a/run-samples.sh +++ b/run-samples.sh @@ -70,8 +70,8 @@ TERRAFORM_SAMPLES=( "samples/web-app-cosmosdb-mongodb-api/dotnet/terraform|bash deploy.sh" "samples/web-app-managed-identity/python/terraform|bash deploy.sh" "samples/web-app-managed-identity/dotnet/terraform|bash deploy.sh" - "samples/web-app-sql-database/python/terraform|bash deploy.sh" - "samples/web-app-sql-database/dotnet/terraform|bash deploy.sh" + "samples/web-app-sql-database/python/terraform|bash deploy.sh|bash ../scripts/validate.sh" + "samples/web-app-sql-database/dotnet/terraform|bash deploy.sh|bash ../scripts/validate.sh" "samples/web-app-mysql-flexible-server/python/terraform|bash deploy.sh" "samples/web-app-mysql-flexible-server/dotnet/terraform|bash deploy.sh" "samples/web-app-postgresql-flexible-server/python/terraform|bash deploy.sh" diff --git a/samples/web-app-sql-database/dotnet/README.md b/samples/web-app-sql-database/dotnet/README.md index 399940b..04b7548 100644 --- a/samples/web-app-sql-database/dotnet/README.md +++ b/samples/web-app-sql-database/dotnet/README.md @@ -1,6 +1,6 @@ # Azure Web App with Azure SQL Database and Azure Key Vault -This sample demonstrates a ASP.NET Core Razor Pages single-page web application called *Vacation Planner* hosted on an [Azure Web App](https://learn.microsoft.com/en-us/azure/app-service/overview). The app runs on an Azure App Service Plan and stores activity data in an `activities` table within the `sampledb` database on an [Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/) instance. The connection string of the SQL database is stored as a secret in [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview). The application also retrieves its certificate from Key Vault to serve traffic over HTTPS. +This sample demonstrates a ASP.NET Core Razor Pages single-page web application called *Vacation Planner* hosted on an [Azure Web App](https://learn.microsoft.com/en-us/azure/app-service/overview). The app runs on an Azure App Service Plan and stores activity data in an `activities` table within the `sampledb` database on an [Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/) instance. The connection string of the SQL database is stored as a secret in [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview). The application also retrieves its certificate from Key Vault to serve traffic over HTTPS. The SQL server encrypts its databases at rest with [Transparent Data Encryption (TDE)](https://learn.microsoft.com/en-us/azure/azure-sql/database/transparent-data-encryption-byok-overview) protected by a customer-managed key: an RSA key in Key Vault that the server reaches through a user-assigned managed identity. ## Architecture @@ -11,8 +11,9 @@ The following diagram illustrates the architecture of the solution: - **Azure Web App**: Hosts the ASP.NET Core application - **Azure App Service Plan**: Provides compute resources for the web app -- **Azure SQL Database**: Stores activity data in a relational table -- **Azure Key Vault**: Stores the database connection string and the certificate used to secure HTTPS traffic +- **Azure SQL Database**: Stores activity data in a relational table, encrypted at rest with TDE +- **Azure Key Vault**: Stores the database connection string, the certificate used to secure HTTPS traffic, and the RSA key that serves as the TDE protector of the SQL server +- **User-Assigned Managed Identity**: The identity the SQL server uses to wrap and unwrap its database encryption keys with the Key Vault key ## Prerequisites @@ -44,12 +45,16 @@ The Vacation Planner Web App supports two common approaches for accessing Azure This flexibility allows the app to run securely in Azure or in emulated environments like [LocalStack for Azure](https://docs.localstack.cloud/azure/). The client code supports both authentication modes using [`ClientSecretCredential`](https://learn.microsoft.com/en-us/dotnet/api/azure.identity.clientsecretcredential) or [`DefaultAzureCredential`](https://learn.microsoft.com/en-us/dotnet/api/azure.identity.defaultazurecredential) from the Azure SDK. ## Azure Key Vault Integration -The application integrates with Azure Key Vault for managing secrets and certificates: +The application integrates with Azure Key Vault for managing secrets and certificates, and the SQL server uses a Key Vault key to protect its data at rest: Secrets: The SQL connection string is stored as a secret in Key Vault. At runtime, the app retrieves it using the Azure Key Vault Secrets SDK. This is configured via the KEY_VAULT_NAME and SECRET_NAME environment variables. Certificates: A self-signed certificate is created in Key Vault during deployment. The app exposes a GET /api/certificate endpoint that retrieves the certificate using the Azure Key Vault Certificates SDK and returns its name, confirming the integration works. This is configured via the KEYVAULT_URI and CERT_NAME environment variables. +Keys: An RSA key in Key Vault is the TDE protector of the SQL server, the customer-managed key that encrypts the database encryption key of every database on the server. The server reaches the key through its user-assigned managed identity, which holds the `get`, `wrapKey` and `unwrapKey` key permissions, and picks up new versions of the key automatically (auto-rotation). Azure requires soft delete and purge protection on the vault. With purge protection, a deleted vault cannot be purged: after the resource group is deleted, the vault stays soft-deleted for the 7-day retention period, and its name cannot be reused anywhere until then. To redeploy to Azure within that window, change `PREFIX` or `SUFFIX` in the deployment script you use and in `scripts/validate.sh` and `scripts/call-web-app.sh`. + +On LocalStack, the emulator registers the key on the server and checks that the key exists, but it does not encrypt the database with it. The Azure CLI variant registers the key and the protector with `az resource create`, because `az sql server key create` and `az sql server tde-key set` only accept key ids on the public Key Vault domains and reject the ones the emulator issues. + ## Deployment Set up the Azure emulator using the LocalStack for Azure Docker image. Before starting, ensure you have a valid `LOCALSTACK_AUTH_TOKEN` to access the Azure emulator. Refer to the [Auth Token guide](https://docs.localstack.cloud/getting-started/auth-token/) to obtain your Auth Token and set it in the `LOCALSTACK_AUTH_TOKEN` environment variable. The Azure Docker image is available on the [LocalStack Docker Hub](https://hub.docker.com/r/localstack/localstack-azure). To pull the image, execute: diff --git a/samples/web-app-sql-database/dotnet/bicep/README.md b/samples/web-app-sql-database/dotnet/bicep/README.md index 83500c1..98a764c 100644 --- a/samples/web-app-sql-database/dotnet/bicep/README.md +++ b/samples/web-app-sql-database/dotnet/bicep/README.md @@ -40,7 +40,8 @@ The [deploy.sh](deploy.sh) script creates the [Azure Resource Group](https://lea 3. [Azure App Service Plan](https://learn.microsoft.com/en-us/azure/app-service/overview-hosting-plans): The compute resource that hosts the web application. 4. [Azure Web App](https://learn.microsoft.com/en-us/azure/app-service/overview): Hosts the ASP.NET Core Razor Pages single-page application (*Vacation Planner*), connected to Azure SQL Database. 5. [App Service Source Control](https://learn.microsoft.com/en-us/rest/api/appservice/web-apps/create-or-update-source-control?view=rest-appservice-2024-11-01): (Optional) Configures automatic deployment from a public GitHub repository. -6. [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview): Stores the SQL connection string in a secret. +6. [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview): Stores the SQL connection string in a secret and the RSA key that serves as the [TDE protector](https://learn.microsoft.com/en-us/azure/azure-sql/database/transparent-data-encryption-byok-overview) of the SQL server, registered by the [transparent-data-encryption.bicep](modules/transparent-data-encryption.bicep) module. +7. [User-Assigned Managed Identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview): The identity the SQL server uses to reach its TDE protector key in Key Vault. The web app allows users to plan and manage vacation activities, storing all activity data in the `Activities` table in the `PlannerDB` database. For more information, see [Azure Web App with Azure SQL Database and Azure Key Vault](../README.md). @@ -100,6 +101,7 @@ SQL_DATABASE_NAME='PlannerDB' WEB_APP_NAME="${PREFIX}-webapp-${SUFFIX}" KEY_VAULT_NAME="${PREFIX}-kv-${SUFFIX}" SECRET_NAME="${PREFIX}-secret-${SUFFIX}" +TDE_KEY_NAME="${PREFIX}-tde-key-${SUFFIX}" # Check resource group echo -e "[$RESOURCE_GROUP_NAME] resource group:\n" @@ -130,6 +132,44 @@ az sql db show \ --resource-group "$RESOURCE_GROUP_NAME" \ --output table +# Check that the Key Vault key is the TDE protector of the Azure SQL Server +echo -e "\n[$SQL_SERVER_NAME] SQL server TDE protector:\n" +# Read the key through Azure Resource Manager: the Bicep variant grants the caller no Key Vault data-plane access. +KEY_VAULT_ID=$(az keyvault show \ +--name "$KEY_VAULT_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--query "id" \ +--output tsv) +TDE_KEY_ID=$(az resource show \ +--ids "$KEY_VAULT_ID/keys/$TDE_KEY_NAME" \ +--api-version 2024-11-01 \ +--query "properties.keyUriWithVersion" \ +--output tsv) +TDE_PROTECTOR=$(az sql server tde-key show \ +--server "$SQL_SERVER_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--output json) +echo "$TDE_PROTECTOR" | jq '{serverKeyType, uri, autoRotationEnabled}' +if [[ "$(jq -r .serverKeyType <<< "$TDE_PROTECTOR")" != "AzureKeyVault" || + "$(jq -r .uri <<< "$TDE_PROTECTOR")" != "$TDE_KEY_ID" || + "$(jq -r .autoRotationEnabled <<< "$TDE_PROTECTOR")" != "true" ]]; then + echo "The TDE protector of [$SQL_SERVER_NAME] is not the auto-rotated Key Vault key [$TDE_KEY_ID]" + exit 1 +fi + +# Check that TDE is enabled on the Azure SQL Database +TDE_STATE=$(az sql db tde show \ +--database "$SQL_DATABASE_NAME" \ +--server "$SQL_SERVER_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--query "state" \ +--output tsv) +echo -e "\n[$SQL_DATABASE_NAME] SQL database TDE state: [$TDE_STATE]" +if [[ "$TDE_STATE" != "Enabled" ]]; then + echo "TDE is not enabled on [$SQL_DATABASE_NAME]" + exit 1 +fi + # Check Azure Key Vault echo -e "\n[$KEY_VAULT_NAME] Key Vault:\n" az keyvault show \ diff --git a/samples/web-app-sql-database/dotnet/bicep/main.bicep b/samples/web-app-sql-database/dotnet/bicep/main.bicep index e5bec57..31075ed 100644 --- a/samples/web-app-sql-database/dotnet/bicep/main.bicep +++ b/samples/web-app-sql-database/dotnet/bicep/main.bicep @@ -154,9 +154,6 @@ param administratorLoginPassword string = 'P@ssw0rd1234!' @description('Conditional. The Azure Active Directory (AAD) administrator authentication. Required if no `administratorLogin` & `administratorLoginPassword` is provided.') param administrators object? -@description('Specifies the conditional Developmentresource ID of a user-assigned identityDevelopment to be used by default. This is required if `userAssignedIdentities` is not empty.') -param primaryUserAssignedIdentityResourceId string? - @allowed([ '1.0' '1.1' @@ -325,15 +322,25 @@ var webAppName = '${prefix}-webapp-${suffix}' var appServicePlanName = '${prefix}-app-service-plan-${suffix}' var keyVaultName = '${prefix}-kv-${suffix}' var sqlConnectionStringSecretName = '${prefix}-secret-${suffix}' -var identity = { - type: 'SystemAssigned' - } +var sqlServerIdentityName = '${prefix}-tde-identity-${suffix}' +var tdeKeyName = '${prefix}-tde-key-${suffix}' + +resource sqlServerIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = { + name: sqlServerIdentityName + location: location + tags: tags +} resource sqlServer 'Microsoft.Sql/servers@2024-05-01-preview' = { name: sqlServerName location: location tags: tags - identity: identity + identity: { + type: 'UserAssigned' + userAssignedIdentities: { + '${sqlServerIdentity.id}': {} + } + } properties: { administratorLogin: administratorLogin administratorLoginPassword: administratorLoginPassword @@ -342,7 +349,7 @@ resource sqlServer 'Microsoft.Sql/servers@2024-05-01-preview' = { isIPv6Enabled: isIPv6Enabled version: version minimalTlsVersion: minimalTlsVersion - primaryUserAssignedIdentityId: primaryUserAssignedIdentityResourceId + primaryUserAssignedIdentityId: sqlServerIdentity.id publicNetworkAccess: publicNetworkAccess restrictOutboundNetworkAccess: restrictOutboundNetworkAccess } @@ -453,10 +460,45 @@ resource keyVault 'Microsoft.KeyVault/vaults@2023-07-01' = { ] } } + { + tenantId: subscription().tenantId + objectId: sqlServerIdentity.properties.principalId + permissions: { + keys: [ + 'get' + 'wrapKey' + 'unwrapKey' + ] + } + } ] enableRbacAuthorization: false enableSoftDelete: true softDeleteRetentionInDays: 7 + enablePurgeProtection: true + } +} + +resource tdeKey 'Microsoft.KeyVault/vaults/keys@2024-11-01' = { + parent: keyVault + name: tdeKeyName + properties: { + kty: 'RSA' + keySize: 2048 + keyOps: [ + 'wrapKey' + 'unwrapKey' + ] + } +} + +module transparentDataEncryption 'modules/transparent-data-encryption.bicep' = { + name: 'transparentDataEncryption' + params: { + sqlServerName: sqlServer.name + keyVaultName: keyVault.name + keyName: tdeKey.name + keyUri: tdeKey.properties.keyUriWithVersion } } diff --git a/samples/web-app-sql-database/dotnet/bicep/modules/transparent-data-encryption.bicep b/samples/web-app-sql-database/dotnet/bicep/modules/transparent-data-encryption.bicep new file mode 100644 index 0000000..95d4a10 --- /dev/null +++ b/samples/web-app-sql-database/dotnet/bicep/modules/transparent-data-encryption.bicep @@ -0,0 +1,35 @@ +@description('Specifies the name of the SQL logical server.') +param sqlServerName string + +@description('Specifies the name of the Key Vault that holds the TDE protector key.') +param keyVaultName string + +@description('Specifies the name of the Key Vault key used as the TDE protector.') +param keyName string + +@description('Specifies the versioned URI of the Key Vault key used as the TDE protector.') +param keyUri string + +resource sqlServer 'Microsoft.Sql/servers@2024-05-01-preview' existing = { + name: sqlServerName +} + +// A server key must be named after the vault, key and key version it points to. +resource serverKey 'Microsoft.Sql/servers/keys@2023-08-01' = { + parent: sqlServer + name: '${keyVaultName}_${keyName}_${last(split(keyUri, '/'))}' + properties: { + serverKeyType: 'AzureKeyVault' + uri: keyUri + } +} + +resource encryptionProtector 'Microsoft.Sql/servers/encryptionProtector@2023-08-01' = { + parent: sqlServer + name: 'current' + properties: { + serverKeyType: 'AzureKeyVault' + serverKeyName: serverKey.name + autoRotationEnabled: true + } +} diff --git a/samples/web-app-sql-database/dotnet/scripts/README.md b/samples/web-app-sql-database/dotnet/scripts/README.md index 4891865..c9abdda 100644 --- a/samples/web-app-sql-database/dotnet/scripts/README.md +++ b/samples/web-app-sql-database/dotnet/scripts/README.md @@ -40,7 +40,8 @@ The [deploy.sh](deploy.sh) Bash script creates the following Azure resources usi 4. [Azure App Service Plan](https://learn.microsoft.com/en-us/azure/app-service/overview-hosting-plans): The compute resource that hosts the web application. 5. [Azure Web App](https://learn.microsoft.com/en-us/azure/app-service/overview): Hosts the ASP.NET Core Razor Pages single-page application (*Vacation Planner*), connected to Azure SQL Database. 6. [App Service Source Control](https://learn.microsoft.com/en-us/rest/api/appservice/web-apps/create-or-update-source-control?view=rest-appservice-2024-11-01): (Optional) Configures automatic deployment from a public GitHub repository. -7. [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview): Stores the SQL connection string in a secret. +7. [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview): Stores the SQL connection string in a secret and the RSA key that serves as the [TDE protector](https://learn.microsoft.com/en-us/azure/azure-sql/database/transparent-data-encryption-byok-overview) of the SQL server. +8. [User-Assigned Managed Identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview): The identity the SQL server uses to reach its TDE protector key in Key Vault. The system implements a Vacation Planner web application that stores and retrieves activity data from Azure SQL Database. For more information, see [Azure Web App with Azure SQL Database and Azure Key Vault](../README.md). @@ -100,6 +101,7 @@ SQL_DATABASE_NAME='PlannerDB' WEB_APP_NAME="${PREFIX}-webapp-${SUFFIX}" KEY_VAULT_NAME="${PREFIX}-kv-${SUFFIX}" SECRET_NAME="${PREFIX}-secret-${SUFFIX}" +TDE_KEY_NAME="${PREFIX}-tde-key-${SUFFIX}" # Check resource group echo -e "[$RESOURCE_GROUP_NAME] resource group:\n" @@ -130,6 +132,44 @@ az sql db show \ --resource-group "$RESOURCE_GROUP_NAME" \ --output table +# Check that the Key Vault key is the TDE protector of the Azure SQL Server +echo -e "\n[$SQL_SERVER_NAME] SQL server TDE protector:\n" +# Read the key through Azure Resource Manager: the Bicep variant grants the caller no Key Vault data-plane access. +KEY_VAULT_ID=$(az keyvault show \ +--name "$KEY_VAULT_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--query "id" \ +--output tsv) +TDE_KEY_ID=$(az resource show \ +--ids "$KEY_VAULT_ID/keys/$TDE_KEY_NAME" \ +--api-version 2024-11-01 \ +--query "properties.keyUriWithVersion" \ +--output tsv) +TDE_PROTECTOR=$(az sql server tde-key show \ +--server "$SQL_SERVER_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--output json) +echo "$TDE_PROTECTOR" | jq '{serverKeyType, uri, autoRotationEnabled}' +if [[ "$(jq -r .serverKeyType <<< "$TDE_PROTECTOR")" != "AzureKeyVault" || + "$(jq -r .uri <<< "$TDE_PROTECTOR")" != "$TDE_KEY_ID" || + "$(jq -r .autoRotationEnabled <<< "$TDE_PROTECTOR")" != "true" ]]; then + echo "The TDE protector of [$SQL_SERVER_NAME] is not the auto-rotated Key Vault key [$TDE_KEY_ID]" + exit 1 +fi + +# Check that TDE is enabled on the Azure SQL Database +TDE_STATE=$(az sql db tde show \ +--database "$SQL_DATABASE_NAME" \ +--server "$SQL_SERVER_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--query "state" \ +--output tsv) +echo -e "\n[$SQL_DATABASE_NAME] SQL database TDE state: [$TDE_STATE]" +if [[ "$TDE_STATE" != "Enabled" ]]; then + echo "TDE is not enabled on [$SQL_DATABASE_NAME]" + exit 1 +fi + # Check Azure Key Vault echo -e "\n[$KEY_VAULT_NAME] Key Vault:\n" az keyvault show \ diff --git a/samples/web-app-sql-database/dotnet/scripts/deploy.sh b/samples/web-app-sql-database/dotnet/scripts/deploy.sh index eee4ed8..48ffaa8 100755 --- a/samples/web-app-sql-database/dotnet/scripts/deploy.sh +++ b/samples/web-app-sql-database/dotnet/scripts/deploy.sh @@ -6,6 +6,7 @@ SUFFIX='test' LOCATION='westeurope' RESOURCE_GROUP_NAME="${PREFIX}-rg" SQL_SERVER_NAME="${PREFIX}-sqlserver-${SUFFIX}" +SQL_SERVER_IDENTITY_NAME="${PREFIX}-tde-identity-${SUFFIX}" FIREWALL_RULE_NAME="AllowAllIPs" ADMIN_USER='sqladmin' ADMIN_PASSWORD='P@ssw0rd1234!' @@ -24,6 +25,7 @@ DEPLOY_APP=1 KEY_VAULT_NAME="${PREFIX}-kv-${SUFFIX}" SECRET_NAME="${PREFIX}-secret-${SUFFIX}" CERT_NAME="${PREFIX}-cert-${SUFFIX}" +TDE_KEY_NAME="${PREFIX}-tde-key-${SUFFIX}" # Change the current directory to the script's directory cd "$CURRENT_DIR" || exit @@ -41,6 +43,38 @@ else exit 1 fi +# Create the user-assigned managed identity the SQL server uses to reach the TDE protector key +echo "Creating user-assigned managed identity [$SQL_SERVER_IDENTITY_NAME]..." +az identity create \ + --name "$SQL_SERVER_IDENTITY_NAME" \ + --resource-group "$RESOURCE_GROUP_NAME" \ + --location "$LOCATION" \ + --only-show-errors 1>/dev/null + +if [ $? -eq 0 ]; then + echo "User-assigned managed identity [$SQL_SERVER_IDENTITY_NAME] created successfully." +else + echo "Failed to create user-assigned managed identity [$SQL_SERVER_IDENTITY_NAME]." + exit 1 +fi + +SQL_SERVER_IDENTITY_ID=$(az identity show \ + --name "$SQL_SERVER_IDENTITY_NAME" \ + --resource-group "$RESOURCE_GROUP_NAME" \ + --query "id" \ + --output tsv) + +SQL_SERVER_IDENTITY_PRINCIPAL_ID=$(az identity show \ + --name "$SQL_SERVER_IDENTITY_NAME" \ + --resource-group "$RESOURCE_GROUP_NAME" \ + --query "principalId" \ + --output tsv) + +if [[ -z "$SQL_SERVER_IDENTITY_ID" || -z "$SQL_SERVER_IDENTITY_PRINCIPAL_ID" ]]; then + echo "Failed to retrieve the resource ID or principalId of [$SQL_SERVER_IDENTITY_NAME]" + exit 1 +fi + # Create a sql server echo "Checking if [$SQL_SERVER_NAME] sql server exists in the [$RESOURCE_GROUP_NAME] resource group..." az sql server show \ @@ -60,7 +94,9 @@ else --admin-user $ADMIN_USER \ --admin-password $ADMIN_PASSWORD \ --assign-identity \ - --identity-type SystemAssigned \ + --identity-type UserAssigned \ + --user-assigned-identity-id "$SQL_SERVER_IDENTITY_ID" \ + --primary-user-assigned-identity-id "$SQL_SERVER_IDENTITY_ID" \ --minimal-tls-version 1.2 \ --tags environment=test \ --only-show-errors 1>/dev/null @@ -334,6 +370,8 @@ az keyvault create \ --resource-group "$RESOURCE_GROUP_NAME" \ --location "$LOCATION" \ --enable-rbac-authorization false \ + --enable-purge-protection true \ + --retention-days 7 \ --only-show-errors 1>/dev/null if [ $? -eq 0 ]; then @@ -397,6 +435,78 @@ else exit 1 fi +# Assign access policy to the SQL server managed identity +echo "Assigning Key Vault access policy to the SQL server identity [$SQL_SERVER_IDENTITY_NAME]..." +az keyvault set-policy \ + --name "$KEY_VAULT_NAME" \ + --object-id "$SQL_SERVER_IDENTITY_PRINCIPAL_ID" \ + --key-permissions get wrapKey unwrapKey \ + --only-show-errors 1>/dev/null + +if [ $? -eq 0 ]; then + echo "Key Vault access policy for [$SQL_SERVER_IDENTITY_NAME] assigned successfully." +else + echo "Failed to assign Key Vault access policy for [$SQL_SERVER_IDENTITY_NAME]." + exit 1 +fi + +# Create the RSA key that protects the database encryption keys of the SQL server +echo "Creating key [$TDE_KEY_NAME] in Key Vault [$KEY_VAULT_NAME]..." +TDE_KEY_ID=$(az keyvault key create \ + --vault-name "$KEY_VAULT_NAME" \ + --name "$TDE_KEY_NAME" \ + --kty RSA \ + --size 2048 \ + --ops wrapKey unwrapKey \ + --query "key.kid" \ + --output tsv \ + --only-show-errors) + +if [ -n "$TDE_KEY_ID" ]; then + echo "Key [$TDE_KEY_ID] created successfully." +else + echo "Failed to create key [$TDE_KEY_NAME] in Key Vault [$KEY_VAULT_NAME]." + exit 1 +fi + +# Register the key on the SQL server and make it the TDE protector. The generic az resource create +# is used because az sql server key create and az sql server tde-key set only accept key ids on the +# public Key Vault domains, which rejects the key ids the LocalStack emulator issues. +SQL_SERVER_ID=$(az sql server show \ + --name "$SQL_SERVER_NAME" \ + --resource-group "$RESOURCE_GROUP_NAME" \ + --query "id" \ + --output tsv) +SERVER_KEY_NAME="${KEY_VAULT_NAME}_${TDE_KEY_NAME}_${TDE_KEY_ID##*/}" + +echo "Adding key [$TDE_KEY_NAME] to the [$SQL_SERVER_NAME] sql server..." +az resource create \ + --id "$SQL_SERVER_ID/keys/$SERVER_KEY_NAME" \ + --api-version 2023-08-01 \ + --properties "{\"serverKeyType\": \"AzureKeyVault\", \"uri\": \"$TDE_KEY_ID\"}" \ + --only-show-errors 1>/dev/null + +if [ $? -eq 0 ]; then + echo "Key [$TDE_KEY_NAME] added successfully to the [$SQL_SERVER_NAME] sql server." +else + echo "Failed to add key [$TDE_KEY_NAME] to the [$SQL_SERVER_NAME] sql server." + exit 1 +fi + +echo "Setting key [$TDE_KEY_NAME] as the TDE protector of the [$SQL_SERVER_NAME] sql server..." +az resource create \ + --id "$SQL_SERVER_ID/encryptionProtector/current" \ + --api-version 2023-08-01 \ + --properties "{\"serverKeyType\": \"AzureKeyVault\", \"serverKeyName\": \"$SERVER_KEY_NAME\", \"autoRotationEnabled\": true}" \ + --only-show-errors 1>/dev/null + +if [ $? -eq 0 ]; then + echo "Key [$TDE_KEY_NAME] set successfully as the TDE protector of the [$SQL_SERVER_NAME] sql server." +else + echo "Failed to set key [$TDE_KEY_NAME] as the TDE protector of the [$SQL_SERVER_NAME] sql server." + exit 1 +fi + # Get Key Vault URI echo "Retrieving Key Vault URI..." KEYVAULT_URI=$(az keyvault show \ diff --git a/samples/web-app-sql-database/dotnet/scripts/validate.sh b/samples/web-app-sql-database/dotnet/scripts/validate.sh index 19d5997..c6ca9bf 100755 --- a/samples/web-app-sql-database/dotnet/scripts/validate.sh +++ b/samples/web-app-sql-database/dotnet/scripts/validate.sh @@ -9,6 +9,7 @@ SQL_DATABASE_NAME='PlannerDB' WEB_APP_NAME="${PREFIX}-webapp-${SUFFIX}" KEY_VAULT_NAME="${PREFIX}-kv-${SUFFIX}" SECRET_NAME="${PREFIX}-secret-${SUFFIX}" +TDE_KEY_NAME="${PREFIX}-tde-key-${SUFFIX}" # Check resource group echo -e "[$RESOURCE_GROUP_NAME] resource group:\n" @@ -39,6 +40,44 @@ az sql db show \ --resource-group "$RESOURCE_GROUP_NAME" \ --output table +# Check that the Key Vault key is the TDE protector of the Azure SQL Server +echo -e "\n[$SQL_SERVER_NAME] SQL server TDE protector:\n" +# Read the key through Azure Resource Manager: the Bicep variant grants the caller no Key Vault data-plane access. +KEY_VAULT_ID=$(az keyvault show \ +--name "$KEY_VAULT_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--query "id" \ +--output tsv) +TDE_KEY_ID=$(az resource show \ +--ids "$KEY_VAULT_ID/keys/$TDE_KEY_NAME" \ +--api-version 2024-11-01 \ +--query "properties.keyUriWithVersion" \ +--output tsv) +TDE_PROTECTOR=$(az sql server tde-key show \ +--server "$SQL_SERVER_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--output json) +echo "$TDE_PROTECTOR" | jq '{serverKeyType, uri, autoRotationEnabled}' +if [[ "$(jq -r .serverKeyType <<< "$TDE_PROTECTOR")" != "AzureKeyVault" || + "$(jq -r .uri <<< "$TDE_PROTECTOR")" != "$TDE_KEY_ID" || + "$(jq -r .autoRotationEnabled <<< "$TDE_PROTECTOR")" != "true" ]]; then + echo "The TDE protector of [$SQL_SERVER_NAME] is not the auto-rotated Key Vault key [$TDE_KEY_ID]" + exit 1 +fi + +# Check that TDE is enabled on the Azure SQL Database +TDE_STATE=$(az sql db tde show \ +--database "$SQL_DATABASE_NAME" \ +--server "$SQL_SERVER_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--query "state" \ +--output tsv) +echo -e "\n[$SQL_DATABASE_NAME] SQL database TDE state: [$TDE_STATE]" +if [[ "$TDE_STATE" != "Enabled" ]]; then + echo "TDE is not enabled on [$SQL_DATABASE_NAME]" + exit 1 +fi + # Check Azure Key Vault echo -e "\n[$KEY_VAULT_NAME] Key Vault:\n" az keyvault show \ diff --git a/samples/web-app-sql-database/dotnet/terraform/README.md b/samples/web-app-sql-database/dotnet/terraform/README.md index fac861f..4a776a0 100644 --- a/samples/web-app-sql-database/dotnet/terraform/README.md +++ b/samples/web-app-sql-database/dotnet/terraform/README.md @@ -40,8 +40,9 @@ The [main.tf](main.tf) Terraform module creates the following Azure resources: 3. [Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/): The `PlannerDB` database storing relational vacation activity data. 4. [Azure App Service Plan](https://learn.microsoft.com/en-us/azure/app-service/overview-hosting-plans): The compute resource that hosts the web application. 5. [Azure Web App](https://learn.microsoft.com/en-us/azure/app-service/overview): Hosts the ASP.NET Core Razor Pages single-page application (*Vacation Planner*), connected to Azure SQL Database. -6. [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview): Stores the SQL connection string as a secret and a self-signed certificate for HTTPS. +6. [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview): Stores the SQL connection string as a secret, a self-signed certificate for HTTPS, and the RSA key that serves as the [TDE protector](https://learn.microsoft.com/en-us/azure/azure-sql/database/transparent-data-encryption-byok-overview) of the SQL server. 7. [App Service Source Control](https://learn.microsoft.com/en-us/rest/api/appservice/web-apps/create-or-update-source-control?view=rest-appservice-2024-11-01): (Optional) Configures automatic deployment from a public GitHub repository. +8. [User-Assigned Managed Identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview): The identity the SQL server uses to reach its TDE protector key in Key Vault. The system implements a Vacation Planner web application that stores and retrieves activity data from Azure SQL Database. For more information, see [Azure Web App with Azure SQL Database and Azure Key Vault](../README.md). @@ -124,6 +125,7 @@ SQL_DATABASE_NAME='PlannerDB' WEB_APP_NAME="${PREFIX}-webapp-${SUFFIX}" KEY_VAULT_NAME="${PREFIX}-kv-${SUFFIX}" SECRET_NAME="${PREFIX}-secret-${SUFFIX}" +TDE_KEY_NAME="${PREFIX}-tde-key-${SUFFIX}" # Check resource group echo -e "[$RESOURCE_GROUP_NAME] resource group:\n" @@ -154,6 +156,44 @@ az sql db show \ --resource-group "$RESOURCE_GROUP_NAME" \ --output table +# Check that the Key Vault key is the TDE protector of the Azure SQL Server +echo -e "\n[$SQL_SERVER_NAME] SQL server TDE protector:\n" +# Read the key through Azure Resource Manager: the Bicep variant grants the caller no Key Vault data-plane access. +KEY_VAULT_ID=$(az keyvault show \ +--name "$KEY_VAULT_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--query "id" \ +--output tsv) +TDE_KEY_ID=$(az resource show \ +--ids "$KEY_VAULT_ID/keys/$TDE_KEY_NAME" \ +--api-version 2024-11-01 \ +--query "properties.keyUriWithVersion" \ +--output tsv) +TDE_PROTECTOR=$(az sql server tde-key show \ +--server "$SQL_SERVER_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--output json) +echo "$TDE_PROTECTOR" | jq '{serverKeyType, uri, autoRotationEnabled}' +if [[ "$(jq -r .serverKeyType <<< "$TDE_PROTECTOR")" != "AzureKeyVault" || + "$(jq -r .uri <<< "$TDE_PROTECTOR")" != "$TDE_KEY_ID" || + "$(jq -r .autoRotationEnabled <<< "$TDE_PROTECTOR")" != "true" ]]; then + echo "The TDE protector of [$SQL_SERVER_NAME] is not the auto-rotated Key Vault key [$TDE_KEY_ID]" + exit 1 +fi + +# Check that TDE is enabled on the Azure SQL Database +TDE_STATE=$(az sql db tde show \ +--database "$SQL_DATABASE_NAME" \ +--server "$SQL_SERVER_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--query "state" \ +--output tsv) +echo -e "\n[$SQL_DATABASE_NAME] SQL database TDE state: [$TDE_STATE]" +if [[ "$TDE_STATE" != "Enabled" ]]; then + echo "TDE is not enabled on [$SQL_DATABASE_NAME]" + exit 1 +fi + # Check Azure Key Vault echo -e "\n[$KEY_VAULT_NAME] Key Vault:\n" az keyvault show \ diff --git a/samples/web-app-sql-database/dotnet/terraform/main.tf b/samples/web-app-sql-database/dotnet/terraform/main.tf index 6353bc9..089a413 100644 --- a/samples/web-app-sql-database/dotnet/terraform/main.tf +++ b/samples/web-app-sql-database/dotnet/terraform/main.tf @@ -6,6 +6,8 @@ locals { app_service_plan_name = "${var.prefix}-app-service-plan-${var.suffix}" web_app_name = "${var.prefix}-webapp-${var.suffix}" key_vault_name = "${var.prefix}-kv-${var.suffix}" + sql_identity_name = "${var.prefix}-tde-identity-${var.suffix}" + tde_key_name = "${var.prefix}-tde-key-${var.suffix}" } # Retrieve the current Azure client configuration @@ -18,6 +20,20 @@ resource "azurerm_resource_group" "example" { tags = var.tags } +# Create the user-assigned managed identity the SQL server uses to reach the TDE protector key +resource "azurerm_user_assigned_identity" "sql_server" { + name = local.sql_identity_name + resource_group_name = azurerm_resource_group.example.name + location = azurerm_resource_group.example.location + tags = var.tags + + lifecycle { + ignore_changes = [ + tags + ] + } +} + # Create a SQL server resource "azurerm_mssql_server" "example" { name = local.sql_server_name @@ -29,11 +45,19 @@ resource "azurerm_mssql_server" "example" { public_network_access_enabled = var.public_network_access_enabled outbound_network_restriction_enabled = var.outbound_network_restriction_enabled version = var.sql_version + primary_user_assigned_identity_id = azurerm_user_assigned_identity.sql_server.id tags = var.tags + identity { + type = "UserAssigned" + identity_ids = [azurerm_user_assigned_identity.sql_server.id] + } + + # The TDE protector is managed by azurerm_mssql_server_transparent_data_encryption below lifecycle { ignore_changes = [ - tags + tags, + transparent_data_encryption_key_vault_key_id ] } } @@ -138,6 +162,7 @@ resource "azurerm_key_vault" "example" { sku_name = "standard" rbac_authorization_enabled = false soft_delete_retention_days = 7 + purge_protection_enabled = true tags = var.tags lifecycle { @@ -163,11 +188,72 @@ resource "azurerm_key_vault_access_policy" "web_app" { ] } +# Grant the identity running Terraform access to manage the Key Vault key, secret and certificate +resource "azurerm_key_vault_access_policy" "deployer" { + key_vault_id = azurerm_key_vault.example.id + tenant_id = data.azurerm_client_config.current.tenant_id + object_id = data.azurerm_client_config.current.object_id + + key_permissions = [ + "Create", + "Delete", + "Get", + "GetRotationPolicy", + ] + + secret_permissions = [ + "Delete", + "Get", + "Set", + ] + + certificate_permissions = [ + "Create", + "Delete", + "Get", + ] +} + +# Grant the SQL server managed identity access to the TDE protector key +resource "azurerm_key_vault_access_policy" "sql_server" { + key_vault_id = azurerm_key_vault.example.id + tenant_id = data.azurerm_client_config.current.tenant_id + object_id = azurerm_user_assigned_identity.sql_server.principal_id + + key_permissions = [ + "Get", + "UnwrapKey", + "WrapKey", + ] +} + +# Create the RSA key that protects the database encryption keys of the SQL server +resource "azurerm_key_vault_key" "tde" { + name = local.tde_key_name + key_vault_id = azurerm_key_vault.example.id + key_type = "RSA" + key_size = 2048 + key_opts = ["unwrapKey", "wrapKey"] + + depends_on = [azurerm_key_vault_access_policy.deployer] +} + +# Make the Key Vault key the TDE protector of the SQL server +resource "azurerm_mssql_server_transparent_data_encryption" "example" { + server_id = azurerm_mssql_server.example.id + key_vault_key_id = azurerm_key_vault_key.tde.id + auto_rotation_enabled = true + + depends_on = [azurerm_key_vault_access_policy.sql_server] +} + # Create a Key Vault secret for SQL connection string resource "azurerm_key_vault_secret" "sql_connection_string" { name = var.secret_name value = "Server=tcp:${azurerm_mssql_server.example.fully_qualified_domain_name},1433;Database=${azurerm_mssql_database.example.name};User ID=${var.sql_database_username};Password=${var.sql_database_password};Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;" key_vault_id = azurerm_key_vault.example.id + + depends_on = [azurerm_key_vault_access_policy.deployer] } # Create a self-signed certificate in Key Vault @@ -201,4 +287,6 @@ resource "azurerm_key_vault_certificate" "example" { ] } } + + depends_on = [azurerm_key_vault_access_policy.deployer] } diff --git a/samples/web-app-sql-database/python/README.md b/samples/web-app-sql-database/python/README.md index 66c1eac..9f090e9 100644 --- a/samples/web-app-sql-database/python/README.md +++ b/samples/web-app-sql-database/python/README.md @@ -1,6 +1,6 @@ # Azure Web App with Azure SQL Database and Azure Key Vault -This sample demonstrates a Python Flask single-page web application called *Vacation Planner* hosted on an [Azure Web App](https://learn.microsoft.com/en-us/azure/app-service/overview). The app runs on an Azure App Service Plan and stores activity data in an `activities` table within the `sampledb` database on an [Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/) instance. The connection string of the SQL database is stored as a secret in [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview). The application also retrieves its certificate from Key Vault to serve traffic over HTTPS. +This sample demonstrates a Python Flask single-page web application called *Vacation Planner* hosted on an [Azure Web App](https://learn.microsoft.com/en-us/azure/app-service/overview). The app runs on an Azure App Service Plan and stores activity data in an `activities` table within the `sampledb` database on an [Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/) instance. The connection string of the SQL database is stored as a secret in [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview). The application also retrieves its certificate from Key Vault to serve traffic over HTTPS. The SQL server encrypts its databases at rest with [Transparent Data Encryption (TDE)](https://learn.microsoft.com/en-us/azure/azure-sql/database/transparent-data-encryption-byok-overview) protected by a customer-managed key: an RSA key in Key Vault that the server reaches through a user-assigned managed identity. ## Architecture @@ -11,8 +11,9 @@ The following diagram illustrates the architecture of the solution: - **Azure Web App**: Hosts the Python Flask application - **Azure App Service Plan**: Provides compute resources for the web app -- **Azure SQL Database**: Stores activity data in a relational table -- **Azure Key Vault**: Stores the database connection string and the certificate used to secure HTTPS traffic +- **Azure SQL Database**: Stores activity data in a relational table, encrypted at rest with TDE +- **Azure Key Vault**: Stores the database connection string, the certificate used to secure HTTPS traffic, and the RSA key that serves as the TDE protector of the SQL server +- **User-Assigned Managed Identity**: The identity the SQL server uses to wrap and unwrap its database encryption keys with the Key Vault key ## Prerequisites @@ -44,12 +45,16 @@ The Vacation Planner Web App supports two common approaches for accessing Azure This flexibility allows the app to run securely in Azure or in emulated environments like [LocalStack for Azure](https://docs.localstack.cloud/azure/). The client code supports both authentication modes using [`ClientSecretCredential`](https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.clientsecretcredential?view=azure-python) or [`DefaultAzureCredential`](https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) from the Azure SDK. ## Azure Key Vault Integration -The application integrates with Azure Key Vault for managing secrets and certificates: +The application integrates with Azure Key Vault for managing secrets and certificates, and the SQL server uses a Key Vault key to protect its data at rest: Secrets: The SQL connection string is stored as a secret in Key Vault. At runtime, the app retrieves it using the Azure Key Vault Secrets SDK. This is configured via the KEY_VAULT_NAME and SECRET_NAME environment variables. Certificates: A self-signed certificate is created in Key Vault during deployment. The app exposes a GET /api/certificate endpoint that retrieves the certificate using the Azure Key Vault Certificates SDK and returns its name, confirming the integration works. This is configured via the KEYVAULT_URI and CERT_NAME environment variables. +Keys: An RSA key in Key Vault is the TDE protector of the SQL server, the customer-managed key that encrypts the database encryption key of every database on the server. The server reaches the key through its user-assigned managed identity, which holds the `get`, `wrapKey` and `unwrapKey` key permissions, and picks up new versions of the key automatically (auto-rotation). Azure requires soft delete and purge protection on the vault. With purge protection, a deleted vault cannot be purged: after the resource group is deleted, the vault stays soft-deleted for the 7-day retention period, and its name cannot be reused anywhere until then. To redeploy to Azure within that window, change `PREFIX` or `SUFFIX` in the deployment script you use and in `scripts/validate.sh` and `scripts/call-web-app.sh`. + +On LocalStack, the emulator registers the key on the server and checks that the key exists, but it does not encrypt the database with it. The Azure CLI variant registers the key and the protector with `az resource create`, because `az sql server key create` and `az sql server tde-key set` only accept key ids on the public Key Vault domains and reject the ones the emulator issues. + ## Deployment Set up the Azure emulator using the LocalStack for Azure Docker image. Before starting, ensure you have a valid `LOCALSTACK_AUTH_TOKEN` to access the Azure emulator. Refer to the [Auth Token guide](https://docs.localstack.cloud/getting-started/auth-token/) to obtain your Auth Token and set it in the `LOCALSTACK_AUTH_TOKEN` environment variable. The Azure Docker image is available on the [LocalStack Docker Hub](https://hub.docker.com/r/localstack/localstack-azure). To pull the image, execute: diff --git a/samples/web-app-sql-database/python/bicep/README.md b/samples/web-app-sql-database/python/bicep/README.md index 90a73e4..fd3ca0c 100644 --- a/samples/web-app-sql-database/python/bicep/README.md +++ b/samples/web-app-sql-database/python/bicep/README.md @@ -40,7 +40,8 @@ The [deploy.sh](deploy.sh) script creates the [Azure Resource Group](https://lea 3. [Azure App Service Plan](https://learn.microsoft.com/en-us/azure/app-service/overview-hosting-plans): The compute resource that hosts the web application. 4. [Azure Web App](https://learn.microsoft.com/en-us/azure/app-service/overview): Hosts the Python Flask single-page application (*Vacation Planner*), connected to Azure SQL Database. 5. [App Service Source Control](https://learn.microsoft.com/en-us/rest/api/appservice/web-apps/create-or-update-source-control?view=rest-appservice-2024-11-01): (Optional) Configures automatic deployment from a public GitHub repository. -6. [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview): Stores the SQL connection string in a secret. +6. [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview): Stores the SQL connection string in a secret and the RSA key that serves as the [TDE protector](https://learn.microsoft.com/en-us/azure/azure-sql/database/transparent-data-encryption-byok-overview) of the SQL server, registered by the [transparent-data-encryption.bicep](modules/transparent-data-encryption.bicep) module. +7. [User-Assigned Managed Identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview): The identity the SQL server uses to reach its TDE protector key in Key Vault. The web app allows users to plan and manage vacation activities, storing all activity data in the `Activities` table in the `PlannerDB` database. For more information, see [Azure Web App with Azure SQL Database and Azure Key Vault](../README.md). @@ -120,6 +121,12 @@ az sql db show \ --server local-sqlserver-test \ --resource-group local-rg \ --output table + +# Check the TDE protector of the Azure SQL Server +az sql server tde-key show \ +--server local-sqlserver-test \ +--resource-group local-rg \ +--output table ``` ## Cleanup diff --git a/samples/web-app-sql-database/python/bicep/main.bicep b/samples/web-app-sql-database/python/bicep/main.bicep index cdd900c..f077ace 100644 --- a/samples/web-app-sql-database/python/bicep/main.bicep +++ b/samples/web-app-sql-database/python/bicep/main.bicep @@ -153,9 +153,6 @@ param administratorLoginPassword string = 'P@ssw0rd1234!' @description('Conditional. The Azure Active Directory (AAD) administrator authentication. Required if no `administratorLogin` & `administratorLoginPassword` is provided.') param administrators object? -@description('Specifies the conditional Developmentresource ID of a user-assigned identityDevelopment to be used by default. This is required if `userAssignedIdentities` is not empty.') -param primaryUserAssignedIdentityResourceId string? - @allowed([ '1.0' '1.1' @@ -324,15 +321,25 @@ var webAppName = '${prefix}-webapp-${suffix}' var appServicePlanName = '${prefix}-app-service-plan-${suffix}' var keyVaultName = '${prefix}-kv-${suffix}' var sqlConnectionStringSecretName = '${prefix}-secret-${suffix}' -var identity = { - type: 'SystemAssigned' - } +var sqlServerIdentityName = '${prefix}-tde-identity-${suffix}' +var tdeKeyName = '${prefix}-tde-key-${suffix}' + +resource sqlServerIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = { + name: sqlServerIdentityName + location: location + tags: tags +} resource sqlServer 'Microsoft.Sql/servers@2024-05-01-preview' = { name: sqlServerName location: location tags: tags - identity: identity + identity: { + type: 'UserAssigned' + userAssignedIdentities: { + '${sqlServerIdentity.id}': {} + } + } properties: { administratorLogin: administratorLogin administratorLoginPassword: administratorLoginPassword @@ -341,7 +348,7 @@ resource sqlServer 'Microsoft.Sql/servers@2024-05-01-preview' = { isIPv6Enabled: isIPv6Enabled version: version minimalTlsVersion: minimalTlsVersion - primaryUserAssignedIdentityId: primaryUserAssignedIdentityResourceId + primaryUserAssignedIdentityId: sqlServerIdentity.id publicNetworkAccess: publicNetworkAccess restrictOutboundNetworkAccess: restrictOutboundNetworkAccess } @@ -452,10 +459,45 @@ resource keyVault 'Microsoft.KeyVault/vaults@2023-07-01' = { ] } } + { + tenantId: subscription().tenantId + objectId: sqlServerIdentity.properties.principalId + permissions: { + keys: [ + 'get' + 'wrapKey' + 'unwrapKey' + ] + } + } ] enableRbacAuthorization: false enableSoftDelete: true softDeleteRetentionInDays: 7 + enablePurgeProtection: true + } +} + +resource tdeKey 'Microsoft.KeyVault/vaults/keys@2024-11-01' = { + parent: keyVault + name: tdeKeyName + properties: { + kty: 'RSA' + keySize: 2048 + keyOps: [ + 'wrapKey' + 'unwrapKey' + ] + } +} + +module transparentDataEncryption 'modules/transparent-data-encryption.bicep' = { + name: 'transparentDataEncryption' + params: { + sqlServerName: sqlServer.name + keyVaultName: keyVault.name + keyName: tdeKey.name + keyUri: tdeKey.properties.keyUriWithVersion } } diff --git a/samples/web-app-sql-database/python/bicep/modules/transparent-data-encryption.bicep b/samples/web-app-sql-database/python/bicep/modules/transparent-data-encryption.bicep new file mode 100644 index 0000000..95d4a10 --- /dev/null +++ b/samples/web-app-sql-database/python/bicep/modules/transparent-data-encryption.bicep @@ -0,0 +1,35 @@ +@description('Specifies the name of the SQL logical server.') +param sqlServerName string + +@description('Specifies the name of the Key Vault that holds the TDE protector key.') +param keyVaultName string + +@description('Specifies the name of the Key Vault key used as the TDE protector.') +param keyName string + +@description('Specifies the versioned URI of the Key Vault key used as the TDE protector.') +param keyUri string + +resource sqlServer 'Microsoft.Sql/servers@2024-05-01-preview' existing = { + name: sqlServerName +} + +// A server key must be named after the vault, key and key version it points to. +resource serverKey 'Microsoft.Sql/servers/keys@2023-08-01' = { + parent: sqlServer + name: '${keyVaultName}_${keyName}_${last(split(keyUri, '/'))}' + properties: { + serverKeyType: 'AzureKeyVault' + uri: keyUri + } +} + +resource encryptionProtector 'Microsoft.Sql/servers/encryptionProtector@2023-08-01' = { + parent: sqlServer + name: 'current' + properties: { + serverKeyType: 'AzureKeyVault' + serverKeyName: serverKey.name + autoRotationEnabled: true + } +} diff --git a/samples/web-app-sql-database/python/scripts/README.md b/samples/web-app-sql-database/python/scripts/README.md index 3543fa2..f2d001a 100644 --- a/samples/web-app-sql-database/python/scripts/README.md +++ b/samples/web-app-sql-database/python/scripts/README.md @@ -40,7 +40,8 @@ The [deploy.sh](deploy.sh) Bash script creates the following Azure resources usi 4. [Azure App Service Plan](https://learn.microsoft.com/en-us/azure/app-service/overview-hosting-plans): The compute resource that hosts the web application. 5. [Azure Web App](https://learn.microsoft.com/en-us/azure/app-service/overview): Hosts the Python Flask single-page application (*Vacation Planner*), connected to Azure SQL Database. 6. [App Service Source Control](https://learn.microsoft.com/en-us/rest/api/appservice/web-apps/create-or-update-source-control?view=rest-appservice-2024-11-01): (Optional) Configures automatic deployment from a public GitHub repository. -7. [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview): Stores the SQL connection string in a secret. +7. [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview): Stores the SQL connection string in a secret and the RSA key that serves as the [TDE protector](https://learn.microsoft.com/en-us/azure/azure-sql/database/transparent-data-encryption-byok-overview) of the SQL server. +8. [User-Assigned Managed Identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview): The identity the SQL server uses to reach its TDE protector key in Key Vault. The system implements a Vacation Planner web application that stores and retrieves activity data from Azure SQL Database. For more information, see [Azure Web App with Azure SQL Database and Azure Key Vault](../README.md). @@ -120,6 +121,12 @@ az sql db show \ --server local-sqlserver-test \ --resource-group local-rg \ --output table + +# Check the TDE protector of the Azure SQL Server +az sql server tde-key show \ +--server local-sqlserver-test \ +--resource-group local-rg \ +--output table ``` ## Cleanup diff --git a/samples/web-app-sql-database/python/scripts/deploy.sh b/samples/web-app-sql-database/python/scripts/deploy.sh index 58d66a5..47d00b7 100755 --- a/samples/web-app-sql-database/python/scripts/deploy.sh +++ b/samples/web-app-sql-database/python/scripts/deploy.sh @@ -6,6 +6,7 @@ SUFFIX='test' LOCATION='westeurope' RESOURCE_GROUP_NAME="${PREFIX}-rg" SQL_SERVER_NAME="${PREFIX}-sqlserver-${SUFFIX}" +SQL_SERVER_IDENTITY_NAME="${PREFIX}-tde-identity-${SUFFIX}" FIREWALL_RULE_NAME="AllowAllIPs" ADMIN_USER='sqladmin' ADMIN_PASSWORD='P@ssw0rd1234!' @@ -24,6 +25,7 @@ DEPLOY_APP=1 KEY_VAULT_NAME="${PREFIX}-kv-${SUFFIX}" SECRET_NAME="${PREFIX}-secret-${SUFFIX}" CERT_NAME="${PREFIX}-cert-${SUFFIX}" +TDE_KEY_NAME="${PREFIX}-tde-key-${SUFFIX}" # Change the current directory to the script's directory cd "$CURRENT_DIR" || exit @@ -41,6 +43,38 @@ else exit 1 fi +# Create the user-assigned managed identity the SQL server uses to reach the TDE protector key +echo "Creating user-assigned managed identity [$SQL_SERVER_IDENTITY_NAME]..." +az identity create \ + --name "$SQL_SERVER_IDENTITY_NAME" \ + --resource-group "$RESOURCE_GROUP_NAME" \ + --location "$LOCATION" \ + --only-show-errors 1>/dev/null + +if [ $? -eq 0 ]; then + echo "User-assigned managed identity [$SQL_SERVER_IDENTITY_NAME] created successfully." +else + echo "Failed to create user-assigned managed identity [$SQL_SERVER_IDENTITY_NAME]." + exit 1 +fi + +SQL_SERVER_IDENTITY_ID=$(az identity show \ + --name "$SQL_SERVER_IDENTITY_NAME" \ + --resource-group "$RESOURCE_GROUP_NAME" \ + --query "id" \ + --output tsv) + +SQL_SERVER_IDENTITY_PRINCIPAL_ID=$(az identity show \ + --name "$SQL_SERVER_IDENTITY_NAME" \ + --resource-group "$RESOURCE_GROUP_NAME" \ + --query "principalId" \ + --output tsv) + +if [[ -z "$SQL_SERVER_IDENTITY_ID" || -z "$SQL_SERVER_IDENTITY_PRINCIPAL_ID" ]]; then + echo "Failed to retrieve the resource ID or principalId of [$SQL_SERVER_IDENTITY_NAME]" + exit 1 +fi + # Create a sql server echo "Checking if [$SQL_SERVER_NAME] sql server exists in the [$RESOURCE_GROUP_NAME] resource group..." az sql server show \ @@ -60,7 +94,9 @@ else --admin-user $ADMIN_USER \ --admin-password $ADMIN_PASSWORD \ --assign-identity \ - --identity-type SystemAssigned \ + --identity-type UserAssigned \ + --user-assigned-identity-id "$SQL_SERVER_IDENTITY_ID" \ + --primary-user-assigned-identity-id "$SQL_SERVER_IDENTITY_ID" \ --minimal-tls-version 1.2 \ --tags environment=test \ --only-show-errors 1>/dev/null @@ -334,6 +370,8 @@ az keyvault create \ --resource-group "$RESOURCE_GROUP_NAME" \ --location "$LOCATION" \ --enable-rbac-authorization false \ + --enable-purge-protection true \ + --retention-days 7 \ --only-show-errors 1>/dev/null if [ $? -eq 0 ]; then @@ -397,6 +435,78 @@ else exit 1 fi +# Assign access policy to the SQL server managed identity +echo "Assigning Key Vault access policy to the SQL server identity [$SQL_SERVER_IDENTITY_NAME]..." +az keyvault set-policy \ + --name "$KEY_VAULT_NAME" \ + --object-id "$SQL_SERVER_IDENTITY_PRINCIPAL_ID" \ + --key-permissions get wrapKey unwrapKey \ + --only-show-errors 1>/dev/null + +if [ $? -eq 0 ]; then + echo "Key Vault access policy for [$SQL_SERVER_IDENTITY_NAME] assigned successfully." +else + echo "Failed to assign Key Vault access policy for [$SQL_SERVER_IDENTITY_NAME]." + exit 1 +fi + +# Create the RSA key that protects the database encryption keys of the SQL server +echo "Creating key [$TDE_KEY_NAME] in Key Vault [$KEY_VAULT_NAME]..." +TDE_KEY_ID=$(az keyvault key create \ + --vault-name "$KEY_VAULT_NAME" \ + --name "$TDE_KEY_NAME" \ + --kty RSA \ + --size 2048 \ + --ops wrapKey unwrapKey \ + --query "key.kid" \ + --output tsv \ + --only-show-errors) + +if [ -n "$TDE_KEY_ID" ]; then + echo "Key [$TDE_KEY_ID] created successfully." +else + echo "Failed to create key [$TDE_KEY_NAME] in Key Vault [$KEY_VAULT_NAME]." + exit 1 +fi + +# Register the key on the SQL server and make it the TDE protector. The generic az resource create +# is used because az sql server key create and az sql server tde-key set only accept key ids on the +# public Key Vault domains, which rejects the key ids the LocalStack emulator issues. +SQL_SERVER_ID=$(az sql server show \ + --name "$SQL_SERVER_NAME" \ + --resource-group "$RESOURCE_GROUP_NAME" \ + --query "id" \ + --output tsv) +SERVER_KEY_NAME="${KEY_VAULT_NAME}_${TDE_KEY_NAME}_${TDE_KEY_ID##*/}" + +echo "Adding key [$TDE_KEY_NAME] to the [$SQL_SERVER_NAME] sql server..." +az resource create \ + --id "$SQL_SERVER_ID/keys/$SERVER_KEY_NAME" \ + --api-version 2023-08-01 \ + --properties "{\"serverKeyType\": \"AzureKeyVault\", \"uri\": \"$TDE_KEY_ID\"}" \ + --only-show-errors 1>/dev/null + +if [ $? -eq 0 ]; then + echo "Key [$TDE_KEY_NAME] added successfully to the [$SQL_SERVER_NAME] sql server." +else + echo "Failed to add key [$TDE_KEY_NAME] to the [$SQL_SERVER_NAME] sql server." + exit 1 +fi + +echo "Setting key [$TDE_KEY_NAME] as the TDE protector of the [$SQL_SERVER_NAME] sql server..." +az resource create \ + --id "$SQL_SERVER_ID/encryptionProtector/current" \ + --api-version 2023-08-01 \ + --properties "{\"serverKeyType\": \"AzureKeyVault\", \"serverKeyName\": \"$SERVER_KEY_NAME\", \"autoRotationEnabled\": true}" \ + --only-show-errors 1>/dev/null + +if [ $? -eq 0 ]; then + echo "Key [$TDE_KEY_NAME] set successfully as the TDE protector of the [$SQL_SERVER_NAME] sql server." +else + echo "Failed to set key [$TDE_KEY_NAME] as the TDE protector of the [$SQL_SERVER_NAME] sql server." + exit 1 +fi + # Get Key Vault URI echo "Retrieving Key Vault URI..." KEYVAULT_URI=$(az keyvault show \ diff --git a/samples/web-app-sql-database/python/scripts/validate.sh b/samples/web-app-sql-database/python/scripts/validate.sh index 19d5997..c6ca9bf 100755 --- a/samples/web-app-sql-database/python/scripts/validate.sh +++ b/samples/web-app-sql-database/python/scripts/validate.sh @@ -9,6 +9,7 @@ SQL_DATABASE_NAME='PlannerDB' WEB_APP_NAME="${PREFIX}-webapp-${SUFFIX}" KEY_VAULT_NAME="${PREFIX}-kv-${SUFFIX}" SECRET_NAME="${PREFIX}-secret-${SUFFIX}" +TDE_KEY_NAME="${PREFIX}-tde-key-${SUFFIX}" # Check resource group echo -e "[$RESOURCE_GROUP_NAME] resource group:\n" @@ -39,6 +40,44 @@ az sql db show \ --resource-group "$RESOURCE_GROUP_NAME" \ --output table +# Check that the Key Vault key is the TDE protector of the Azure SQL Server +echo -e "\n[$SQL_SERVER_NAME] SQL server TDE protector:\n" +# Read the key through Azure Resource Manager: the Bicep variant grants the caller no Key Vault data-plane access. +KEY_VAULT_ID=$(az keyvault show \ +--name "$KEY_VAULT_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--query "id" \ +--output tsv) +TDE_KEY_ID=$(az resource show \ +--ids "$KEY_VAULT_ID/keys/$TDE_KEY_NAME" \ +--api-version 2024-11-01 \ +--query "properties.keyUriWithVersion" \ +--output tsv) +TDE_PROTECTOR=$(az sql server tde-key show \ +--server "$SQL_SERVER_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--output json) +echo "$TDE_PROTECTOR" | jq '{serverKeyType, uri, autoRotationEnabled}' +if [[ "$(jq -r .serverKeyType <<< "$TDE_PROTECTOR")" != "AzureKeyVault" || + "$(jq -r .uri <<< "$TDE_PROTECTOR")" != "$TDE_KEY_ID" || + "$(jq -r .autoRotationEnabled <<< "$TDE_PROTECTOR")" != "true" ]]; then + echo "The TDE protector of [$SQL_SERVER_NAME] is not the auto-rotated Key Vault key [$TDE_KEY_ID]" + exit 1 +fi + +# Check that TDE is enabled on the Azure SQL Database +TDE_STATE=$(az sql db tde show \ +--database "$SQL_DATABASE_NAME" \ +--server "$SQL_SERVER_NAME" \ +--resource-group "$RESOURCE_GROUP_NAME" \ +--query "state" \ +--output tsv) +echo -e "\n[$SQL_DATABASE_NAME] SQL database TDE state: [$TDE_STATE]" +if [[ "$TDE_STATE" != "Enabled" ]]; then + echo "TDE is not enabled on [$SQL_DATABASE_NAME]" + exit 1 +fi + # Check Azure Key Vault echo -e "\n[$KEY_VAULT_NAME] Key Vault:\n" az keyvault show \ diff --git a/samples/web-app-sql-database/python/terraform/README.md b/samples/web-app-sql-database/python/terraform/README.md index 082ff14..8b7aed6 100644 --- a/samples/web-app-sql-database/python/terraform/README.md +++ b/samples/web-app-sql-database/python/terraform/README.md @@ -40,8 +40,9 @@ The [main.tf](main.tf) Terraform module creates the following Azure resources: 3. [Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/): The `PlannerDB` database storing relational vacation activity data. 4. [Azure App Service Plan](https://learn.microsoft.com/en-us/azure/app-service/overview-hosting-plans): The compute resource that hosts the web application. 5. [Azure Web App](https://learn.microsoft.com/en-us/azure/app-service/overview): Hosts the Python Flask single-page application (*Vacation Planner*), connected to Azure SQL Database. -6. [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview): Stores the SQL connection string as a secret and a self-signed certificate for HTTPS. +6. [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview): Stores the SQL connection string as a secret, a self-signed certificate for HTTPS, and the RSA key that serves as the [TDE protector](https://learn.microsoft.com/en-us/azure/azure-sql/database/transparent-data-encryption-byok-overview) of the SQL server. 7. [App Service Source Control](https://learn.microsoft.com/en-us/rest/api/appservice/web-apps/create-or-update-source-control?view=rest-appservice-2024-11-01): (Optional) Configures automatic deployment from a public GitHub repository. +8. [User-Assigned Managed Identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview): The identity the SQL server uses to reach its TDE protector key in Key Vault. The system implements a Vacation Planner web application that stores and retrieves activity data from Azure SQL Database. For more information, see [Azure Web App with Azure SQL Database and Azure Key Vault](../README.md). @@ -144,6 +145,12 @@ az sql db show \ --server local-sqlserver-test \ --resource-group local-rg \ --output table + +# Check the TDE protector of the Azure SQL Server +az sql server tde-key show \ +--server local-sqlserver-test \ +--resource-group local-rg \ +--output table ``` ## Cleanup diff --git a/samples/web-app-sql-database/python/terraform/main.tf b/samples/web-app-sql-database/python/terraform/main.tf index 1a5a296..9e5a04e 100644 --- a/samples/web-app-sql-database/python/terraform/main.tf +++ b/samples/web-app-sql-database/python/terraform/main.tf @@ -6,6 +6,8 @@ locals { app_service_plan_name = "${var.prefix}-app-service-plan-${var.suffix}" web_app_name = "${var.prefix}-webapp-${var.suffix}" key_vault_name = "${var.prefix}-kv-${var.suffix}" + sql_identity_name = "${var.prefix}-tde-identity-${var.suffix}" + tde_key_name = "${var.prefix}-tde-key-${var.suffix}" } # Retrieve the current Azure client configuration @@ -18,6 +20,20 @@ resource "azurerm_resource_group" "example" { tags = var.tags } +# Create the user-assigned managed identity the SQL server uses to reach the TDE protector key +resource "azurerm_user_assigned_identity" "sql_server" { + name = local.sql_identity_name + resource_group_name = azurerm_resource_group.example.name + location = azurerm_resource_group.example.location + tags = var.tags + + lifecycle { + ignore_changes = [ + tags + ] + } +} + # Create a SQL server resource "azurerm_mssql_server" "example" { name = local.sql_server_name @@ -29,11 +45,19 @@ resource "azurerm_mssql_server" "example" { public_network_access_enabled = var.public_network_access_enabled outbound_network_restriction_enabled = var.outbound_network_restriction_enabled version = var.sql_version + primary_user_assigned_identity_id = azurerm_user_assigned_identity.sql_server.id tags = var.tags + identity { + type = "UserAssigned" + identity_ids = [azurerm_user_assigned_identity.sql_server.id] + } + + # The TDE protector is managed by azurerm_mssql_server_transparent_data_encryption below lifecycle { ignore_changes = [ - tags + tags, + transparent_data_encryption_key_vault_key_id ] } } @@ -138,6 +162,7 @@ resource "azurerm_key_vault" "example" { sku_name = "standard" rbac_authorization_enabled = false soft_delete_retention_days = 7 + purge_protection_enabled = true tags = var.tags lifecycle { @@ -163,11 +188,72 @@ resource "azurerm_key_vault_access_policy" "web_app" { ] } +# Grant the identity running Terraform access to manage the Key Vault key, secret and certificate +resource "azurerm_key_vault_access_policy" "deployer" { + key_vault_id = azurerm_key_vault.example.id + tenant_id = data.azurerm_client_config.current.tenant_id + object_id = data.azurerm_client_config.current.object_id + + key_permissions = [ + "Create", + "Delete", + "Get", + "GetRotationPolicy", + ] + + secret_permissions = [ + "Delete", + "Get", + "Set", + ] + + certificate_permissions = [ + "Create", + "Delete", + "Get", + ] +} + +# Grant the SQL server managed identity access to the TDE protector key +resource "azurerm_key_vault_access_policy" "sql_server" { + key_vault_id = azurerm_key_vault.example.id + tenant_id = data.azurerm_client_config.current.tenant_id + object_id = azurerm_user_assigned_identity.sql_server.principal_id + + key_permissions = [ + "Get", + "UnwrapKey", + "WrapKey", + ] +} + +# Create the RSA key that protects the database encryption keys of the SQL server +resource "azurerm_key_vault_key" "tde" { + name = local.tde_key_name + key_vault_id = azurerm_key_vault.example.id + key_type = "RSA" + key_size = 2048 + key_opts = ["unwrapKey", "wrapKey"] + + depends_on = [azurerm_key_vault_access_policy.deployer] +} + +# Make the Key Vault key the TDE protector of the SQL server +resource "azurerm_mssql_server_transparent_data_encryption" "example" { + server_id = azurerm_mssql_server.example.id + key_vault_key_id = azurerm_key_vault_key.tde.id + auto_rotation_enabled = true + + depends_on = [azurerm_key_vault_access_policy.sql_server] +} + # Create a Key Vault secret for SQL connection string resource "azurerm_key_vault_secret" "sql_connection_string" { name = var.secret_name value = "Server=tcp:${azurerm_mssql_server.example.fully_qualified_domain_name},1433;Database=${azurerm_mssql_database.example.name};User ID=${var.sql_database_username};Password=${var.sql_database_password};Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;" key_vault_id = azurerm_key_vault.example.id + + depends_on = [azurerm_key_vault_access_policy.deployer] } # Create a self-signed certificate in Key Vault @@ -201,4 +287,6 @@ resource "azurerm_key_vault_certificate" "example" { ] } } + + depends_on = [azurerm_key_vault_access_policy.deployer] }